Scalable cache coherency protocol

Abstract

A scalable cache coherency protocol is described for a system comprising multiple coherent agents coupled to one or more memory controllers. The memory controllers may implement an accurate directory of cache blocks from the memory to which the memory controllers are coupled. Multiple requests for cache blocks may be outstanding, and snoops and completions for requests may include expected cache states at the receiving agent so that the receiving agent can detect race conditions as indicated in the memory controller's directory when the request is processed. In one embodiment, cache states may include a primary shared state and a secondary shared state. The primary shared state may be applied to a coherent agent responsible for transmitting a copy of the cache block to the requesting agent. In one embodiment, at least two types of snoops, namely snoop forward and snoop back, may be supported.

Description

Background Technology

[0001] Technology field

[0002] The embodiments described in this specification relate to cache-coherent systems.

[0003] Background Technology

[0004] Various systems may include multiple agents accessing memory during use. If one or more agents cache data, it may be possible for another agent to have cached (and possibly modified) data while one agent is reading data from memory. Many systems implement cache coherence mechanisms to resolve data in memory and cached copies of data, providing a memory ordering model to order accesses to the same data and pass data to agents for caching.

[0005] Additional communications between the coherence controller, various agents, and memory are typically used to maintain cache coherence. As the number of agents increases, the number of these messages also increases and can overwhelm the interconnections between the agents and memory. Even if sufficient bandwidth exists to deliver messages and read / write traffic to memory, the number of messages can degrade system performance by increasing memory latency. Brief explanation of the drawing

[0006] The following detailed description refers to the attached drawings, which are now briefly explained. FIG. 1 is a block diagram of one embodiment of a cache coherent system implemented as a system-on-chip (SOC). FIG. 2 is a block diagram illustrating one embodiment of a three-hop protocol for coherent delivery of cache blocks. FIG. 3 is a block diagram illustrating an example of managing competition between a fill for one coherent transaction and a snoop for another coherent transaction. FIG. 4 is a block diagram illustrating an embodiment of managing a competition between a snoop on one coherent transaction and an acknowledgment on another coherent transaction. FIG. 5 is a block diagram of a part of one embodiment of a coherent agent. FIG. 6 is a flowchart illustrating the operation of one embodiment of processing a request in a coherence controller. FIG. 7 is a flowchart illustrating the operation of one embodiment of a coherent agent that transmits a request to a memory controller to process completions related to the request. FIG. 8 is a flowchart illustrating the operation of one embodiment of a coherent agent receiving a snoop. FIG. 9 is a block diagram illustrating a chain of conflicting requests for a cache block according to one embodiment. FIG. 10 is a flowchart illustrating one embodiment of a coherent agent that absorbs snoop. FIG. 11 is a block diagram illustrating one example of a non-cacheable request. FIG. 12 is a flowchart illustrating the operation of one embodiment of a coherence controller for generating snoops based on the cacheable and non-caching attributes of requests. FIG. 13 is a table illustrating multiple cache states according to one embodiment of a coherent protocol. FIG. 14 is a table illustrating a plurality of messages that can be used in one embodiment of a coherence protocol. FIG. 15 is a flowchart illustrating the operation of one embodiment of a coherence controller for processing a change to exclusive conditional request. FIG. 16 is a flowchart illustrating the operation of one embodiment of a coherence controller for reading directory entries and generating snoops. FIG. 17 is a flowchart illustrating the operation of one embodiment of a coherence controller for processing an exclusive no data request. FIG. 18 is a block diagram of one embodiment of a system including the SOC exemplified in FIG. 1. FIG. 19 is a block diagram of one embodiment of a computer-accessible storage medium. While the embodiments described in the present invention may allow for various modifications and alternative forms, specific embodiments thereof are illustrated by way of example in the drawings and will be described in detail herein. However, it should be understood that the drawings and detailed description thereof are not intended to limit the embodiments to the specific forms disclosed, but on the contrary, the intention is to cover all modifications, equivalents, and alternatives that fall within the spirit and scope of the appended claims. The headings used herein are for configuration purposes only and are not intended to be used to limit the scope of the description. Specific details for implementing the invention

[0007] A scalable cache coherence protocol is described for a system comprising multiple coherent agents coupled to one or more memory controllers. A coherent agent may generally include a cache for caching memory data or otherwise may take ownership of one or more cache blocks and may include any circuitry capable of potentially modifying cache blocks locally. Coherent agents participate in the cache coherence protocol to ensure that modifications made by one coherent agent are visible to other agents subsequently reading the same data, and to ensure that modifications made by two or more coherent agents in a specific order (as determined at a system ordering point, such as a memory controller for the memory storing the cache blocks) are observed in that order by each of the coherent agents.

[0008] The cache coherence protocol may specify a set of messages or commands that may be transmitted between agents and memory controllers (or coherence controllers within memory controllers) to complete coherent transactions. Messages may include requests, snoops, snoop responses, and completions. A “request” is a message that initiates a transaction and specifies the requested cache block (e.g., having the address of the cache block) and the state in which the requestor will receive the cache block (or minimum state; in some cases, a more permissive state may be provided). As used herein, “snoop” or “snoop message” refers to a message transmitted to a coherent agent to request a change in state within a cache block, and may also request that the cache block be provided by the coherent agent if the coherent agent holds an exclusive copy of the cache block or is otherwise responsible for the cache block. "Completion" or "snoop response" may be a message from a coherent agent indicating that a state change has occurred and, where applicable, providing a copy of the cache block. In some cases, completion may also be provided by the source of the request for specific requests.

[0009] "State" or "cache state" generally refers to a value indicating whether a copy of a cache block is valid in the cache, and may also indicate other attributes of the cache block. For example, the state may indicate whether the cache block is modified relative to its copy in memory. The state may indicate the ownership level of the cache block (e.g., whether the agent holding the cache is allowed to modify the cache block, whether the agent is responsible for providing the cache block, or whether the agent is responsible for returning the cache block to the memory controller if it is evicted from the cache, etc.). The state may also indicate the possible existence of the cache block in other coherent agents (e.g., a "shared" state may indicate that a copy of the cache block can be stored in one or more other caching agents).

[0010] Various features may be included in various embodiments of the cache coherence protocol. For example, memory controller(s) may implement a directory for cache blocks corresponding to the coherence controller and the memory controlled by the memory controller. The directory tracks the states of cache blocks across multiple caching agents, allowing the coherence controller to determine which caching agents to snoop on in order to change the state of a cache block and possibly provide a copy of the cache block. That is, snoops do not need to be broadcast to all caching agents based on a request received by the cache controller; rather, snoops may be sent to those agents that possess a copy of the cache block affected by the request. Once snoops are generated, the directory may be updated to reflect the state of the cache block in each coherent agent after the snoops are processed and the data is provided to the source of the request. Thus, the directory can be accurate for the next request processed with the same cache block. For example, compared to a broadcast solution, traffic on the interconnection between coherent agents and the memory controller can be reduced, thereby minimizing snoops. In one embodiment, a "3-hop" protocol may be supported in which one of the caching coherent agents provides a copy of the cache block to the source of the request, or if there is no caching agent, the memory controller provides the copy. Thus, data is provided in three "hops" (or messages transmitted through the interface): a request from the source to the memory controller, a snoop to the coherent agent responding to the request, and completion of the data in the cache block from the coherent agent to the source of the request. In cases where there is no cached copy, there may be two hops: a request from the source to the memory controller and completion of the data from the memory controller to the source.There may be additional messages (e.g., completions from other agents indicating that the requested state change has been made when there are multiple snoops on the request), but the data itself may be provided within three hops. In contrast, many cache coherence protocols are four-hop protocols, where the coherent agent responds to the snoop by returning the cache block to the memory controller, and the memory controller forwards the cache block to the source. In one embodiment, four-hop flows may be supported by the protocol in addition to three-hop flows.

[0011] In one embodiment, requests for a cache block may be processed by a coherence controller, and the directory may be updated when snoops (and / or completions from the memory controller if there is no cached copy) are generated. Subsequently, other requests for the same cache block may be serviced. Thus, as in some other cache coherence protocols, requests for the same cache block may not be serialized. Various race conditions may occur when there are many outstanding requests for a cache block, because messages related to subsequent requests may reach a given coherent agent before messages related to previous requests (where "subsequent" and "previous" refer to requests ordered by the coherence controller of the memory controller). To allow agents to sort requests, messages (e.g., snoops and completions) may include the expected cache state at the receiving agent, as indicated by the directory when the request is processed. Accordingly, if the receiving agent does not have a cache block of the state indicated in the message, the receiving agent may delay the processing of the message until the cache state changes to the expected state. The change to the expected state may occur through messages related to the previous request. Further descriptions of race conditions and the use of the expected cache state to resolve them are provided below with respect to FIGS. 3 and 4, and FIGS. 6 through 8.

[0012] In one embodiment, cache states may include a primary shared state and a secondary shared state. The primary shared state may apply to a coherent agent responsible for transmitting a copy of a cache block to a requesting agent. Secondary shared agents may not even need to be snooped during the processing of a given request (e.g., a read for a cache block allowed to return to a shared state). Further details regarding the primary and secondary shared states will be described in connection with FIGS. 14 and FIGS. 16.

[0013] In one embodiment, at least two types of snoops, namely snoop forward and snoop back, may be supported. Snoop forward messages may be used to cause a coherent agent to forward a cache block to a requesting agent, while snoop back messages may be used to cause a coherent agent to return a cache block to a memory controller. In one embodiment, snoop invalidation messages may also be supported (and may include forward and back variations to specify destinations for completions). Snoop invalidation messages cause a cache coherent agent to invalidate a cache block. Supporting snoop forward and snoop back flows may provide, for example, both cacheable (snoop forward) behavior and non-caching (snoop back) behavior. Snoop forwards can be used to minimize the number of messages when a cache block is provided to a caching agent, because the caching agent can store the cache block and potentially use the data within it. Meanwhile, a non-coherent agent may not store the entire cache block, and thus sending a copy back to memory can ensure that the entire cache block is captured by the memory controller. Accordingly, snoop forward and snoop back variations or types may be selected based on the capabilities of the requesting agent (e.g., based on the identity of the requesting agent) and / or based on the type of request (e.g., a cacheable type or a non-caching type). Further details regarding snoop forward and snoop back messages are provided below in relation to FIGS. 11, 12, and 14. Various other features are exemplified in the remaining figures and will be described in more detail below.

[0014] FIG. 1 is a block diagram of an embodiment of a system comprising a system-on-chip (SOC) (10) coupled to one or more memories, such as memories (12A to 12m). The SOC (10) may include a plurality of coherent agents (CAs) (14A to 14n). The coherent agents may include one or more processors (P) (16) that combine one or more caches (e.g., cache (18)). The SOC (10) may include one or more non-coherent agents (NCAs) (20A to 20p). The SOC (10) may include one or more memory controllers (22A to 22m) that are each coupled to their respective memories (12A to 12m) during use. Each memory controller (22A to 22m) may include a coherence controller circuit (24) (more briefly "coherence controller", or "CC") coupled to a directory (26). The memory controllers (22A to 22m), non-coherent agents (20A to 20p), and coherent agents (14A to 14n) may be coupled to an interconnection section (28) to communicate between the various components (22A to 22m, 20A to 20p, 14A to 14n). As indicated by the name, the components of the SOC (10) may be integrated onto a single integrated circuit "chip" in one embodiment. In other embodiments, the various components may be located on the SOC (10) on different chips or otherwise outside of separate components. Any amount of integrated or separate components may be used. In one embodiment, a subset of coherent agents (14A to 14n) and memory controllers (22A to 22m) may be implemented in one of a plurality of integrated circuit chips combined together to form the components illustrated in the SOC (10) of FIG. 1.

[0015] The coherence controller (24) can implement the memory controller portion of the cache coherence protocol. Generally, the coherence controller (24) may be configured to receive requests from the interconnection portion (28) targeted in cache blocks mapped to memory (12A to 12m) to which the memory controllers (22A to 22m) are coupled (e.g., through one or more unillustrated queues within the memory controllers (22A to 22m)). The directory may contain multiple entries, each of which may track the coherence status of its respective cache block within the system. The coherence status may include, for example, the cache status of a cache block within various coherent agents (14A to 14N) (e.g., within caches (18) or other caches (not illustrated), such as caches within processors (16)). Accordingly, based on the directory entry for the cache block corresponding to the given request and the type of the given request, the coherence controller (24) may be configured to determine which coherent agents (14A to 14n) will receive snoops and the type of snoop (e.g., snoop invalidation, snoop sharing, change to shared state, change to owned state, change to invalid state, etc.). The coherence controller (24) may also independently determine whether a snoop forward or a snoop back will be transmitted. The coherent agents (14A to 14n) may receive snoops, process the snoops to update the cache block state in the coherent agents (14A to 14n), and provide a copy of the cache block (if specified by the snoop) to the requesting coherent agent (14A to 14n) or memory controller (22A to 22m) that transmitted the snoop. Further details will be provided below.

[0016] As described above, coherent agents (14A to 14n) may include one or more processors (16). The processors (16) may serve as the central processing unit (CPU) of the SOC (10). The CPU of the system includes processor(s) that execute the system's main control software, such as an operating system. Generally, the software executed by the CPU during use can control other components of the system to realize the desired functions of the system. The processors may also execute other software, such as application programs. Application programs may provide user functions and may rely on the operating system for low-level device control, scheduling, memory management, etc. Therefore, the processors may also be referred to as application processors. Coherent agents (14A to 14n) may further include interfaces to other hardware and / or other components of the system, such as a cache (18) (e.g., an interface to an interconnection unit (28)). Other coherent agents may include processors other than CPUs. Additionally, other coherent agents may not include processors (for example, fixed-function circuits such as display controllers or other peripheral circuits, embedded processors, or fixed-function circuits with processor assistance through processors may be coherent agents).

[0017] Generally, a processor may include any circuitry and / or microcode configured to execute instructions defined in an instruction set architecture implemented by the processor. Processors may encompass processor cores implemented on an integrated circuit together with other components as a system-on-chip (SOC (10)) or other levels of integration. Processors may further include separate microprocessors, processor cores and / or microprocessors integrated into multi-chip module implementations, processors implemented as multiple integrated circuits, etc. The number of processors (16) in a given coherent agent (14A to 14n) may differ from the number of processors (16) in other coherent agents (14A to 14n). Generally, one or more processors may be included. Additionally, the processors (16) may differ in microarchitecture implementation, performance and power characteristics, etc. In some cases, processors may also differ in the instruction set architecture they implement and their functions (e.g., CPUs, graphics processing unit (GPU) processors, microcontrollers, digital signal processors, image signal processors, etc.).

[0018] The caches (18) may have any capacity and configuration, such as set associative, direct mapping, or full associative. The cache block size may be any desired size (e.g., 32 bytes, 64 bytes, 128 bytes, etc.). A cache block may be a unit of allocation and deallocation within the cache (18). Additionally, a cache block may be a unit in which coherence is maintained in this embodiment (e.g., a segment of aligned coherence-granule size in memory address space). A cache block may also be referred to as a cache line in some cases.

[0019] In addition to the coherence controller (24) and directory (26), memory controllers (22A to 22m) may generally include circuitry for receiving memory operations from other components of the SOC (10) and accessing the memories (12A to 12m) to complete the memory operations. The memory controllers (22A to 22m) may be configured to access any type of memory (12A to 12m). For example, the memories (12A to 12m) may be static random access memory (SRAM), dual data rate (DDR, DDR2, DDR3, DDR4, etc.) DRAM, non-volatile memories, graphics DRAM such as graphics DDR DRAM (GDDR), and synchronous RAM (DRAM) such as high bandwidth memory (HBM). Low-power / mobile versions of DDR DRAM (e.g., LPDDR, mDDR, etc.) may be supported. Memory controllers (22A to 22m) may include queues for memory operations for ordering (and potentially reordering) operations and for presenting operations to the memories (12A to 12m). Memory controllers (22A to 22m) may further include data buffers for storing write data waiting to be written to memory and read data waiting to be returned to the source of the memory operation (when no data is provided in the snoop). In some embodiments, memory controllers (22A to 22m) may include a memory cache for storing recently accessed memory data. In SOC implementations, for example, the memory cache may reduce power consumption in the SOC by avoiding re-access of data from the memories (12A to 12m) when it is expected to be accessed again soon.In some cases, the memory cache may also be referred to as a system cache in contrast to private caches that serve only certain components, such as the cache (18) within the processors (16) or the caches. Additionally, in some embodiments, the system cache does not need to be located within the memory controllers (22A to 22m).

[0020] Non-coherent agents (20A through 20p) may generally include various additional hardware functions included in the SOC (10) (e.g., “peripherals”). For example, peripherals may include video peripherals, such as image signal processors configured to process image capture data from a camera or other image sensor, GPUs, video encoders / decoders, scalers, rotators, blenders, etc. Peripherals may include audio peripherals, such as microphones, speakers, interfaces for microphones and speakers, audio processors, digital signal processors, mixers, etc. Peripherals may include interface controllers for various interfaces outside the SOC (10), such as USB (Universal Serial Bus), PCI, serial and parallel ports including PCIe (peripheral component interconnect (PCI) Express), etc. Peripherals may include networking peripherals such as Media Access Controllers (MACs). Any set of hardware may be included. Non-coherent agents (20A to 20p) may also include bridges to a set of peripheral devices in one embodiment.

[0021] The interconnection section (28) may be any communication interconnection section and protocol for communication between components of the SOC (10). The interconnection section (28) may be bus-based, including shared bus configurations, crossbar configurations, and hierarchical buses using bridges. The interconnection section (28) may also be packet-based or circuit-switching, and may be a hierarchical structure, crossbar, point-to-point, or other interconnection sections having bridges. In one embodiment, the interconnection section (28) may include a plurality of independent communication fabrics.

[0022] Generally, the number of each component (22A to 22m, 20A to 20p, 14A to 14n) may vary from embodiment to embodiment, and any number may be used. As indicated by the suffixes "m", "p", and "n", the number of components of one type may differ from the number of components of another type. However, the number of a given type may be the same as the number of another type. Additionally, although the system of FIG. 1 is illustrated with a plurality of memory controllers (22A to 22m), embodiments having a single memory controller (22A to 22m) are also considered and may implement the cache coherence protocol described herein.

[0023] Next, referring to FIG. 2, a block diagram illustrating a plurality of coherent agents (12A to 12D) and a memory controller (22A) performing a coherent transaction for a cacheable read exclusive request (CRdEx) according to one embodiment of a scalable cache coherence protocol is shown. Since a read exclusive request may be a request for an exclusive copy of a cache block, any other copies are invalidated by the coherent agents (14A to 14D), and when the transaction is completed, the requester has a unique valid copy. A memory (12A to 12m) having memory locations allocated to a cache block contains data at the location allocated to the cache block within the memory (12A to 12m), but if the requester modifies the data, that data will also become "stale." A read exclusive request may be used, for example, to enable the requester to modify a cache block without sending an additional request in the cache coherence protocol. If an exclusive copy is not required, other requests may be used (e.g., if a writable copy is not strictly necessary for the requester, a read share request (CRdSh) may be used). In the label "CRdEx," "C" may denote "caching possible." Other transactions may be issued by non-coherent agents (e.g., agents of FIG. 1 (20A through 20p)), and these transactions may be labeled "NC" (e.g., NCRd). Further discussion of request types and other messages in transactions is provided below in relation to FIG. 14 for one embodiment, and further discussion of cache states is provided below in relation to FIG. 13 for one embodiment.

[0024] In the example of FIG. 2, the coherent agent (14A) can initiate a transaction by sending a read-exclusive request to a memory controller (22A) (which controls memory locations assigned to addresses in the read-exclusive request). The memory controller (22A) (and more specifically, the coherence controller (24) of the memory controller (22A)) can read an entry in the directory (26) and determine that the coherent agent (14D) has a cache block of primary shared state (P) and thus may be a coherent agent that provides the cache block to the requesting coherent agent (14D). The coherence controller (24) can generate a snoop-forward (SnpFwd[st]) message for the coherent agent (14D) and can issue the snoop-forward message to the coherent agent (14D). The coherence controller (24) may include an identifier of the current state within the coherent agent receiving the snoop according to the directory (26). For example, in this case, the current state is "P" in the coherent agent (14D) according to the directory (26). Based on the snoop, the coherent agent (14D) may access the cache storing the cache block and generate a fill completion (Fill in FIG. 2) with data corresponding to the cache block. The coherent agent (14D) may transmit the fill completion to the coherent agent (14A). Thus, the system implements a "3-hop" protocol for delivering data to the requester: CRdEx, SnpFwd[st], and Fill. As indicated by "[st]" in the SnpFwd[st] message, the snoop forward message may also be coded into the state of the cache block to which the coherent agent will switch after processing the snoop. In various embodiments, there may be different variations of the message, or the state may be conveyed as a field within the message.In the example of Fig. 2, because the request is a read-exclusive request, the new state of the cache block within the coherent agent may be invalid. Other requests may allow the new shared state.

[0025] Additionally, the coherence controller (24) can determine from a directory entry for the cache block that the coherent agents (14B, 14C) have the cache block in a secondary shared state (S). Accordingly, snoops can be issued to each coherent agent that (i) has a cached copy of the cache block; and (ii) changes the state of the block within the coherent agent based on a transaction. Since the coherent agent (14A) has obtained an exclusive copy, the shared copies must be invalidated, and thus, the coherence controller (24) can generate Snoop Invalidate (SnpInvFw) messages for the coherent agents (14B, 14C) and issue snoops to the coherent agents (14B, 14C). Snoop invalidation messages include identifiers indicating that the current state within the coherent agents (14B, 14C) is 'shared'. The coherent agents (14B, 14C) can process snoop invalidation requests and provide acknowledgment (Ack) completions to the coherent agent (14A). Note that in the illustrated protocol, messages from the snooping agents to the coherence controller (24) are not implemented in this embodiment. The coherence controller (24) can update directory entries based on the issuance of snoops and process the next transaction. Thus, as previously mentioned, transactions for the same cache block may not be serialized in this embodiment. The coherence controller (24) can initiate additional transactions for the same cache block and can rely on the current state indications within the snoops to identify which snoops belong to which transactions (for example, the next transaction for the same cache block will detect cache states corresponding to the previous completed transaction).In the illustrated embodiment, the snoop nullification message is the SnpInvFw message because the completion is transmitted to the initiating coherent agent (14A) as part of the 3-hop protocol. In one embodiment, a 4-hop protocol is also supported for certain agents. In this embodiment, the SnpInvBk message may be used to indicate that the snooping agent will transmit the completion back to the coherence controller (24).

[0026] Accordingly, the cache state identifiers of the snoops can enable coherent agents to resolve competition between messages forming different transactions for the same cache block. That is, messages may be received out of order from the order in which corresponding requests were processed by the coherence controller. The order in which the coherence controller (24) processes requests for the same cache block through the directory (26) may define the order of the requests. That is, the coherence controller (24) may be the ordering point for transactions received from a given memory controller (22A to 22m). Meanwhile, the serialization of messages may be managed by the coherent agents (14A to 14n) based on the current cache state corresponding to each message and the cache state within the coherent agents (14A to 14n). A given coherent agent may access cache blocks within the coherent agent based on a snoop and may be configured to compare the cache state specified by the snoop with the current cache state within the cache. If the states do not match, the snoop belongs to a transaction ordered after another transaction, which changes the agent's cache state to the state specified by the snoop. Accordingly, the snooping agent may be configured to delay the processing of the snoop based on the fact that the first state does not match the second state until the second state is changed to the first state in response to a different communication related to a request different from the first request. For example, the state may be changed based on fill completion received by the snooping agent from a different transaction, etc.

[0027] In one embodiment, the snoops may include a completion count (Cnt) representing the number of completions corresponding to a transaction, so that the requester can determine when all completions related to the transaction have been received. The coherence controller (24) may determine the completion count based on the states indicated in the directory entry for the cache block. The completion count may be, for example, the number of completions minus 1 (for example, 2 in the example of FIG. 2 since there are 3 completions). This implementation may allow the completion count to be used as an initialization for the completion counter for the transaction when the initial completion for the transaction is received by the request agent (for example, already decremented to reflect the reception of a completion that already conveys the completion count). Once the count is initialized, additional completions for the transaction may cause the request agent to update the completion count (for example, decrementing the counter). In other embodiments, the actual completion count may be provided and may be decremented by the requester to initialize the completion count. Generally, the completion count can be an arbitrary value that identifies the number of completions the requester will observe before the transaction is fully completed. In other words, the request agent can complete the request based on the completion count.

[0028] FIGS. 3 and 4 illustrate exemplary race conditions that may occur with transactions for the same cache block, and the use of the current cache state for a given agent (also referred to as "expected cache state") as reflected in the directory when a transaction is processed in the memory controller, and the current cache state within a given agent (as reflected, for example, in the cache(s) or buffers of the given agent capable of temporarily storing cache data). In FIGS. 3 and 4, coherent agents are enumerated as CA0 and CA1, and the memory controller associated with the cache block is illustrated as MC. Vertical lines (30, 32, 34) for CA0, CA1, and MC illustrate the source (base of the arrow) and destination (head of the arrow) of various messages corresponding to transactions. In FIGS. 3 and 4, time progresses from top to bottom. A memory controller may be associated with a cache block if the memory to which the memory controller is coupled includes memory locations assigned to the address of the cache block.

[0029] FIG. 3 illustrates a race condition between a fill completion for one transaction and a snoop for different transactions on the same cache block. In the example of FIG. 3, CA0 initiates a read-exclusive transaction with a CRdEx request to MC (arrow (36)). CA1 also initiates a read-exclusive transaction with a CRdEx request (arrow (38)). The CA0 transaction is processed by MC first, establishing the CA0 transaction ordered ahead of the CA1 request. In this example, the directory indicates that there are no cached copies of the cache block in the system, and therefore MC responds to the CA0 request with a fill in an exclusive state (FillE, arrow (40)). MC updates the directory entry of the cache block with an exclusive state for CA0.

[0030] MC selects a CRdEx respect from CA1 for processing and detects that CA0 has a cache block in an exclusive state. Therefore, MC can generate a snoop-forward request for CA0 to request that CA0 invalidate the cache block within its cache(s) and provide the cache block to CA1 (SnpFwdI). The snoop-forward request also includes an identifier of the E state for CA0's cache block, as this is the cache state reflected in the directory for CA0. MC can issue a snoop (arrow (42)) and update the directory to indicate that CA1 has an exclusive copy and CA0 no longer has a valid copy.

[0031] Snoop and fill completion may reach CA0 in either order at the time. Messages may move through different virtual channels and / or other delays within the interconnection allow messages to reach in either order. In the example illustrated, the snoop reaches CA0 before the fill completion. However, because the expected state in the snoop (E) does not match the current state of the cache block in CA0 (I), CA0 may delay the processing of the snoop. Subsequently, the fill completion may reach CA0. CA0 may write the cache block to the cache and set the state to 'exclusive' (E). CA0 may also be allowed to perform at least one operation on the cache block to support the forward progress of the task in CA0, and that operation may change the state to 'modified' (M). In the cache coherence protocol, the directory (26) may not track the state M individually (e.g., it may be treated as E), but may match the state E as the expected state in the snoop. CA0 can issue a fill completion to CA1 in a modified state (FillM, arrow (44)). Thus, the race condition between the snoop and fill completion for the two transactions is correctly handled.

[0032] Although a CRdEx request is issued by CA1 following a CRdEx request from CA0, in the example of FIG. 3, a CRdEx request can be issued by CA1 before a CRdEx request from CA0, and a CRdEx request from CA0 can still be ordered by MC before a CRdEx request from CA1, because MC is the ordering point for transactions.

[0033] FIG. 4 illustrates a race condition between a snoop for one coherent transaction and the completion of another coherent transaction for the same cache block. In FIG. 4, CA0 initiates a write-back transaction (CWB) to write the modified cache block to memory (arrow (46)), but in reality, the cache block may be tracked as 'exclusive' in the directory as previously described. The CWB may be transmitted, for example, when CA0 evicts a cache block from its caches but the cache block is in a modified state. CA1 initiates a read-shared transaction (CRdS) for the same cache block (arrow (48)). The CA1 transaction is ordered ahead of the CA0 transaction by the MC, which reads the directory entry for the cache block and determines that CA0 has the cache block in an exclusive state. The MC issues a snoop-forward request to CA0 and requests a change to a secondary shared state (SnpFwdS, arrow (50)). The identifier in the snoop indicates the current 'exclusive' (E) cache state in CA0. MC updates the directory entry to indicate that CA0 has a cache block in the second shared state and CA1 has a copy in the first shared state (because the previous exclusive copy is provided to CA1).

[0034] MC processes the CWB request from CA0 and reads back the directory entry for the cache block. MC issues an Ack completion, indicating that the current cache state is secondary shared (S) in CA0 by the identifier of the cache state within the Ack completion (arrow (52)). Based on the fact that the expected state 'secondary shared' does not match the current state 'modified', CA0 may delay processing the Ack completion. Processing the Ack completion would allow CA0 to discard the cache block, and thus CA0 would not have a copy of the cache block to provide to CA1 in response to a subsequent SnpFwdS request. When a SnpFwdS request is received, CA0 may provide a fill completion to CA1 (arrow (54)) to provide the cache block in primary shared state (P). CA0 may also change the state of the cache block within CA0 to 'secondary shared' (S). The change in state matches the expected state for Ack completion, and therefore CA0 can invalidate the cache block and complete the CWB transaction.

[0035] FIG. 5 is a more detailed block diagram of a part of one embodiment of coherent agents (14A). Other coherent agents (14B through 14n) may be similar. In the illustrated embodiment, the coherent agent (14A) may include a request control circuit (60) and a request buffer (62). The request buffer (62) is coupled to the request control circuit (60), and both the request buffer (62) and the request control circuit (60) are coupled to a cache (18) and / or processors (16) and an interconnection unit (28).

[0036] The request buffer (62) may be configured to store multiple requests generated by the cache (18) / processors (16) for coherent cache blocks. That is, the request buffer (62) may store requests initiating transactions on the interconnection unit (28). One entry of the request buffer (62) is exemplified in FIG. 5, and other entries may be similar. An entry may include a valid (V) field (63), a request (Req.) field (64), a count valid (CV) field (66), and a completion count (CompCnt) field (68). The valid field (63) may store a valid indicator (e.g., valid bit) indicating whether the entry is valid (e.g., storing an unprocessed request). The request field (64) may store data defining the request (e.g., request type, address of the cache block, tag for the transaction, or other identifier). The count valid field (66) may store a valid indication for the completion count field (68) to indicate that the completion count field (68) has been initialized. When the request control circuit (68) processes a completion received from the interconnect (28) for a request, it may use the count valid field (66) to determine whether the request control circuit (68) initializes the field with the completion count included in the completion (count field invalid) or updates the completion count, such as by decreasing the completion count (count field valid). The completion count field (68) may store the current completion count.

[0037] The request control circuit (60) can receive requests from the cache (18) / processors (16) and can assign request buffer entries in the request buffer (62) to the requests. The request control circuit (60) can track requests in the buffer (62) to cause requests to be transmitted over the interconnect (28) (e.g., according to any classification of arbitration techniques), track received completions in the requests to complete the transaction, and forward the cache block to the cache (18) / processors (16).

[0038] Now, referring to FIG. 6, a flowchart illustrating the operation of one embodiment of a coherence controller (24) within memory controllers (22A to 22m) based on receiving requests to be processed is shown. The operation of FIG. 6 may be performed when a request for service is selected from among the received requests within memory controllers (22A to 22m) through any desired arbitration algorithm. Although the blocks are shown in a specific order for ease of understanding, other orders may also be used. The blocks may be executed in parallel in the combination logic of the coherence controller (24). The blocks, combinations of blocks, and / or the entire flowchart may be pipelined over multiple clock cycles. The coherence controller (24) may be configured to implement the operation shown in FIG. 6.

[0039] The coherence controller (24) may be configured to read directory entries from the directory (26) based on the address of the request. The coherence controller (24) may be configured to determine which snoops will be generated based on the type of request (e.g., the state requested for a cache block by the requester) and the current state of the cache block within the various coherent agents (14A to 14n) as indicated in the directory entry (block (70)). Additionally, the coherence controller (24) may generate a current state to be included in each snoop based on the current state of the coherent agent (14A to 14n) to receive the snoop as indicated in the directory. The coherence controller (24) may be configured to insert the current state into the snoop (block (72)). The coherence controller (24) may also be configured to generate a completion count and insert the completion count into each snoop (block (74)). As previously mentioned, in one embodiment, the completion count may be the number of completions minus 1, or the total number of completions. The number of completions may be the number of snoops, and if the memory controller (22A to 22m) provides a cache block, it may be a fill completion from the memory controller (22A to 22m). In most cases where there is a snoop for a cacheable request, one of the snooped coherent agents (14A to 14n) may provide a cache block, and thus the number of completions may be the number of snoops. However, for example, if the coherent agent (14A to 14n) does not have a copy of the cache block (no snoops), the memory controller may provide a fill completion. The coherence controller (24) may be configured to queue snoops for transmission to coherent agents (14A to 14n) (block (76)).Once the snoops are successfully queued, the coherence controller (24) may be configured to update a directory entry to reflect the completion of the request (block (78)). For example, the updates may change the tracked cache states within the directory entry to match the cache states requested by the snoops, and change the agent identifier indicating the agent that will provide a copy of the cache block to the coherent agent (14A through 14n) that will have the cache block in the exclusive, modified, owned, or primary shared state when the transaction is completed.

[0040] Now, referring to FIG. 7, a flowchart illustrating the operation of one embodiment of a request control circuit (60) in a coherent agent (14A through 14n) based on receiving completion for an unprocessed request in the request buffer (62) is shown. Although the blocks are shown in a specific order for ease of understanding, other orders may also be used. The blocks may be executed in parallel in the combination logic of the request control circuit (60). The blocks, combinations of blocks, and / or the entire flowchart may be pipelined over a number of clock cycles. The request control circuit (60) may be configured to implement the operation shown in FIG. 7.

[0041] The request control circuit (60) may be configured to access a request buffer entry within the request buffer (62) associated with the request to which the received completion is associated. If the completion count is indicated as valid in the count valid field (66) (decision block (80), "yes" leg), the request control circuit (60) may be configured to decrease the count in the request count field (68) (block (82)). If the count is 0 (decision block (84), "yes" leg), the request is completed, and the request control circuit (60) may be configured to forward the indication of completion (and, if applicable, the received cache block) to the cache (18) and / or processors (16) that generated the request (block (86)). The completion may cause the state of the cache block to be updated. If the new state of the cache block after the update matches the expected state in the waiting snoop (decision block (88), "yes" leg), the request control circuit (60) may be configured to process the waiting snoop (block (90)). For example, the request control circuit (60) may be configured to pass the snoop to the cache (18) / processors (16) to generate a completion corresponding to the waiting snoop (and change the state of the cache block as indicated by the snoop).

[0042] If the new state is identical to the expected state, the new state may match the expected state. Additionally, if the expected state is a state tracked by the directory (26) for the new state, the new state may match the expected state. For example, in one embodiment, the modified state is tracked as an exclusive state in the directory (26), and thus the modified state matches the expected state 'exclusive'. The new state may be modified, for example, if the state is provided within a fill completion transmitted by another coherent agent (14A to 14n) that has the cache block 'exclusively' and has locally modified the cache block.

[0043] If the completion count is indicated as valid in the count valid field (66) (decision block (80)) and the completion count is not zero after decrement (decision block (84), "No" leg), the request is not completed and is therefore kept in a waiting state in the request buffer (62) (and any waiting snoop waiting for the request to be completed may be kept in a waiting state). If the completion count is indicated as invalid in the count valid field (66) (decision block (80), "No" leg), the request control circuit (60) may be configured to complete the completion count and initialize the provided completion count field (68) (block (92)). The request control circuit (60) may still be configured to check that the completion count is zero (e.g., if there is only one completion for the request, the completion count may be zero in completion) (decision block (84)), processing may continue as discussed above.

[0044] FIG. 8 is a flowchart illustrating the operation of one embodiment of a coherent agent (14A to 14n) based on receiving a snoop. Although the blocks are shown in a specific order for ease of understanding, other orders may also be used. The blocks may be executed in parallel within the combination logic of the coherent agent (14A to 14n). The blocks, combinations of blocks, and / or the entire flowchart may be pipelined over multiple clock cycles. The coherent agent (14A to 14n) may be configured to implement the operation illustrated in FIG. 8.

[0045] Coherent agents (14A through 14n) may be configured to check the expected state in a snoop against the state in the cache (18) (decision block (100)). If the expected state does not match the current state of the cache block (decision block (100), "No" leg), completion is unprocessed, which will change the current state of the cache block to the expected state. Completion corresponds to a transaction ordered prior to the transaction corresponding to the snoop. Accordingly, coherent agents (14A through 14n) may be configured to hold the snoop, delaying the processing of the snoop until the current state changes to the expected state indicated in the snoop (block (102)). In one embodiment, the waiting snoop may be stored in a buffer specifically provided for waiting snoops. Alternatively, the waiting snoop may be absorbed into an entry of a request buffer (62) storing conflicting requests, as discussed in more detail below in relation to FIG. 10.

[0046] If the expected state matches the current state (decision block (100), "yes" leg), the coherent agents (14A through 14n) may be configured to process a state change based on the snoop (block (104)). That is, the snoop may represent a desired state change. The coherent agents (14A through 14n) may be configured to generate a completion (e.g., fill if the snoop is a snoop forward request, copy back snoop response if the snoop is a snoop back request, or acknowledgment response (forward or back based on the snoop type) if the snoop is a state change request). The coherent agents may be configured to generate a completion with a completion count from the snoop (block (106)) and to queue the completion for transmission to the request coherent agents (14A through 14n) (block (108)).

[0047] Using the cache coherence algorithm described herein, a cache block can be transferred from one coherent agent (14A to 14n) to another agent through a series of conflicting requests having low message bandwidth overhead. For example, FIG. 9 is a block diagram illustrating the transfer of a cache block between four coherent agents (CA0 to CA3). Similar to FIG. 3 and 4, the coherent agents are enumerated as CA0 to CA3, and the memory controller associated with the cache block is illustrated as MC. Vertical lines (110, 112, 114, 116, 118) for CA0, CA1, CA2, CA3, and MC respectively illustrate the source (base of the arrow) and destination (head of the arrow) of various messages corresponding to transactions. In FIG. 9, time progresses from top to bottom. At the time corresponding to the top of Fig. 9, the coherent agent (CA3) has a cache block involved in transactions in a modified state (tracked as 'exclusive' in the directory (26)). In Fig. 9, all transactions are for the same cache block.

[0048] The coherent agent (CA0) initiates a read-exclusive transaction with a CRdEx request to the memory controller (arrow (120)). The coherent agents (CA1, CA2) also initiate read-exclusive transactions (arrows (122, 124) respectively). As indicated by the heads of the arrows (120, 122, 124) in line (118), the memory controller (MC) orders the transactions as CA0, then CA1, and then finally CA2. The directory status for the transaction from CA0 is CA3 in an exclusive state, and thus a snoop forward and invalidate (SnpFwdI) is transmitted with the current cache state 'exclusive' (arrow (126)). The coherent agent (CA3) receives the snoop and forwards the FillM completion with data to the coherent agent (CA0) (arrow (128)). Similarly, the directory status for a transaction from CA1 is the exclusive state of the coherent agent (CA0) (from the previous transaction for CA0), and thus the memory controller (MC) issues SnpFwdI to the coherent agent (CA0) having the current cache state E (arrow 130), and the directory status for a transaction from CA2 is the coherent agent (CA1) having the current cache state E (arrow (132)). Once the coherent agent (CA0) has had the opportunity to perform at least one memory operation on the cache block, the coherent agent (CA0) responds to the coherent agent (CA1) with FillM complete (arrow (134)). Similarly, once the coherent agent (CA1) has had the opportunity to perform at least one memory operation on the cache block, the coherent agent (CA1) responds to the coherent agent (CA2) with his snoop with FillM complete (arrow (136)).The order and timing of various messages may change (e.g., similar to the race conditions illustrated in FIGS. 3 and 4), but generally, a cache block may move from agent to agent with one additional message (FillM complete) as conflicting requests are resolved.

[0049] In one embodiment, due to the race conditions mentioned above, a snoop may be received before a fill completion is snooped (detected by a snoop conveying an expected cache state). Additionally, a snoop may be received before Ack completions are collected and the fill completion can be processed. Ack completions are the result of snoops and therefore depend on the progress of the virtual channel conveying the snoops. Thus, conflicting snoops (delayed waits on the expected cache state) can fill internal buffers and exert reverse pressure into the fabric, which can cause a deadlock. In one embodiment, coherent agents (14A through 14n) may be configured to absorb one snoop forward and one snoop invalidation into an unprocessed request in the request buffer rather than allocating separate entries. Subsequently, non-conflicting snoops or conflicting snoops that reach a point where they can be processed without further interconnection dependencies can flow by bypassing the conflicting snoops and avoid deadlocks. Absorption of one snoop forward and one snoop invalidation may be sufficient, as the forwarding responsibility is transferred to the target when a snoop forward occurs. Therefore, no other snoop will occur again until the requester completes its current request and issues another new request after the previous snoop forward is completed. When a snoop invalidation is completed, the requester is in an invalidated state according to the directory, and will not receive another invalidation again until the requester processes the previous invalidation, requests the cache block again, and obtains a new copy.

[0050] Accordingly, coherent agents (14A through 14n) may be configured to ensure forward progress and / or prevent deadlocks by detecting a snoop received by the coherent agent for a cache block having an unprocessed request ordered before the snoop. The coherent agent may be configured to absorb the second snoop into the unprocessed request (e.g., into a request buffer entry storing the request). The coherent agent may process the absorbed snoop following completion of the unprocessed request. For example, if the absorbed snoop is a snoop forward request, the coherent agent may be configured to forward the cache block to another coherent agent indicated in the snoop forward request (and change the cache state to the state indicated by the snoop forward request) following completion of the unprocessed request. If the absorbed snoop is a snoop invalidation request, the coherent agent can update the cache state to invalidate and send an acknowledgment completion following the completion of the pending request. For example, absorbing a snoop into a conflicting request can be implemented by including additional storage for data describing the absorbed snoop within each request buffer entry.

[0051] FIG. 10 is a flowchart illustrating the operation of one embodiment of a coherent agent (14A to 14n) based on receiving a snoop. Although the blocks are shown in a specific order for ease of understanding, other orders may also be used. The blocks may be executed in parallel within the combination logic of the coherent agent (14A to 14n). The blocks, combinations of blocks, and / or the entire flowchart may be pipelined over multiple clock cycles. The coherent agent (14A to 14n) may be configured to implement the operation illustrated in FIG. 10. For example, the operation illustrated in FIG. 10 may be part of the detection of a waiting snoop where the expected cache state does not match the expected cache state (decision block (100) and block (102) of FIG. 8).

[0052] Coherent agents (14A through 14n) may be configured to compare the address of a snoop that must be waited due to the absence of a matching cache state with the addresses of unprocessed requests (or waiting requests) in the request buffer (62). If an address conflict is detected (decision block (140), "Yes" leg), the request buffer (62) may absorb the snoop into a buffer entry assigned to the waiting request where the address conflict was detected (block (142)). If there is no address conflict with the waiting request (decision block (140), "No" leg), the coherent agents (14A through 14n) may be configured to assign a separate buffer location for the snoop (e.g., in the request buffer (62) or another buffer of the coherent agents (14A through 14n)) and to store data describing the snoop within the buffer entry (block (144)).

[0053] As previously mentioned, in one embodiment, the cache coherence protocol may support both cacheable and non-caching requests while maintaining the coherence of the associated data. Non-caching requests may be issued, for example, by non-coherent agents (20A to 20p), and the non-coherent agents (20A to 20p) may not have the ability to consistently store cache blocks. In one embodiment, it may also be possible for a coherent agent (14A to 14n) to issue a non-caching request, and the coherent agent may not cache the data provided in response to such request. Therefore, a snoop-forward request for a non-caching request will not be appropriate, for example, if the data requested by a given non-coherent agent (20A to 20p) is in a modified cache block of one of the coherent agents (14A to 14n), and will be forwarded to the given non-coherent agent (20A to 20p), whereby the modified cache block is expected to be preserved by the given non-coherent agent (20A-20p).

[0054] To support coherent non-caching transactions, one embodiment of a scalable cache coherence protocol may include a number of types of snoops. For example, in one embodiment, snoops may include a snoop forward request and a snoop back request. As previously mentioned, a snoop forward request may cause a cache block to be forwarded to a request agent. Meanwhile, a snoop back request may cause a cache block to be sent back to a memory controller. In one embodiment, a snoop invalidation request may also be supported to invalidate a cache block (a forward version and a back version are provided for aiming for completions).

[0055] More specifically, a memory controller (22A to 22m) receiving a request (and more specifically, a coherence controller (24) within the memory controller (22A to 22m)) may be configured to read an entry corresponding to a cache block identified by an address in the request from the directory (26). The memory controller (22A to 22m) may be configured to issue a snoop to a given agent of coherent agents (14A to 14m) having a cached copy of the cache block according to the entry. The snoop indicates that the given agent will send the cache block to the source of the request based on the fact that the first request is of a first type (e.g., a cacheable request). The snoop indicates that the given agent will send the first cache block to the memory controller based on the fact that the first request is of a second type (e.g., a non-caching request). The memory controllers (22A through 22n) may be configured to respond to the source of the request with completion based on receiving a cache block from a given agent. Additionally, as with other coherent requests, the memory controllers (22A through 22n) may be configured to update an entry in the directory (26) to reflect the completion of a non-caching request based on issuing multiple snoops for the non-caching request.

[0056] FIG. 11 is a block diagram illustrating an example of a non-caching transaction managed coherently in one embodiment. FIG. 11 may be an example of a 4-hop protocol that transmits snooped data to a requester via a memory controller. A non-coherent agent is listed as NCA0, a coherent agent is listed as CA1, and a memory controller associated with a cache block is listed as MC. Vertical lines (150, 152, 154) for NCA0, CA1, and MC illustrate the source (base of the arrow) and destination (head of the arrow) of various messages corresponding to the transactions. In FIG. 11, time progresses from top to bottom.

[0057] At the time corresponding to the top of FIG. 11, the coherent agent (CA1) has a cache block in an exclusive state (E). NCA0 issues a non-caching read request (NCRd) to MC (arrow (156)). MC determines from the directory (26) that CA1 has a cache block containing the data requested by the exclusive state NCRd and generates a snoopback request (SnpBkI(E)) for CA1 (arrow (158)). CA1 provides a copyback snoop response (CpBkSR) to MC along with the cache block of data (arrow (160)). If the data is modified, MC can update the memory with the data and complete the request by providing the data for the non-caching read request to NCA0 within the non-caching read response (NCRdRsp) (arrow (162)). In one embodiment, there may be more than one of the following types of NCRd requests: requests to invalidate a cache block within a snooped coherent agent and requests to allow the snooped coherent agent to retain the cache block. The discussion above exemplifies invalidation. In other cases, the snooped agent may retain the cache block in the same state.

[0058] A non-caching write request can be performed in a similar manner, using a snoop-back request to acquire a cache block, and modifying the cache block with non-caching write data before writing the cache block to memory. A non-caching write response may still be provided to notify the non-caching agent (NCA0 in FIG. 11) that the write is complete.

[0059] FIG. 12 is a flowchart illustrating the operation of an embodiment of a memory controller (22A to 22m) (more specifically, in one embodiment, a coherence controller (24) within the memory controller (22A to 22m)) in response to a request, illustrating caching-capable and non-caching-capable operations. The operation illustrated in FIG. 12 may be a more detailed example of a part of the operation illustrated in FIG. 6, for example. Although the blocks are illustrated in a specific order for ease of understanding, other orders may be used. The blocks may be executed in parallel in the combination logic of the coherence controller (24). The blocks, combinations of blocks, and / or the entire flowchart may be pipelined over a number of clock cycles. The coherence controller (24) may be configured to implement the operation illustrated in FIG. 12.

[0060] The coherence controller (24) may be configured to read the directory based on the address in the request. If the request is a directory hit (decision block (170), "Yes" leg), the cache block exists in one or more caches within the coherent agents (14A through 14n). If the request is not cacheable (decision block (172), "Yes" leg), the coherence controller (24) may be configured to issue a snoop-back request to the coherent agent (14A through 14n) responsible for providing a copy of the cache block (and, where applicable, to snoop on invalidation requests for the shared agents (back variant) - block (174)). The coherence controller (24) may be configured to update the directory to reflect the completed snoops (e.g., to invalidate the cache block within the coherent agents (14A through 14n) - block (176)). The coherence controller (24) may be configured to wait for any Ack snoop response from the shared coherent agent (14A to 14n) as well as a copy back snoop response (decision block (178), "yes" leg) and may be configured to generate a non-caching completion (where appropriate, NCRdRsp or NCWrRsp) for the requesting agent (block (180)). When the cache block is modified, the data may also be written to memory by the memory controller (22A to 22m).

[0061] If the request is cacheable (decision block (172), "No" leg), the coherence controller (24) may be configured to generate snoop forward requests to the coherent agents (14A through 14n) responsible for forwarding the cache block (block (182)), as well as other snoops to other caching coherent agents (14A through 14n) if necessary. The coherence controller (24) may update the directory (24) to reflect the completion of the transaction (block (184)).

[0062] If the request is not a hit within the directory (26) (decision block (170), "No" leg), there is no cached copy of the cache block within the coherent agents (14A through 14n). In this case, no snoops may be generated, and the memory controller (22A through 22m) may be configured to generate a fill completion (for a cacheable request) or a non-caching completion (for a non-caching request) to provide data or complete the request (block (186)). For a cacheable request, the coherence controller (24) may update the directory (26) to generate an entry for the cache block and initialize the request coherent agents (14A through 14n) to have a copy of the cache block in the cache state requested by the coherent agents (14A through 14n) (block (188)).

[0063] FIG. 13 is a table (190) illustrating exemplary cache states that may be implemented in one embodiment of coherent agents (14A to 14n). Other embodiments may employ different cache states, a subset of the illustrated cache states and other cache states, a superset of the illustrated cache states and other cache states, etc. A modified state (M), or "dirty exclusive" state, may be a state of coherent agents (14A to 14n) having a unique cached copy of a cache block (the copy is exclusive), wherein the data in the cached copy has been modified in relation to the corresponding data in memory (e.g., at least one byte of the data differs from the corresponding byte in memory). The modified data may also be referred to as dirty data. The ownership state (O) or “dirty shared” state may be a state within a coherent agent (14A to 14n) that has a modified copy of a cache block but may have shared the copy with at least one other coherent agent (14A to 14n) (however, the other coherent agent (14A to 14n) may have subsequently evicted the shared cache block). The other coherent agent (14A to 14n) will have a cache block in a secondary shared state. The exclusive state (E), or “clean exclusive” state may be a state within a coherent agent (14A to 14n) that has a unique cached copy of a cache block, but the cached copy has the same data as the corresponding data in memory. The exclusive data no (EnD) state, or the “clean exclusive, data no” state, may be a state within a coherent agent (14A to 14n) similar to the exclusive (E) state, except that a cache block of data is not being delivered to the coherent agent.These states may be used when coherent agents (14A through 14n) modify each byte within a cache block, and thus there may be no benefit or coherence reason to supply previous data within the cache block. The End state may be an optimization to reduce traffic on the interconnect (28) and may not be implemented in other embodiments. The primary sharing (P) state or "clean sharing primary" state may be a state within a coherent agent (14A through 14n) that has a shared copy of the cache block but is also responsible for forwarding the cache block to another coherent agent based on a snoop forward request. The secondary sharing (S) state or "clean sharing secondary" state may be a state within a coherent agent (14A through 14n) that has a shared copy of the cache block but is not responsible for providing the cache block when another coherent agent (14A through 14n) has the cache block of the primary sharing state. In some embodiments, if the coherent agents (14A to 14n) do not have a cache block in the primary shared state, the coherence controller (24) may select a secondary shared agent to provide the cache block (and may send a snoop-forward request to the selected coherent agent). In other embodiments, the coherence controller (24) may cause the memory controller (22A to 22m) to provide the cache block to the requester when there is no coherent agent (14A to 14n) in the primary shared state. The invalid state (I) may be a state within the coherent agents (14A to 14n) that do not have a cached copy of the cache block. Coherent agents (14A through 14n) in an invalid state may not have previously requested a copy, or may have had any copy and invalidated it based on snooping or evicting of a cache block to cache a different cache block.

[0064] FIG. 14 is a table (192) illustrating various messages that may be used in one embodiment of a scalable cache coherence protocol. There may be alternative messages in other embodiments, subsets of the illustrated messages and additional messages, supersets of the illustrated messages and additional messages, etc. Messages may carry transaction identifiers (e.g., initial requests, snoops, completions) linking messages from the same transaction. Initial requests and snoops may carry the addresses of cache blocks affected by the transaction. Some other messages may also carry addresses. In some embodiments, all messages may carry addresses.

[0065] Cacherable read transactions may be initiated by a Cacherable Read Request message (CRd). There may be various versions of the CRd request to request different cache states. For example, a CRdEx may request an exclusive state, and a CRdS may request a secondary shared state, etc. The cache state actually provided in response to a Cacherable Read Request may be at least as permissive as the requested state, or more permissive. For example, a CRdEx may receive a cache block in an exclusive or modified state. A CRdS may receive a block in a primary shared, exclusive, owned, or modified state. In one embodiment, an opportunistic CRd request may be implemented, and the most permissive state possible (not invalidating other copies of the cache block) may be granted (e.g., exclusive if there are no copies cached in other coherent agents, and owned or primary shared if there are cached copies).

[0066] A change to exclusive (CtoE) message may be used by a coherent agent that holds a copy of a cache block in a non-modifiable state (e.g., owned, primary shared, secondary shared) and is attempting to modify the cache block (e.g., the coherent agent requires exclusive access to change the cache block to a modified state). In one embodiment, a conditional CtoE message may be used for a store conditional instruction. The store conditional instruction is part of a load reserve / store conditional pair, where the load acquires a copy of the cache block and sets a reservation for the cache block. Coherent agents (14A to 14n) can monitor access to cache blocks by other agents and can conditionally perform a save based on whether the cache block has not been modified by other coherent agents (14A to 14n) between the load and the save (successfully save if the cache block has not been modified, and not save if the cache block has been modified). Additional details are provided below.

[0067] In one embodiment, a cache read-exclusive, data-only (CRdE-Donly) message may be used when coherent agents (14A to 14n) modify an entire cache block. If the cache block has not been modified by other coherent agents (14A to 14n), the requesting coherent agents (14A to 14n) may use the EnD cache state and modify all bytes of the block without transmitting previous data within the cache block to the agent. If the cache block is modified, the modified cache block may be passed to the requesting coherent agents (14A to 14n), and the requesting coherent agents (14A to 14n) may use the M cache state.

[0068] Non-caching transactions may be initiated with non-caching read and non-caching write (NCRd and NCWr) messages.

[0069] Snoop forward and snoop back (SnpFwd and SnpBk, respectively) can be used for snoops as described above. After processing the snoop, there may be messages to request various states within the receiving coherent agent (14A to 14n) (e.g., invalidation state or shared state). There may also be a snoop forward message for a CRdE-Donly request, which requests forwarding if the cache block is modified, but otherwise requests invalidation at the receiver without forwarding. In one embodiment, there may also be invalidation-only snoop forward and snoop back requests (e.g., snoops that cause the receiver to invalidate without returning data and to acknowledge to the requester or memory controller, respectively).

[0070] Completion messages may include a Fill message and an Acknowledgment message (Ack). The Fill message may specify the state of the cache block to be taken by the requester upon completion. A Caching Write Back (CWB) message may be used to transfer the cache block to the memory controller (22A through 22m) (e.g., based on evicting the cache block from the cache). A Copy Back Snoop Response (CpBkSR) may be used to transfer the cache block to the memory controller (22A through 22m) (e.g., based on the Snoop Back message). A Non-Cacheable Write Complete (NCWrRsp) and a Non-Cacheable Read Complete (NCRdRsp) may be used to complete non-caching requests.

[0071] FIG. 15 is a flowchart illustrating the operation of one embodiment of a coherence controller (24) based on receiving a Change to Conditionally Exclusive (CtoECond) message. For example, FIG. 15 may be a more detailed description of a portion of the block (70) of FIG. 6 in one embodiment. Although the blocks are shown in a specific order for ease of understanding, other orders may also be used. The blocks may be executed in parallel in the combination logic of the coherence controller (24). The blocks, combinations of blocks, and / or the entire flowchart may be pipelined over multiple clock cycles. The coherence controller (24) may be configured to implement the operation illustrated in FIG. 15.

[0072] A CtoECond message may be issued by a coherent agent (14A through 14n) ("source") based on the execution of a store conditional command. A store conditional command may fail locally at the source if the source has lost a copy of the cache block prior to the store conditional command (e.g., the copy is no longer valid). If the source still has a valid copy (e.g., a secondary or primary shared state, or an owned state), there is still a possibility that another transaction will be ordered prior to an exclusive change message from the source that causes the source to invalidate its cached copy when the store conditional command is executed. The same transaction that invalidates the cached copy will also cause the store conditional command to fail at the source. To avoid invalidations of cache blocks and the delivery of cache blocks to the source where the store conditional command would fail, a CoECond message may be provided and used by the source.

[0073] A CtoECond message may be defined to have at least two possible outcomes when ordered by a coherence controller (24). When the CtoECond message is ordered and processed, if the source still has a valid copy of the cache block as indicated in the directory (26), the CtoECond may proceed similarly to a non-conditional CtoE message: that is, it issues snoops and obtains an exclusive status for the cache block. If the source does not have a valid copy of the cache block, the coherence controller (24) may fail the CtoE transaction and return an Ack completion to the source with an indication that the CtoE failed. The source may terminate the CtoE transaction based on the Ack completion.

[0074] As illustrated in FIG. 15, the coherence controller (24) may be configured to read a directory entry for an address (block (194)). If the source holds a valid copy of the cache block (e.g., in a shared state) (decision block (196), "yes" leg), the coherence controller (24) may be configured to generate snoops based on the cache states within the directory entry (e.g., snoops to invalidate the cache block so that the source can change to an exclusive state) (block (198)). If the source does not hold a valid copy of the cache block (decision block (196), "no" leg), the cache controller (24) may be configured to send a acknowledgment completion to the source to indicate the failure of the CtoECond message (block (200)). Thus, the CtoE transaction may be terminated.

[0075] Now, referring to FIG. 16, a flowchart illustrating the operation of one embodiment of a coherence controller (24) for reading directory entries and determining snoops (e.g., at least a portion of block (70) of FIG. 6 in one embodiment) is shown. Although the blocks are shown in a specific order for ease of understanding, other orders may also be used. The blocks may be executed in parallel in the combination logic of the coherence controller (24). The blocks, combinations of blocks, and / or the entire flowchart may be pipelined over a number of clock cycles. The coherence controller (24) may be configured to implement the operation shown in FIG. 16.

[0076] As illustrated in FIG. 16, the coherence controller (24) may be configured to read a directory entry for the address of the request (block (202)). Based on the cache states within the directory entry, the coherence controller (24) may be configured to generate snoops. For example, based on the cache state of one of the agents being at least primary shared (decision block (204), "Yes" leg), the coherence controller (24) may be configured to send a SnpFwd snoop to the primary shared agent to indicate that the primary shared agent will send the cache block to the request agent. For other agents (e.g., secondary shared state), the coherence controller (24) may be configured to generate invalidation-only snoops (SnpInv) indicating that the other agents will not send the cache block to the request agent (block (206)). In some cases (e.g., a CRdS request requesting a shared copy of a cache block), other agents do not need to receive snoops because they do not need to change the state. An agent may have at least a primary shared cache state if the cache state is allowed as at least a primary share (e.g., primary share, ownership, exclusive, or modification in the embodiment of FIG. 13).

[0077] If there is no agent having at least a primary shared cache state (decision block (204), "No" leg), the coherence controller (24) may be configured to determine whether one or more agents have a cache block of a secondary shared state (decision block (208)). In such a case (decision block (208), "Yes" leg), the coherence controller (24) may be configured to select one of the agents having a secondary shared state and may send a SnpFwd request command to the selected agent to forward the cache block to the requesting agent. The coherence controller (24) may be configured to generate SnpInv requests for other agents of the secondary shared state, which indicate that the other agents will not send the cache block to the requesting agent (block (210)). As described above, SnpInv messages may not be generated or sent if the other agents do not need to change the state.

[0078] If there is no agent with a cache state of secondary sharing (decision block (208), "No" leg), the coherence controller (24) may be configured to generate a fill completion and may be configured to have the memory controller read a cache block for transmission to the requesting agent (block (212)).

[0079] FIG. 17 is a flowchart illustrating the operation of a coherence controller (24) for reading directory entries and determining snoops in response to a CRdE-Donly request in one embodiment (e.g., at least part of block (70) of FIG. 6 in one embodiment). Although the blocks are shown in a specific order for ease of understanding, other orders may be used. The blocks may be executed in parallel in the combination logic of the coherence controller (24). The blocks, combinations of blocks, and / or the entire flowchart may be pipelined over multiple clock cycles. The coherence controller (24) may be configured to implement the operation illustrated in FIG. 17.

[0080] As described above, a CRdE-Donly request can be used by coherent agents (14A through 14n) to modify all bytes within a cache block. Thus, the coherence controller (24) can cause other agents to invalidate the cache block. If an agent has modified the cache block, the agent may supply the modified cache block to the requesting agent. Otherwise, the agents may not supply the cache block.

[0081] The coherence controller (24) may be configured to read a directory entry for the address of the request (block (220)). Based on the cache states within the directory entry, the coherence controller (24) may be configured to generate snoops. More specifically, if a given agent may have a modified copy of a cache block (e.g., a given agent may have a cache block in an exclusive or primary state) (block (222), "Yes" leg), the cache controller (24) may generate a Snoop Forward-Dirty Donly (SnpFwdDonly) for the agent to send the cache block to the requesting agent (block (224)). As previously mentioned, the SnpFwdDonly request may cause the receiving agent to send the cache block if the data is modified, but otherwise not send the cache block. In either case, the receiving agent may invalidate the cache block. The receiving agent may send a fill completion if the data is modified and provide the modified cache block. Otherwise, the receiving agent may send an Ack completion. If no agent has a modified copy (decision block (222), "No" leg), the coherence controller (24) may be configured to generate a snoop invalidation (SnpInv) for each agent having a cached copy of the cache block (block (226)). In another embodiment, since the requester will modify the entire cache block, the coherence controller (24) may not request the forwarding of the data even if the cache block is modified. That is, the coherence controller (24) may cause the agent having the modified copy to invalidate the data without forwarding the data.

[0082] computer system

[0083] Next, referring to FIG. 18, a block diagram of one embodiment of a system (700) is illustrated. In the illustrated embodiment, the system (700) includes at least one instance of a system-on-chip (SOC) (10) coupled to one or more peripherals (704) and an external memory (702). A power supply unit (PMU) (708) is provided to supply supply voltages to the SOC (10) as well as to supply one or more supply voltages to the memory (702) and / or peripherals (154). In some embodiments, more than one instance of the SOC (10) may be included (and more than one memory (702) may also be included). In one embodiment, the memory (702) may include the memories (12A to 12m) illustrated in FIG. 1.

[0084] Peripherals (704) may include any desired circuitry depending on the type of system (700). For example, in one embodiment, the system (704) may be a mobile device (e.g., a personal digital assistant (PDA), a smartphone, etc.), and the peripherals (704) may include various types of wireless communication devices such as Wi-Fi, Bluetooth, cellular, a global positioning system, etc. The peripherals (704) may also include additional storage including RAM storage, solid state storage, or disk storage. The peripherals (704) may include user interface devices, such as a display screen including touch display screens or a multi-touch display screen, a keyboard or other input device, a microphone, a speaker, etc. In other embodiments, the system (700) may be any type of computing system (e.g., a desktop personal computer, a laptop, a workstation, a nettop, etc.).

[0085] The external memory (702) may include any type of memory. For example, the external memory (702) may be SRAM, dynamic RAM (DRAM), e.g., synchronous DRAM (SDRAM), double data rate (DDR, DDR2, DDR3, etc.) SDRAM, RAMBUS DRAM, low-power versions of DDR DRAM (e.g., LPDDR, mDDR, etc.). The external memory (702) may include one or more memory modules on which memory devices are mounted, e.g., single inline memory modules (SIMM), dual inline memory modules (DIMM), etc. Alternatively, the external memory (702) may include one or more memory devices mounted on the SOC (10) in a chip-on-chip or package-on-package implementation example.

[0086] As illustrated, the system (700) is depicted as having applications in a wide range of areas. For example, the system (700) may be used as part of chips, circuits, components, etc. of a desktop computer (710), a laptop computer (720), a tablet computer (730), a cellular or mobile phone (740), or a television (750) (or a set-top box coupled to the television). A smartwatch and a health monitoring device (760) are also illustrated. In some embodiments, the smartwatch may include various general-purpose computing-related functions. For example, the smartwatch may provide access to email, cell phone services, user calendars, etc. In various embodiments, the health monitoring device may be a dedicated medical device or otherwise may include dedicated health-related functions. For example, the health monitoring device may monitor the user's vital signs, track the user's proximity to other users for the purpose of epidemiological social distancing, track contacts, and provide communication to emergency services in case of health risk. In various embodiments, the smartwatch mentioned above may or may not include some or any health monitoring-related functions. Other wearable devices, such as devices worn around the neck, devices implantable in the human body, and glasses designed to provide augmented and / or virtual reality experiences, are also considered.

[0087] The system (700) may additionally be used as part of a cloud-based service(s) (770). For example, the previously mentioned devices and / or other devices may access computing resources within the cloud (i.e., remotely located hardware and / or software resources). Additionally, the system (700) may be utilized in one or more devices in the home other than those previously mentioned. For example, devices in the home may monitor and detect conditions requiring attention. For example, various devices in the home (e.g., refrigerator, cooling system, etc.) may monitor the status of the device and provide an alarm to the homeowner (or, for example, a repair facility) when a specific event is detected. Alternatively, a thermostat may monitor the temperature in the home and automate adjustments to the heating / cooling system based on the history of responses to various conditions by the homeowner. Additionally, the application of the system (700) to various modes of transport is illustrated in FIG. 18. For example, the system (700) may be used in control and / or entertainment systems for aircraft, trains, buses, rental cars, private cars, watercraft ranging from private boats to cruise liners, scooters (for rental or ownership), etc. In various cases, the system (700) may be used to provide automated guidance (e.g., autonomous vehicles), general system control, etc. Any many other embodiments of these are possible and considered. It should be noted that the devices and applications illustrated in FIG. 18 are merely illustrative and are not intended to be limiting. Other devices are possible and considered.

[0088] computer-readable storage media

[0089] Now, referring to FIG. 19, a block diagram of one embodiment of a computer-readable storage medium (800) is illustrated. Generally speaking, a computer-accessible storage medium may include any storage medium accessible by a computer during use to provide instructions and / or data to the computer. For example, a computer-accessible storage medium may include magnetic or optical media, such as disks (fixed or removable), tapes, CD-ROMs, DVD-ROMs, CD-Rs, CD-RWs, DVD-Rs, DVD-RWs, or Blu-ray. The storage media may further include volatile or non-volatile memory media such as RAM (e.g., synchronous dynamic RAM (SDRAM), Rambus DRAM (RDRAM), static RAM (SRAM), etc.), ROM, or flash memory. The storage media may be physically contained within the computer to which the storage media provide instructions / data. Alternatively, the storage media may be connected to the computer. For example, the storage media may be connected to the computer via a network or wireless link, such as network-attached storage. Storage media can be connected via a peripheral interface such as USB (Universal Serial Bus). Generally, computer-accessible storage media (800) can store data in a non-transient manner, and in this context, non-transient may refer to not transmitting commands / data over a signal. For example, non-transient storage may be volatile (and may lose stored commands / data in response to a power cut) or non-volatile.

[0090] The computer-accessible storage medium (800) of FIG. 19 may store a database (804) representing an SOC (10). Generally, the database (804) may be a database that can be read by a program and used directly or indirectly to manufacture hardware containing the SOC (10). For example, the database may be a behavioral-level description or a register-transfer level (RTL) description of hardware functions in a high-level design language (HDL), such as Verilog or VHDL. The description may be read by a synthesis tool capable of synthesizing the description to generate a netlist containing a list of gates from a synthesis library. The netlist contains a set of gates that also represent the functions of the hardware containing the SOC (10). Subsequently, the netlist may be placed and routed to generate a data set describing geometric shapes to be applied to masks. Subsequently, the masks may be used in various semiconductor manufacturing steps to create a semiconductor circuit or circuits corresponding to the SOC (10). Alternatively, the database (804) on the computer-accessible storage medium (800) may be a netlist (with or without a synthesis library) or a data set, as desired.

[0091] A computer-accessible storage medium (800) stores a representation of the SOC (10), but other embodiments may convey a representation of any part of the SOC (10) as desired, including any subset of coherent agents (14A to 14n), memory controllers (22A to 22m), non-coherent agents (20A to 20p), coherence controllers (24), a directory (26), etc. A database (804) may represent any of the above parts.

[0092] Based on the present disclosure, a system may include a plurality of coherent agents, and a given agent among the plurality of coherent agents includes one or more caches for caching memory data. The system further includes a memory controller coupled to one or more memory devices, and the memory controller includes a directory configured to track which of the plurality of coherent agents is caching copies of a plurality of cache blocks within the memory devices and the states of the cached copies within the plurality of coherent agents. Based on a first request for a first cache block by a first agent among the plurality of coherent agents, the memory controller may be configured to read an entry corresponding to the first cache block from the directory, issue a snoop to a second agent among the plurality of coherent agents having a cached copy of the first cache block according to the entry, and include in the snoop an identifier of a first state of the first cache block within the second agent. Based on the snoop, the second agent may be configured to compare the first state with the second state of the first cache block within the second agent and delay the processing of the snoop based on the fact that the first state does not match the second state until the second state changes to the first state in response to a different communication related to a request different from the first request. In one embodiment, the memory controller may be configured to determine a completion count representing the number of completions to be received by the first agent for the first request—the determination is based on states from the entry—and to include the completion count in a plurality of snoops issued based on the first request, including the snoop issued to the second agent.A first agent may be configured to initialize a completion counter with a completion count based on receiving an initial completion from one of a plurality of coherent agents, update the completion counter based on receiving a subsequent completion from another of the plurality of coherent agents, and complete a first request based on the completion counter. In one embodiment, a memory controller may be configured to update states within an entry of a directory to reflect the completion of the first request based on issuing a plurality of snoops based on the first request. In one embodiment, the first agent may be configured to detect a second snoop received by the first agent for a first cache block, and the first agent may be configured to absorb the second snoop into the first request. In one embodiment, the first agent may be configured to process the second snoop following the completion of the first request. In one embodiment, the first agent may be configured to forward the first cache block to a third agent indicated in the second snoop following the completion of the first request. In one embodiment, the third agent may be configured to generate a change request to a conditionally exclusive state based on a storage conditional command for a second cache block that is in a valid state in the third agent. The memory controller may be configured to determine whether the third agent holds a valid copy of the second cache block based on a second entry in the directory associated with the second cache block, and the memory controller may be configured to send a completion indicating failure to the third agent based on the determination that the third agent no longer holds a valid copy of the second cache block, and to terminate the change request to a conditionally exclusive state.In one embodiment, the memory controller may be configured to issue one or more snoops to other coherent agents among a plurality of coherent agents, as indicated by the second entry, based on a determination that the third agent holds a valid copy of the second cache block. In one embodiment, the snoop indicates that the second agent will transfer the first cache block to the first agent based on a first state that is shared a second time, and the snoop indicates that the second agent will not transfer the first cache block based on a first state that is shared a second time. In one embodiment, the snoop indicates that the second agent will transfer the first cache block even if the first state is shared a second time.

[0093] In another embodiment, the system includes a plurality of coherent agents, wherein a given agent among the plurality of coherent agents includes one or more caches for caching memory data. The system further includes a memory controller coupled to one or more memory devices. The memory controller may include a directory configured to track which of the plurality of coherent agents is caching copies of a plurality of cache blocks within the memory devices, and the states of the cached copies within the plurality of coherent agents. Based on a first request for a first cache block by a first agent among the plurality of coherent agents, the memory controller may be configured to read an entry corresponding to the first cache block from the directory and, according to the entry, issue a snoop to a second agent among the plurality of coherent agents that has a cached copy of the first cache block. The snoop may indicate that the second agent will transfer the first cache block to the first agent based on an entry indicating that the second agent has the first cache block in at least a first shared state. Snoop indicates that the second agent will not transfer the first cache block to the first agent based on a different agent having at least a first cache block in a primary shared state. In one embodiment, when a different agent is in a primary shared state, the first agent is in a secondary shared state for the first cache block. In one embodiment, snoop indicates that the first agent will invalidate the first cache block based on a different agent having at least a first cache block in a primary shared state. In one embodiment, the memory controller is configured not to issue a snoop to the second agent based on a first request that is a request for a shared copy of the first cache block and a different agent having a first cache block in a primary shared state.In one embodiment, the first request may be for an exclusive state for the first cache block, and the first agent modifies the entire first cache block. A snoop may indicate that the second agent will transmit the first cache block if the first cache block is in a modified state by the second agent. In one embodiment, a snoop indicates that the second agent will invalidate the first cache block if the first cache block is not in a modified state by the second agent.

[0094] In another embodiment, the system includes a plurality of coherent agents, wherein a given agent among the plurality of coherent agents includes one or more caches for caching memory data. The system further includes a memory controller coupled to one or more memory devices. The memory controller may include a directory configured to track which of the plurality of coherent agents is caching copies of a plurality of cache blocks within the memory devices, and the states of the cached copies within the plurality of coherent agents. Based on a first request for a first cache block, the memory controller may be configured to read an entry corresponding to the first cache block from the directory and, according to the entry, issue a snoop to a second agent among the plurality of coherent agents that has a cached copy of the first cache block. The snoop may indicate that the second agent will transfer the first cache block to the source of the first request based on an attribute associated with the first request having a first value, and the snoop may indicate that the second agent will transfer the first cache block to the memory controller based on an attribute having a second value. In one embodiment, the attribute is the type of request, the first value is cacheable, and the second value is not cacheable. In another embodiment, the attribute is the source of the first request. In one embodiment, the memory controller may be configured to respond to the source of the first request based on receiving a first cache block from a second agent. In one embodiment, the memory controller is configured to update the states within the directory entries to reflect the completion of the first request based on issuing a plurality of snoops based on the first request.

[0095] ***

[0096] The present disclosure includes references to "examples" or "groups of examples" (e.g., "some examples" or "various examples"). Examples are different implementations or instances of the disclosed concepts. References to "one example," "one example," "a specific example," etc., do not necessarily refer to the same example. Many possible examples are considered, including modifications or alternatives that fall within the spirit or scope of the present disclosure, as well as those specifically disclosed.

[0097] The present disclosure may discuss potential benefits that may arise from the disclosed embodiments. Not all implementations of these embodiments will necessarily represent any or all of the potential benefits. Whether benefits are realized for a particular implementation depends on many factors, some of which are outside the scope of the present disclosure. In fact, there are many reasons why an implementation within the scope of the claims may not represent some or all of any disclosed benefits. For example, a particular implementation may include other circuitry outside the scope of the present disclosure that, together with one of the disclosed embodiments, nullifies or weakens one or more of the disclosed benefits. Furthermore, suboptimal design practices of a particular implementation (e.g., implementation techniques or tools) may also nullify or weaken the disclosed benefits. Even assuming a skilled implementation, the realization of benefits may still depend on other factors, such as the environmental conditions in which the implementation is deployed. For example, inputs supplied to a particular implementation may prevent one or more of the problems addressed in the present disclosure from occurring at a particular opportunity, and as a result, the benefits of the solution may not be realized. Given the existence of possible factors outside of the present disclosure, it is expressly intended that any potential benefits described herein should not be interpreted as claim limitations that must be satisfied to prove infringement. Rather, the identification of such potential benefits is intended to exemplify the type(s) of improvement available to designers who benefit from the present disclosure. That such benefits are described as acceptable (e.g., that a particular benefit is referred to as “may occur”) is not intended to convey doubt as to whether such benefits can actually be realized, but rather to acknowledge the technical reality that the realization of such benefits often depends on additional factors.

[0098] Unless otherwise noted, the embodiments are non-limiting. That is, the disclosed embodiments are not intended to limit the scope of claims drawn up based on the present disclosure, even if only a single example is described for a particular feature. The disclosed embodiments are intended to be illustrative rather than limiting, without any statement in the present disclosure to the contrary. Accordingly, the present application is intended to allow not only claims covering the disclosed embodiments but also such alternatives, modifications, and equivalents that will be apparent to those skilled in the art having an interest in the present disclosure.

[0099] For example, the features in this application may be combined in any suitable manner. Accordingly, new claims may be drafted during the examination of this application (or the application claiming priority thereto) for any such combination of features. In particular, with reference to the appended claims, features from dependent claims may be combined with features of other dependent claims where appropriate, including claims that depend on other independent claims. Similarly, features from individual independent claims may be combined where appropriate.

[0100] Accordingly, while the appended dependent claims may each be drafted to depend on a single other claim, additional dependencies are also taken into account. Any combination of features in dependencies consistent with the present disclosure is taken into account and may be claimed in such or other applications. Simply put, the combinations are not limited to those specifically enumerated in the appended claims.

[0101] Where appropriate, claims written in one format or statutory type (e.g., device) are also considered to be intended to support corresponding claims in another format or statutory type (e.g., method).

[0102] ***

[0103] Because the present disclosure is a legal document, various terms and phrases may be subject to administrative and judicial interpretation. It is given in this specification that the following paragraphs, as well as the definitions provided throughout the present disclosure, will be used to determine how to interpret claims drafted based on the present disclosure.

[0104] References to the singular form of an article (i.e., a noun or noun phrase preceded by "a," "an," or "the") are intended to mean "one or more" unless clearly indicated otherwise by the context. Accordingly, references to "items" in a claim do not exclude additional instances of the items without accompanying circumstances. "Plural" items refer to sets of two or more items.

[0105] The word "can" is used in this specification not in a mandatory sense (i.e., must), but in a permissive sense (i.e., having the possibility of, able to).

[0106] The terms "comprising" and "including" and their forms are open-ended and mean "comprising but not limited to."

[0107] When the term “or” is used in this disclosure with respect to this list of options, it shall be understood to be used in a generally inclusive sense unless the context otherwise provides. Accordingly, references to “x or y” are equivalent to “x or y, or both,” and thus cover 1) x but not y, 2) y but not x, and 3) both x and y. On the other hand, phrases such as “either x or y, but neither” make it clear that “or” is used in an exclusive sense.

[0108] References to “w, x, y, or z, or any combination thereof” or “... at least one of w, x, y, and z” are intended to cover all possibilities involving a single element up to the total number of elements in the set. For example, given the set [w, x, y, z], these phrases cover any single element of the set (e.g., w but not x, y, or z), any two elements (e.g., w and x but not y or z), any three elements (e.g., w, x, and y but not z), and all four elements. Thus, the phrase “... at least one of w, x, y, and z” refers to at least one element of the set [w, x, y, z], thereby covering all possible combinations of elements within this list. This phrase should not be interpreted to require that there be at least one instance of w, at least one instance of x, at least one instance of y, and at least one instance of z.

[0109] In this disclosure, various “labels” may precede nouns or noun phrases. Unless the context otherwise provides, different labels used for a feature (e.g., “first circuit,” “second circuit,” “specific circuit,” “given circuit,” etc.) refer to different instances of the feature. Additionally, when applied to a feature, the “first,” “second,” and “third” labels do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) unless otherwise noted.

[0110] The phrase “based on” is used to describe one or more factors that influence a decision. This term does not exclude the possibility that additional factors may influence the decision. That is, the decision may be based solely on specific factors, or on other, unspecified factors in addition to those specific factors. Consider the phrase “determine A based on B.” This phrase specifies that B is a factor used to determine A or that influences the determination of A. This phrase does not exclude that the determination of A may also be based on some other factor, such as C. Furthermore, this phrase is intended to cover embodiments where A is determined solely on B. As used herein, the phrase “based on” is synonymous with the phrase “at least partially based.”

[0111] The phrases “in response to” and “in response to” describe one or more factors that trigger an effect. These phrases do not exclude the possibility that additional factors may influence or otherwise trigger an effect, either jointly with or independently of specific factors. That is, the effect may respond only to these factors, or may respond to specific factors as well as other unspecified factors. Consider the phrase “perform A in response to B.” This phrase specifies that B is a factor that triggers the performance of A or triggers a specific result for A. This phrase does not exclude that performing A may also respond to some other factor, such as C. This phrase also does not exclude that performing A may be performed jointly in response to B and C. This phrase is also intended to cover embodiments where A is performed only in response to B. As used herein, the phrase “in response” is synonymous with the phrase “at least partially in response.” Similarly, the phrase "in response to" is synonymous with the phrase "at least partially in response."

[0112] ***

[0113] Within this disclosure, different entities (which may be variously referred to as “units,” “circuits,” other components, etc.) may be described or claimed to be “configured” to perform one or more tasks or operations. The expression “[entity] configured to perform [one or more tasks]” is used in this specification to refer to a structure (i.e., a physical one). More specifically, this expression is used to indicate that such a structure is arranged to perform one or more tasks during operation. A structure may be said to be “configured” to perform some tasks even if the structure is not currently operating. Accordingly, an entity described or mentioned as being “configured” to perform some tasks refers to a physical thing, such as a device, a circuit, a system having memory and a processor unit that stores program instructions executable to implement the task. Such phrases are not used in this specification to refer to intangible things.

[0114] In some cases, various units / circuits / components may be described herein as performing a set of tasks or operations. It is understood that these entities are "configured" to perform such tasks / operations, even if not specifically mentioned.

[0115] The term "configured to" is not intended to mean "configurable to." For example, an unprogrammed FPGA would not be considered "configured" to perform a specific function. However, such an unprogrammed FPGA can be "configurable" to perform that function. After proper programming, it can then be said that the FPGA is "configured" to perform a specific function.

[0116] For the purposes of U.S. patent applications based on the present disclosure, mentioning in a claim that a structure is "configured" to perform one or more tasks is clearly intended not to apply 35 USC §112(f) to the elements of the claim. If the applicant wishes to apply Section 112(f) during the examination of a U.S. patent application based on the present disclosure, the elements of the claim will be enumerated using a structure that is a "means" for "performing" [a function].

[0117] Different “circuits” may be described in this disclosure. These circuits or “circuit parts” constitute hardware comprising various types of circuit elements, such as combinational logic, clocked storage devices (e.g., flip-flops, registers, latches, etc.), finite state machines, memory (e.g., random access memory, embedded dynamic random access memory), programmable logic arrays, etc. Circuit parts may be custom-designed or taken from standard libraries. In various implementations, circuit parts may appropriately include digital components, analog components, or a combination of both. Specific types of circuits may generally be referred to as “units” (e.g., decoding unit, arithmetic logic unit (ALU), function unit, memory management unit (MMU), etc.). Such units also refer to circuits or circuit parts.

[0118] Accordingly, the disclosed circuits / units / components and other elements illustrated in the drawings and described herein include hardware elements such as those described in the preceding paragraph. In many cases, the internal arrangement of hardware elements within a particular circuit can be specified by describing the function of the circuit. For example, a particular "decoding unit" may be described as performing the function of "processing the op-code of an instruction and routing the instruction to one or more of a plurality of function units," which implies that the decoding unit is "configured" to perform this function. Specification of such function is sufficient to suggest to those skilled in the art of computers a set of possible structures for the circuit.

[0119] In various embodiments, as discussed in the previous paragraph, circuits, units, and other elements, arrangements, and such circuits / units / components defined by functions or operations configured to be implemented, form a microarchitecture definition of hardware that is ultimately manufactured in an integrated circuit or programmed into an FPGA to form a physical implementation of the microarchitecture definition in relation to one another and in the manner in which they interact. Thus, the microarchitecture definition is recognized by those skilled in the art as a structure from which many physical implementations can be derived, all of which belong to the broader structure described by the microarchitecture definition. That is, a person skilled in the art who receives the microarchitecture definition provided according to the present disclosure can implement the structure by coding the descriptions of the circuits / units / components in a hardware description language (HDL), such as Verilog or VHDL, without excessive experimentation and by applying ordinary techniques. HDL descriptions are often expressed in a manner that can be seen as functional. However, to those skilled in the art, such HDL descriptions are a method used to translate the structure of a circuit, unit, or component into the next level of implementation details. Such HDL descriptions may take the form of behavioral code (which is typically not synthesizable), Register Transfer Language (RTL) code (which, in contrast to behavioral code, is typically synthesizable), or structural code (e.g., a netlist specifying logic gates and their connectivity). HDL descriptions can be sequentially synthesized against a library of cells designed for a given integrated circuit manufacturing technology and modified for timing, power, and other reasons to create a final design database, which can then be sent to a foundry to generate masks and ultimately produce an integrated circuit.Some hardware circuits or parts thereof can also be custom-designed using a schematic editor and captured into an integrated circuit design along with the synthesized circuit parts. Integrated circuits may include transistors and other circuit elements (passive elements such as capacitors, resistors, inductors, etc.) and interconnections between the transistors and circuit elements. Some embodiments may implement multiple integrated circuits combined together to implement hardware circuits, and / or in some embodiments, discrete elements may be used. Alternatively, the HDL design can be synthesized into a programmable logic array, such as a Field Programmable Gate Array (FPGA), and implemented on the FPGA. This decoupling between the design of a group of circuits and the subsequent low-level implementation of these circuits generally leads to a scenario where the circuit or logic designer does not specify a particular set of structures for the low-level implementation beyond a description of what the circuit is configured to do, because this process is performed at a different stage of the circuit implementation process.

[0120] The fact that many different low-level combinations of circuit elements can be used to implement the same specifications of a circuit results in multiple equivalent structures for that circuit. As mentioned, these low-level circuit implementations can vary depending on changes in manufacturing technology, the foundry selected to manufacture the integrated circuit, the library of cells provided for a specific project, and so on. In many cases, the choices made by different design tools or methodologies to generate these different implementations can be arbitrary.

[0121] Furthermore, it is common for a single implementation of a specific functional specification of a circuit to involve a large number of devices (e.g., millions of transistors) for a given embodiment. Consequently, the enormous volume of such information makes it impractical to provide a complete description of the low-level structure used to implement a single embodiment, not to mention the vast array of equivalent possible implementations. For this reason, the present disclosure describes the structure of circuits using functional shorthand commonly used in the industry.

[0122] Once the above disclosure is fully understood, numerous variations and modifications will be apparent to those skilled in the art. The following claims are intended to be interpreted as encompassing all such variations and modifications.

Claims

Claim 1 As a device, a plurality of coherent agent circuits each comprising caches configured to cache memory data; a control circuit configured to implement a directory that tracks which of the plurality of coherent agent circuits caches copies of a cache block and the status of one or more cached copies of the cache block, wherein a first agent circuit among the plurality of coherent agent circuits is configured to transmit a cache-read-exclusive-data-only message to the control circuit, and the cache-read-exclusive-data-only message is, In response to a decision that any cached copy of the above cache block is modified, a forward of a copy of the above cache block is requested; Requesting no forward of any copy of the cache block in response to a decision that the cached copies of the cache block are not modified; the control circuit is further configured to cause one or more other agent circuits among the plurality of coherent agent circuits to invalidate their copies of the cache block in response to the cache-read-exclusive-data-only message; and the first agent circuit is further configured to modify the entire cache block in response to the cache-read-exclusive-data-only message, a device. Claim 2 An apparatus according to claim 1, wherein the control circuit is further configured to cause the second agent circuit to forward the modified copy of the cache block to the first agent circuit in response to a determination that the second agent circuit among the plurality of coherent agent circuits has a modified copy of the cache block. Claim 3 In paragraph 2, the device is configured such that the second agent circuit transmits a fill completion along with the forward of the modified copy of the cache block. Claim 4 A device according to paragraph 3, wherein a third agent circuit among the plurality of coherent agent circuits is configured to transmit an acknowledgment completion based on the cache-read-exclusive-data-only message in a situation where the third agent circuit does not have a modified copy of the cache block. Claim 5 The apparatus of claim 1, wherein the control circuit unit forwards the first modified copy of the cache block to the first agent circuit in response to a determination that a second agent circuit among the plurality of coherent agent circuits has a first modified copy of the cache block in a first shared state and a third agent circuit among the plurality of coherent agent circuits has a second modified copy of the cache block in a second shared state; and the invalidation of the copies of the cache block in response to the cache-read-exclusive-data-only message comprises invalidating the first modified copy and the second modified copy. Claim 6 As a method, the step of caching memory data in their respective caches by a plurality of coherent agent circuits; the step of implementing a directory by a control circuit that tracks which of the plurality of coherent agent circuits is caching copies of a cache block and the status of one or more cached copies of the cache block; the step of transmitting a cache-read-exclusive-data-only message to the control circuit by a first agent circuit among the plurality of coherent agent circuits, wherein the cache-read-exclusive-data-only message is, In response to a decision that any cached copy of the above cache block is modified, a forward of a copy of the above cache block is requested; A method comprising: requesting no forwarding of any copy of the cache block in response to a decision that the cached copies of the cache block are not modified; causing one or more other agent circuits among the plurality of coherent agent circuits to invalidate their copies of the cache block in response to the cache-read-exclusive-data-only message by the control circuit; and modifying the entire cache block in response to the cache-read-exclusive-data-only message by the first agent circuit. Claim 7 A method according to claim 6, further comprising the step of, in response to a determination by the control circuit that a second agent circuit among the plurality of coherent agent circuits has a modified copy of the cache block, causing the second agent circuit to forward the modified copy of the cache block to the first agent circuit. Claim 8 A method according to claim 7, further comprising the step of transmitting a fill completion along with the forward of the modified copy of the cache block by the second agent circuit. Claim 9 A method according to claim 8, further comprising the step of transmitting an acknowledgment completion based on the cache-read-exclusive-data-only message by the third agent circuit among the plurality of coherent agent circuits, in a situation where the third agent circuit does not have a modified copy of the cache block. Claim 10 A method according to claim 6, further comprising the step of forwarding the first modified copy of the cache block to the first agent circuit in response to a determination that among the plurality of coherent agent circuits, the second agent circuit has a first modified copy of the cache block in a first shared state and among the plurality of coherent agent circuits, the third agent circuit has a second modified copy of the cache block in a second shared state. Claim 11 As a device, a plurality of coherent agent circuits each comprising caches configured to cache memory data; a control circuit, wherein the control circuit comprises, Implementing a directory that tracks which of the plurality of coherent agent circuits is caching copies of a cache block, and the status of one or more cached copies of the cache block; Sequencing messages from the aforementioned multiple coherent agent circuits and processing the messages according to the said sequence; Receiving a conditionally exclusive change message for the cache block from the first agent circuit among the plurality of coherent agent circuits; In response to the directory indicating that the first agent circuit has a valid copy of the cache block at the time when the above conditionally exclusive change message is processed, the first agent circuit acquires an exclusive state for the first agent circuit regarding the cache block; A device configured to return a failure message to the first agent circuit in response to the directory indicating that the first agent circuit does not have a valid copy of the cache block at the time when the above conditionally exclusive change message is processed. Claim 12 A device according to claim 11, wherein the control circuit is configured to issue one or more snoops to obtain the exclusive state. Claim 13 In paragraph 12, the device comprises one or more snoops, wherein the snoops include: a forward request to the second agent circuit among the plurality of coherent agent circuits based on a determination that the second agent circuit has a copy of the cache block in a primary shared state; and a nullification to the third agent circuit among the plurality of coherent agent circuits based on a determination that the third agent circuit has a copy of the cache block in a different state. Claim 14 In claim 11, the device is configured such that the control circuit is configured to generate a fill operation to obtain the exclusive state. Claim 15 A device according to claim 11, wherein the first agent circuit is configured to transmit a change message to the conditionally exclusive based on the execution of a store conditional instruction. Claim 16 A method comprising: caching memory data in their respective caches by a plurality of coherent agent circuits; implementing a directory by a control circuit that tracks which of the plurality of coherent agent circuits is caching copies of a cache block and the state of one or more cached copies of the cache block; ordering messages from the plurality of coherent agent circuits and processing the messages according to the ordering by the control circuit; receiving a conditionally exclusive change message for the cache block from a first agent circuit among the plurality of coherent agent circuits by the control circuit; and acquiring an exclusive state for the first agent circuit for the cache block by the control circuit in response to the directory indicating that the first agent circuit has a valid copy of the cache block at the time the conditionally exclusive change message is processed. Claim 17 A method according to claim 16, further comprising the step of returning a failure message to the first agent circuit in response to the directory indicating that the first agent circuit does not have a valid copy of the cache block at the time when the second conditionally exclusive change message from the first agent circuit to the cache block is processed by the control circuit. Claim 18 In paragraph 16, the method comprises the step of obtaining one or more snoops. Claim 19 In paragraph 18, the method comprises one or more snoops including: a forward request to the second agent circuit among the plurality of coherent agent circuits based on a determination that the second agent circuit has a copy of the cache block in a primary shared state; and a nullification to the third agent circuit among the plurality of coherent agent circuits based on a determination that the third agent circuit has a copy of the cache block in a different state. Claim 20 In paragraph 16, the method comprises the step of obtaining the above-mentioned acquisition, which includes the step of generating a fill operation. Claim 21 In paragraph 16, the above conditionally exclusive change message is transmitted based on a storage conditional command.

Images