High-performance coupling structure coherence protocol

The HPI architecture addresses communication inefficiencies in complex computer systems by implementing a layered protocol stack with a coherence protocol, enhancing data coherence and reducing latency for high-performance computing platforms.

DE112013007764B4Active Publication Date: 2024-11-28INTEL CORP
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
DE112013007764
Authority / Receiving Office
DE · DE
Patent Type
Patents
Current Assignee / Owner
Priority Date
2012-10-22
Filing Date
2013-03-15
Publication Date
2024-11-28
Estimated Expiration
2033-03-15

AI Technical Summary

Technical Problem

As computer systems evolve with increased complexity and demand for higher performance and energy efficiency, existing interconnect architectures struggle to manage communication between multiple processors and components, leading to inefficiencies and potential deadlocks in data coherence protocols.

Method used

The High-Performance Interconnect (HPI) architecture introduces a layered protocol stack with a coherence protocol layer to ensure data consistency and efficient communication, utilizing point-to-point serial links and virtual channels to manage transactions and caching, supporting multiple devices and platforms, including PCIe and QPI.

Benefits of technology

The HPI architecture enhances data coherence and reduces latency, ensuring efficient communication and resource management across complex computer systems, supporting high-performance computing platforms like workstations and servers.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

Device comprising the following: a node comprising at least one processor, one cache and one first coherence agent (705) to: to receive a snoop request, wherein the snoop request is to be received by a home agent (710), wherein the snoop request includes a snoop invalidation request, wherein the snoop request is to be equivalent to a request sent to the home agent (710) by a second coherence agent (720), and wherein the snoop request includes an address field encoded with a node identifier (NID) of the home agent (710); to generate a snoop response, wherein the snoop response comprises a command field, a destination NID field, a home transaction identifier (TID) field, and data, wherein the command field is to be coded to indicate that the snoop response includes an implicit write-back, wherein the destination NID field is coded with the NID of the home agent (710), wherein the home TID field is coded to identify a resource allocated by the home agent (710) to process the snoop request, and wherein the data is to be written back to a memory (715); to send Snoop's reply to the home agent (710); and to change the state of the cache based on the Snoop response.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to the field of computer engineering, and more particularly to software engineering involving the coordination of mutually dependent constrained systems. BACKGROUND

[0002] Advances in semiconductor processing and logic design have led to an increase in the amount of logic that can be present in integrated circuit devices. As a result, computer system configurations have evolved from a single or multiple integrated circuits in a system to multiple cores, multiple hardware threads, and multiple logical processors present in individual integrated circuits, as well as other interfaces integrated within such processors. A processor or integrated circuit typically comprises a single physical processor die, where the processor die can include any number of cores, hardware threads, or logical processors, interfaces, memory, controller hubs, etc.

[0003] As a result of the increased ability to pack more computing power into smaller packages, smaller computing devices have grown in popularity. Smartphones, tablets, ultra-thin notebooks, and other user terminals have grown exponentially. However, these smaller devices depend on servers for both data storage and complex processing that exceeds the form factor. Therefore, demand in the high-performance computing market (i.e., server memory space) has also increased. For example, modern servers typically contain not just a single multi-core processor, but also multiple physical processors (also known as multi-socket processors) to increase computing power. But as computing power increases along with the number of devices in a computer system, communication between sockets and other components becomes more critical.

[0004] In fact, interconnects have evolved from more traditional multi-drop busses, primarily handling electrical communications, to fully mature interconnect architectures that facilitate high-speed communication. Unfortunately, these requirements are being mapped to the capabilities of existing interconnect architectures, while the demand for future processors with even higher power consumption rates is increasing.

[0005] US 5,644,753 A describes a multiprocessor computer system with data processors and a main memory connected to a system controller. Each data processor has a cache memory. Each cache memory has a cache controller with two ports for receiving access requests. A first port receives access requests from the associated data processor, and a second port receives access requests from the system controller. All cache access requests contain an address value; access requests from the system controller also contain a mode flag.

[0006] US 2008 / 0109565A1 describes a method and apparatus for improving / extending a serial point-to-point interconnect architecture, such as Peripheral Component Interconnect Express (PCIe). Temporal and spatial caching hints and prefetching hints are provided to improve system-wide caching and prefetching. Message codes for atomic operations to mediate ownership between system devices / resources are included to ensure efficient access / ownership of shared data.

[0007] US 5,524,234 A describes a write-back coherence system for implementing write-back caching in an x86 processor installed in a multi-master computer system that does not support a write-back protocol for maintaining coherence between an internal cache and main memory during DMA operations. The write-back coherence system interrupts normal bus arbitration to allow the export of dirty data and includes an X%DIRTY latency control function. If the internal cache contains dirty data in response to an arbitration request (e.g., HOLD), the processor is prevented from providing an arbitration acknowledgement (e.g., HLDA) until the dirty data is exported (the cache is dynamically switched to write-through mode to prevent data in the cache from becoming dirty while the bus is arbitrated away).While the requesting bus master accesses memory, bus snooping is performed, and the invalidation logic invalidates at least the cache locations corresponding to the memory locations selected by the requesting bus master.

[0008] The invention is defined in the independent claims. Embodiments of the invention are described in the dependent claims. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 illustrates a simplified block diagram of a system including a serial point-to-point interconnect for connecting I / O devices in a computer system according to one embodiment; Fig. 2 illustrates a simplified block diagram of a layered protocol stack according to one embodiment; Fig. 3 illustrates one embodiment of a transaction descriptor. Fig.Figure 4 illustrates one embodiment of a serial point-to-point link. Fig. Figure 5 illustrates embodiments of potential high performance interconnect (HPI) system configurations. Fig. Figure 6 illustrates one embodiment of a layered protocol stack associated with an HPI. Fig. Figure 7 illustrates a flowchart of an example coherence protocol conflict management. Fig. Figure 8 illustrates a flowchart of another example coherence protocol conflict management. Fig. Figure 9 illustrates a flowchart of another example coherence protocol conflict management. Fig. Figure 10 illustrates a flow diagram of an example snoop response with write-back to memory. Fig. Figure 11 illustrates a flow diagram of another example of a snoop response with write-back to memory. Fig. 12 illustrates a flow diagram of an example write-back shift operation. Fig. 13 illustrates a flow diagram of an example memory write-back. Fig. 14 illustrates a flow diagram of an example memory controller flush operation. The Fig. Figures 15 to 17 illustrate representations of example protocol state tables. Fig. Figure 18 illustrates a representation of an example nesting of protocol state tables. Fig. Figure 19 illustrates a representation of the use of a set of protocol state tables by an example test engine. Fig. Figure 20 illustrates a representation of the use of a set of protocol state tables by an example test engine. Fig.21 illustrates one embodiment of a block diagram for a computer system including a multi-core processor.

[0009] The same reference numbers and designations in the different drawings refer to similar elements. DETAILED DESCRIPTION

[0010] In the following description, numerous specific details are set forth, such as examples of particular types of processors and system configurations, particular hardware arrangements, particular architectural and microarchitectural details, particular register configurations, particular instruction types, particular system components, particular processor pipeline stages, particular interconnect layers, particular packet / transaction configurations, particular transaction names, particular protocol exchanges, particular link widths, particular implementations and operations, etc., in order to provide a thorough understanding of the present invention. However, it will be apparent to one skilled in the art that these specific details need not be employed to practice the subject matter of the present disclosure. In other instances, a detailed description of well-known components or methods has been provided, such as those described in the prior art.Specific and alternative processor architectures, specific logic circuits / code for described algorithms, specific firmware code, specific low-level interconnect operations, specific logic configurations, specific manufacturing techniques and materials, specific compiler implementations, specific algorithm-to-code translation, specific power-down and gating techniques / logic, and other specific computer system operating details have been avoided in order to avoid unnecessarily obscuring the present invention.

[0011] Although the following embodiments may be described with reference to energy conservation, power efficiency, processing efficiency, and so on in specific integrated circuits such as computer platforms or microprocessors, other embodiments are applicable to other types of integrated circuits and logic assemblies. Similar techniques and teachings of embodiments described herein may be applied to other types of circuits or semiconductor devices that may also benefit from these features. For example, the disclosed embodiments are not limited to server computer systems, desktop computer systems, laptops, Ultrabooks™, but may also be used in other devices, such as handheld devices, smartphones, tablets, other thin notebooks, system on a chip (SOC) devices, and embedded applications. Some examples of handheld devices include:Mobile phones, Internet Protocol devices, digital cameras, personal digital assistants (PDAs), and handheld PCs. Here, similar techniques for a high-performance interconnect can be applied to increase performance (or even save energy) in a low-power interconnect. Embedded applications typically include, among others, a microcontroller, digital signal processor (DSP), system on a chip, network computers (NetPC), set-top boxes, network hubs, wide area network (WAN) switches, or other systems capable of performing the functions and operations taught below. Furthermore, the apparatus, methods, and systems described herein are not limited to physical computing devices but may also pertain to software optimizations for energy conservation and efficiency.As will be readily apparent from the following description, the embodiments of methods, apparatus, and systems described herein (whether with reference to hardware, firmware, software, or a combination thereof) may be considered critical to a "green technology" future, balanced with performance considerations.

[0012] As computer systems evolve, the components within them become more complex. The interconnect architecture used to interconnect and communicate between components has also grown in complexity to ensure that bandwidth demand is met for optimal component operation. Furthermore, different market segments require different aspects of interconnect architectures, making them suitable for the respective market. For example, servers require higher performance, while the mobile ecosystem is sometimes able to sacrifice overall performance for power savings. Yet, a singular purpose of most fabrics is to deliver the highest possible performance with maximum power savings. Furthermore, a variety of different interconnect structures can potentially benefit from the subject matter described here.For example, the Peripheral Component Interconnect (PCI) Express (PCIe) interconnect fabric architecture and the QuickPath Interconnect (QPl) fabric architecture, among other examples, may potentially be enhanced according to one or more of the principles described herein, among other examples.

[0013] Fig.1 illustrates one embodiment of a fabric consisting of point-to-point links connecting a set of components. System 100 includes processor 105 and system memory 110 coupled to controller hub 115. Processor 105 may include any processing element, such as a microprocessor, a host processor, an embedded processor, a coprocessor, or another processor. Processor 105 is coupled to controller hub 115 via front-side bus (FSB) 106. In one embodiment, FSB 106 is a serial point-to-point interconnect, as described below. In another embodiment, link 106 includes a serial differential interconnect architecture conforming to a different interconnect standard.

[0014] System memory 110 includes any memory device, such as random access memory (RAM), non-volatile (NV) memory, or other memory accessible to the components of system 100. System memory 110 is coupled to controller hub 115 via memory interface 116. Examples of a memory interface include a dual data rate (DDR) memory interface, a dual-channel DDR memory interface, and a dynamic random access memory (DRAM) memory interface.

[0015] In one embodiment, controller hub 115 may include a root hub, root complex, or root controller as in a PCIe interconnect hierarchy. Examples of a controller hub 115 include a chipset, memory controller hub (MCH), a northbridge, an interconnect controller hub (ICH), a southbridge, and a root controller / hub. Often, the term chipset refers to two physically separate controller hubs, such as a memory controller hub (MCH) coupled to an interconnect controller hub (ICH). Note that current systems often include the MCH integrated within processor 105, while controller 115 communicates with I / O devices in a similar manner as described below. In some embodiments, peer-to-peer routing is optionally supported by root complex 115.

[0016] Here, controller hub 115 is coupled to switch / bridge 120 via serial link 119. I / O modules 117 and 121, which may also be referred to as interfaces / ports 117 and 121, may include / implement a layered protocol stack for providing communication between controller hub 115 and switch 120. In one embodiment, multiple devices are capable of being coupled to switch 120.

[0017] Switch / bridge 120 routes packets / messages from device 125 upstream, i.e., up one hierarchy toward a root complex to controller hub 115, and downstream, i.e., down one hierarchy away from a root controller of processor 105 or system memory 110, to device 125. Switch 120, in one embodiment, is referred to as a logical assembly of multiple virtual PCI-to-PCI bridge devices. Device 125 includes any internal or external device or component coupled to an electronic system, such as a memory controller. E.g., to an I / O device, a network interface controller (NIC), an add-in card, an audio processor, a network processor, a hard disk drive, a storage device, a CD / DVD-ROM, a monitor, a printer, a mouse, a keyboard, a router, a portable storage device, a Firewire device, a Universal Serial Bus (USB) device, a scanner, and other input / output devices.In PCIe jargon, such a device is often referred to as an endpoint. Although not specifically shown, device 125 may include a bridge (e.g., a PCIe-to-PCI / PCI-X bridge) to support legacy or other versions of devices or interconnect fabrics supported by such devices.

[0018] A graphics accelerator 130 may also be coupled to the controller hub 115 through a serial link 132. In one embodiment, the graphics accelerator 130 is coupled to an MCH, which is coupled to an ICH. The switch 120, and accordingly, I / O device 125, is then coupled to the ICH. The I / O modules 131 and 118 also implement a multi-layer protocol stack and associated logic for communicating between the graphics accelerator 130 and the controller hub 115. Similar to the MCH discussion above, a graphics controller or the graphics accelerator 130 itself may be integrated into the processor 105.

[0019] With reference now to Fig. 2 illustrates one embodiment of a layered protocol stack. The layered protocol stack 200 may include any form of layered communication stack, such as a QPI stack, a PCIe stack, a next-generation HPI stack, or another layered stack. In one embodiment, the protocol stack 200 may include the transaction layer 205, the link layer 210, and the physical layer 220. An interface such as interfaces 117, 118, 121, 122, 126, and 131 in Fig. 1 may be represented as a communication protocol stack 200. The representation as a communication protocol stack may also be referred to as a module or interface that implements / includes a protocol stack.

[0020] Packets can be used to communicate information between components. Packets can be formed in the transaction layer 205 and the data link layer 210 to transport the information from the sending component to the receiving component. As the sent packets move through the other layers, they are augmented with additional information necessary to handle packets at those layers. On the receiving side, the reverse process occurs, and the packets are converted from their physical layer 220 representation to the data link layer 210 representation, and finally (for transaction layer packets) into the form that can be processed by the transaction layer 205 of the receiving device.

[0021] In one embodiment, the transaction layer 205 may provide an interface between a processor core of a device and the interconnect architecture such as the data link layer 210 and the physical layer 220. In this regard, a primary responsibility of the transaction layer 205 may include assembling and disassembling packets (i.e., transaction layer packets or TLPs). The translation layer 205 may also manage credit-based flow control for TLPs. In some implementations, split transactions—i.e., transactions where the request and response are separated by time—may be used, allowing a link, among other examples, to carry other traffic while the destination device gathers data for the response.

[0022] Credit-based flow control can be used to implement virtual channels and networks that use the interconnect fabric. In one example, a device can offer an initial amount of credits for each of the receive buffers in the transaction layer 205. An external device at the opposite end of the link, such as the controller hub 115 in Fig. 1, can count the number of credits consumed by each TLP. A transaction can be sent if the transaction does not exceed a credit limit. After receiving a response, a credit amount is restored. One example of an advantage, among other potential benefits, of such a credit scheme is that the latency of credit return does not affect performance, provided the credit limit is not reached.

[0023] In one embodiment, four transaction address spaces may include a configuration address space, a memory address space, an input / output address space, and a message address space. Memory space transactions include one or more of read requests and write requests to transfer data to / from a memory-mapped location. In one embodiment, memory space transactions are capable of using two different command types, such as a short address format such as a 32-bit address or a long address format such as a 64-bit address. Configuration space transactions may be used to access configuration space from various devices connected to the interconnect. Transactions related to configuration space may include read requests and write requests.Message space transactions (or simply messages) may also be defined to support in-band communication between interconnection fabric agents. Therefore, in an exemplary embodiment, the transaction layer 205 may assemble packet headers / payloads 206.

[0024] With brief reference to Fig.Figure 3 illustrates an exemplary embodiment of a transaction-layer packet descriptor. In one embodiment, transaction descriptor 300 may be a mechanism for transporting transaction information. In this regard, transaction descriptor 300 supports the identification of transactions within a system. Other possible uses include tracking modifications to standard transaction ordering and linking transactions to channels. For example, transaction descriptor 300 may include global identifier field 302, attribute field 304, and channel identifier field 306. In the illustrated example, global identifier field 302 is shown as comprising local transaction identifier field 308 and source identifier field 310. In one embodiment, global transaction identifier 302 is unique for all outstanding requests.

[0025] According to one implementation, the local transaction identifier field 308 is a field generated by a requesting agent and can be unique for all outstanding requests requiring completion for that requesting agent. Furthermore, in this example, the source identifier 310 uniquely identifies the requesting agent within a coupling structure hierarchy. Accordingly, the local transaction identifier field 308, together with the source ID 310, provides the global identification of a transaction within a hierarchy domain.

[0026] The attribute field 304 specifies properties and relationships of the transaction. In this regard, the attribute field 304 is potentially used to provide additional information that enables modification of the default handling of transactions. In one embodiment, the attribute field 304 includes the priority field 312, the reserved field 314, the ordering field 316, and the no-snoop field 318. Here, the priority subfield 312 can be modified by an initiator to assign a priority for the transaction. The reserved attribute field 314 remains reserved for future or vendor-defined use. Possible usage models that utilize priority or security attributes can be implemented using the reserved attribute field.

[0027] In this example, the ordering attribute field 316 is used to provide optional information that conveys the type of ordering that can modify the default ordering rule. According to an example implementation, an ordering attribute of "0" indicates that standard ordering rules apply, where an ordering attribute of "1" indicates a relaxed ordering where writes can be passed in the same direction as writes, and read access executions can be passed in the same direction as writes. The snoop attribute field 318 is used to determine whether transactions are snooped. As shown, the channel identifier field 306 specifies a channel to which a transaction is associated.

[0028] For discussion of Fig.Returning to Figure 2, a link layer 210, also referred to as a data link layer 210, may act as an intermediate layer between the transaction layer 205 and the physical layer 220. In one embodiment, a responsibility of the data link layer 210 is to provide a reliable mechanism for exchanging transaction layer packets (TLPs) between two components on a link. One side of the data link layer 210 accepts TLPs assembled by the transaction layer 205, applies the packet sequence identifier 211, i.e., an identification number or packet number, calculates and applies an error detection code, i.e., CRC 212, and submits the modified TLPs to the physical layer 220 for transmission over a physical layer to an external device.

[0029] In one example, physical layer 220 includes logical sub-block 221 and electrical sub-block 222 for physically transmitting a packet to an external device. Here, logical sub-block 221 is responsible for the "digital" functions of physical layer 221. In this regard, the logical sub-block may include a transmit portion to prepare outgoing information for transmission by physical sub-block 222 and a receive portion to determine and prepare received information before passing it to link layer 210.

[0030] The physical block 222 includes a transmitter and a receiver. The transmitter is supplied with symbols by the logical sub-block 221, which the transmitter serializes and sends to a peripheral device. The receiver is supplied with serialized symbols from a peripheral device and converts the received signals into a bit stream. The bit stream is deserialized and provided to the logical sub-block 221. In one exemplary embodiment, an 8b / 10b transmission code is used, with ten-bit symbols being transmitted / received. Here, special symbols are used to form a packet containing the frames 223. In one example, the receiver also provides a symbol clock recovered from the incoming serial stream.

[0031] Although transaction layer 205, link layer 210, and physical layer 220 are described as indicated above with reference to a specific embodiment of a protocol stack (such as a PCIe protocol stack), a layered protocol stack is not limited in this regard. Indeed, any layered protocol may be included / implemented and adopt features described herein. As an example, a port / interface depicted as a layered protocol may include: (1) a first layer to assemble packets, i.e., a transaction layer; a second layer to sequence packets, i.e., a link layer; and a third layer to transmit the packets, i.e., a physical layer. As a specific example, an HPI layered protocol is used as described hereinafter.

[0032] With reference now to Fig.Figure 4 illustrates an exemplary embodiment of a point-to-point serial fabric. A point-to-point serial link may include any transmission path for sending serial data. In the illustrated embodiment, a link may include two differentially driven low-voltage signal pairs: a transmit pair 406 / 411 and a receive pair 412 / 407. Accordingly, device 405 includes transmit logic 406 for sending data to device 410 and receive logic 407 for receiving data from device 410. In other words, in some implementations of a link, two transmit paths, i.e., paths 416 and 417, and two receive paths, i.e., paths 418 and 419, are included.

[0033] A transmission path refers to any path for sending data, such as a broadcast wire, a copper wire, an optical line, a wireless communication channel, an infrared communication link, or another communication path. A connection between two devices, such as device 405 and device 410, is referred to as a link, such as link 415. A link can support one lane—each lane representing a set of differential signal pairs (one pair for transmit, one pair for receive). To scale bandwidth, a link can aggregate multiple lanes, denoted xN, where N is any supported link width such as 1, 2, 4, 8, 12, 16, 32, 64, or wider.

[0034] A differential pair can refer to two transmission paths, such as lines 416 and 417, for transmitting differential signals. For example, when line 416 transitions from a low voltage level to a high voltage level, i.e., a rising edge, line 417 transitions from a high logic level to a low logic level, i.e., a falling edge. Differential signals have potentially better electrical properties, such as better signal integrity, i.e., crosstalk, voltage overshoot / undershoot, and ringing, among other exemplary advantages. This allows for a better timing window, enabling faster transmit frequencies.

[0035] In one embodiment, a new HPI is provided. The HPI may include a next-generation cache-coherent, link-based interconnect. As an example, the HPI may be used in high-performance computing platforms such as workstations or servers, including systems where PCIe or another interconnect protocol is typically used to connect processors, accelerators, I / O devices, and the like. However, the HPI is not limited in this regard. Instead, the HPI may be used in any of the systems or platforms described herein. Furthermore, the individual ideas developed may be applied to other interconnects and platforms such as PCIe, MIPI, QPI, etc.

[0036] To support multiple devices in an example implementation, the HPI can be instruction set architecture (ISA) agnostic (i.e., the HPI can be implemented on multiple different devices). In another scenario, the HPI can also be used to connect high-performance I / O devices, not just processors or accelerators. For example, a high-performance PCIe device can be coupled to the HPI through a suitable translation bridge (i.e., HPI to PCIe). Furthermore, the HPI links can be used by many HPI-based devices, such as processors, in a variety of ways (e.g., stars, rings, meshes, etc.). Fig.Figure 5 illustrates example implementations of several potential multi-socket configurations. A two-socket configuration 505, as shown, may include two HPI links; however, in other implementations, one HPI link may be used. For larger topologies, any configuration may be used as long as an identifier (ID) is assignable and there is some form of a virtual path, among other additional or replacement features. As shown in one example, a four-socket configuration 510 has an HPI link from each processor to another. However, in the eight-socket implementation shown in configuration 515, not every socket is directly connected by an HPI link. However, if a virtual path or channel exists between the processors, the configuration is supported. A range of supported processors includes 2-32 in a native domain.Higher numbers of processors can be achieved, among other examples, by using multiple domains or other coupling structures between node controllers.

[0037] The HPI architecture includes a definition of a layered protocol architecture, which in some examples includes protocol layers (coherent, incoherent, and optionally other memory-based protocols), a routing layer, a link layer, and a physical layer. Furthermore, the HPI can further include extensions related to power managers (such as power control units (PCUs)), design for test and debug (DFT), error handling, registers, and security, among other examples. Fig. Figure 6 illustrates one embodiment of an exemplary HPI layer protocol stack. In some implementations, at least some of the Fig.6 may be optional. Each layer deals with its own level of granularity or amount of information (the protocol layer 605a,b with the packets 630, the link layer 610a,b with the flits 635, and the physical layer 605a,b with the flits 640). Note that in some embodiments, a packet may include partial flits, a single flit, or multiple flits based on the implementation.

[0038] As a first example, a phit 640 width implies a 1-to-1 mapping of link width to bits (e.g., a 20-bit link width implies a phit of 20 bits, etc.). Flits can be larger in size, such as 184, 192, or 200 bits. Note that if phit 640 is 20 bits wide and the size of flit 635 is 184 bits, then it will take a fraction of phits 640 to transmit a flit 635 (e.g., 9.2 phits at 20 bits to transmit a 184-bit flit 635, or 9.6 at 20 bits to transmit a 192-bit flit, among other examples). Note that elementary link widths can vary at the physical layer. For example, the number of lanes per instruction may include 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, etc. In one embodiment, the link layer 610a,b is capable of embedding multiple parts of different transactions in a single flit, and one or more headers (e.g.,1, 2, 3, 4) can be embedded within the flit. In one example, the HPI splits the headers into corresponding slots to allow for multiple messages in the flit intended for different nodes.

[0039] The physical layer 605a,b, in one embodiment, may be responsible for the rapid transmission of information on the physical medium (electrical or optical, etc.). The physical link may be point-to-point between two link-layer entities, such as layers 605a and 605b. The link layer 610a,b may abstract the physical layer 605a,b from the upper layers and provides the ability to reliably transmit data (as well as requests) and manage flow control between two directly connected entities. The link layer may also be responsible for virtualizing the physical channel into multiple virtual channels and message classes. The protocol layer 620a,b relies on the link layer 610a,b to map protocol messages into the appropriate message classes and virtual channels before passing them to the physical layer 605a,b for transmission over the physical links.The link layer 610a,b may support multiple messages such as a request, snoop response, writeback, incoherent data, among other examples.

[0040] The physical layer 605a,b (or PHY) of the HPI may be implemented above the electrical layer (i.e., electrical conductors connecting two components) and below the link layer 610a,b, as illustrated in Fig.6. The physical layer and corresponding logic can be located at each agent and connect the link layers separately for two agents (A and B) (e.g., for devices on either side of a link). The local and remote electrical layers are connected by physical media (e.g., wires, conductors, optical, etc.). The physical layer 605a,b, in one embodiment, has two essential phases: initialization and operation. During initialization, the connection to the link layer is opaque, and signaling may involve a combination of timed states and handshake events. During operation, the connection to the link layer is transparent, and signaling occurs at a speed at which all lanes operate together as a single link. During the operation phase, the physical layer transports flits from agent A to agent B and from agent B to agent A.The connection is also referred to as a link and abstracts some physical aspects, including media, width, and speed, from the link layers, while flits and control / state of the current configuration (e.g., width) are exchanged with the link layer. The initialization phase includes sub-phases, such as polling and configuration. The operation phase includes sub-phases (e.g., link power management states).

[0041] In one embodiment, the link layer 610a,b may be implemented to provide reliable data transfer between two protocol or routing entities. The link layer may abstract the physical layer 605a,b from the protocol layer 620a,b and may be responsible for flow control between two protocol agents (A, B) and may provide virtual channel services to the protocol layer (message classes) and routing layer (virtual networks). The interface between the protocol layer 620a,b and the link layer 610a,b may typically be at the packet level. In one embodiment, the smallest transfer unit at the link layer is referred to as a flit with a certain number of bits, such as 192 bits, or by some other name.The link layer 610a,b relies on the physical layer 605a,b to form the transmission unit of the physical layer 605a,b (phit) into the transmission unit of the link layer 610a,b (flit). Furthermore, the link layer 610a,b may be logically broken into two parts, a sender and a receiver. A sender / receiver pair at one entity may be connected to a receiver / sender pair at another entity. Flow control is often performed on both a flit and packet basis. Error detection and correction is also potentially performed on a flit-level basis.

[0042] In one embodiment, the routing layer 615a,b may provide a flexible and distributed method for routing HPI transactions from a source to a destination. The scheme is flexible because routing algorithms for multiple topologies may be specified by programmable routing tables at each router (programming is performed in one embodiment by firmware, software, or a combination thereof). The routing functionality may be distributed; routing may be performed through a series of routing steps, where each routing step is defined by a table lookup at either the source, intermediate, or destination routers. The lookup at a source may be used to introduce an HPI packet into the HPI fabric. The lookup at an intermediate router may be used to route an HPI packet from an input port to an output port.A lookup to a destination port can be used to address the destination HPI protocol agent. Note that in some implementations, the routing layer may be thin because the routing tables, and therefore the routing algorithms, are not specifically defined by specification. This allows for flexibility and a variety of usage models, including flexible architectural platform topologies to be defined by the system implementation. The routing layer 615a,b relies on the link layer 610a,b to provide for the use of up to three (or more) virtual networks (VNs)—in one example, two deadlock-free VNs, VN0 and VN1, with multiple message classes defined within each virtual network.A shared adaptive virtual network (VNA) may be defined in the link layer, but this adaptive network may not be directly exposed in routing concepts, since each message class and virtual network may have dedicated resources and guaranteed progress, among other features and examples.

[0043] In one embodiment, the HPI may include a coherency protocol layer 620a,b to assist agents in caching data rows from memory. An agent that wishes to cache memory data may use the coherency protocol to read the data row and load it into its cache. An agent that wishes to modify a data row in its cache may use the coherency protocol to acquire ownership of the row before modifying the data. After modifying a row, an agent may follow protocol requests to keep it in its cache until it either writes the row back to memory or includes it in a response to an external request. Finally, an agent may satisfy external requests to invalidate a row in its cache. The protocol ensures data coherency by dictating the rules that all caching agents must follow.It also provides the means for agents without caches to read and write memory data contiguously.

[0044] Two conditions can be enforced to support transactions using the HPI coherence protocol. First, the protocol can maintain data consistency, as an example, on a per-address basis between data in the agent's caches and between that data and the data in memory. Informally, data consistency can refer to any valid data row in an agent's cache representing a most recent value of the data, and data sent in a coherence protocol packet can represent the most recent value of the data at the time it was sent. If no valid copy of the data exists in caches or in transit, the protocol can ensure that the most recent value of the data is in memory. Second, the protocol can provide well-defined commitment points for requests.Commitment points for reads can indicate when the data is usable; and for writes, they can indicate when the written data is globally discoverable and will be loaded by subsequent reads. The protocol can support these commitment points for both cacheable and non-cachable (UC) requests in coherent memory space.

[0045] The HPI coherence protocol can also ensure the progress of coherence requests made by an agent to an address in the coherent memory space. Certainly, transactions can eventually be satisfied and reordered for proper system operation. In some embodiments, the HPI coherence protocol may be unaware of retry to resolve resource allocation conflicts. Therefore, the protocol itself can be defined to avoid circular resource dependencies, and implementations can be careful in their designs to avoid introducing dependencies that could result in deadlocks. Additionally, the protocol can indicate where designs can provide appropriate access to protocol resources.

[0046] Logically, the HPI coherence protocol, in one embodiment, may include three elements: coherence (or caching) agents, home agents, and the HPI interconnect fabric that connects the agents. Coherence agents and home agents may cooperate to achieve data consistency by exchanging messages across the interconnect fabric. Link layer 610a,b and its associated description may provide the details of the interconnect fabric, including how it adheres to the coherence protocol requirements described herein. (Note that the division into coherence agents and home agents is for clarity. A design may include multiple agents of both types within a socket, among other examples, or may combine agent behaviors into a single design unit.)

[0047] In one embodiment, home agents may be configured to protect physical memory. Each home agent may be responsible for a region of coherent memory space. Regions may be non-overlapping, with a single address protected by a home agent, and the home agent regions in a system collectively covering the coherent memory space. For example, each address may be protected by at least one home agent. Therefore, in one embodiment, each address in a coherent memory space of an HPI system may be associated with exactly one home agent.

[0048] Home agents in the HPI coherence protocol, in one embodiment, may be responsible for servicing requests to the coherent memory space. For read (Rd) requests, home agents may generate snoops (Snp), process their responses, send a data response, and send a completion response. For invalidate (Inv) requests, home agents may generate required snoops, process their responses, and send a completion response. For write requests, home agents may transfer the data to memory and send a completion response.

[0049] Home agents can deploy snoops in the HPI coherence protocol and process snoop responses from coherence agents. Home agents can also process forward requests, which are a special snoop response from coherence agents for conflict resolution. When a home agent receives a forward request, it can send a forward response to the coherence agent that generated the forward request (i.e., the agent that detected a conflicting snoop request). Coherence agents can use the ordering of these forward responses and completion responses from the home agent to resolve conflicts.

[0050] A coherence agent can issue supported coherence protocol requests. Requests can be issued to an address in the coherent memory space. Data received for read requests (Rd) other than RdCur can be consistent. Data for RdCur requests can have been consistent when the data packet was generated (although it may be stale during delivery). Table 1 shows an example, non-exhaustive list of potentially supported requests: TABLE 1 name semantics Cache status RdCode Request a cache line in F or S state. F or S Rd Data Request a cache line in E, F, or S state. F or S RdMigr Request a cache line in M, E, F, or S state. M and (F or S) Rdlnv Request a cache line in the E state. If the line was previously cached in the M state, the line is written to memory before E data is delivered. E RdlnvOwn Request a cache line in M ​​or E state. M RdCur Request a non-cacheable snapshot of a cache line. InvItoE Request exclusive ownership of a cache line without receiving any data. M or E InvItoM Request exclusive ownership of a cache line without receiving any data and with the intention of performing a writeback soon afterward. M or E InvXtol Flush a cache line from all caches. The requesting agent must invalidate the line in its cache before issuing this request. WbMtol Write a cache line in M ​​state back to memory and invalidate the line in the cache. M WbMtoS Write a cache line in M ​​state back to memory and transition the line to S state. M and S WbMtoE Write a cache line in M ​​state back to memory and transition the line to E state. M and E WbMtolPtl Write a cache line in M ​​state back to memory according to a byte deallocation mask and transition the line to I state. M WbMtoEPtl Write a cache line in M ​​state back to memory according to a byte deallocation mask, transition the line to E state, and clear the mask of the line in the cache. M and E EvctCln Notification to the home agent that a cache line in E state has been invalidated in the cache. E WbPushMtol Send an M-state line to the home agent and invalidate the line in the cache; the home agent can either write the line back to memory or send it to a local M-state cache agent. M WbFlush Request that Home perform writes to application-specific addresses in its Memory hierarchy flushed. No data is sent with the request.

[0051] The HPI can support a coherence protocol that leverages principles of the MESI protocol. Each cache line can be marked with one or more supported states (e.g., encoded in the cache line). An "M" or "Modified" state can indicate that the cache line value has been modified relative to the value residing in main memory. A line in the M state exists only in that particular state, and the corresponding cache agent may be required to write the modified data back to memory at some future time, such as before permitting any other reading of the (no longer valid) main memory state. A write-back can transition the line from the M state to the E state. The "E" or "Exclusive" state can indicate that the cache line exists only in the current cache, but that its value matches that in main memory.The cache line in the E state may transition to the S state at any time in response to a read request, or it may be changed to the M state by writing to the line. The "S" or "Shared" state may indicate that the cache line may be stored in other caches on the machine and has a value that matches that in main memory. The line may be retired (changed to the I state) at any time. The "I" or "Invalid" state may indicate that a cache line is invalid or unused. Another state may also be supported in the HPI, such as an "F" or shared "forward" state, which indicates that the particular shared line value is forwarded to other caches that also share the line, among other examples.

[0052] Table 2 includes example information that may be included in some Coherence Protocol messages, including snoop, read, and write requests, among other examples: TABLE 2 Field use cmd Message command (or name or command code). addr Address of a coherent cache line. destNID Node ID (NID) of the target (home or coherence) agent. reqNID NID of the requesting coherence agent. peerNID NID of the coherence agent that sent the (forward request) message. reqTID ID of the resource allocated by the requesting agent for the transaction, also known as RTID (or requesting transaction identifier). homeTID ID of the resource allocated by the home agent to process the request, also known as HTID (or home transaction identifier). data A cache line of data. mask Byte mask to qualify the data.

[0053] Snoop messages can be generated by home agents and directed to coherence agents. A snoop virtual (SNP) channel can be used for snoops and, in one embodiment, is the only message that uses the SNP virtual channel. Snoops can include the NID and RTID the requesting agent assigns for the request in the case that the snoop results in data being sent directly to the requesting agent. Snoops can, in one embodiment, also include the HTID assigned by the home agent to process the request. The coherence agent processing the snoop can include the HTID in the snoop response it sends back to the home agent. Snoops cannot, in one case, include the home agent's NID, as it can be derived from the enclosed address the addressed coherence agent creates when sending the response.Fanout snoops (those with the "SnpF" prefix) cannot include a destination NID, as the routing layer is responsible for generating the appropriate snoop messages to all peers in the fanout region. An example list of snoop channel messages is shown in Table 3. TABLE 3 command semantics fields SnpCode Snoop to get data in F or S state. cmd, addr, destNID, reqNID, reqTID, homeTID SnpData Snoop to get data in E, F, or S state. SnpMigr Snoop to get data in M, E, F, or S state. SnpInv Snoop to invalidate the peer's cache, flush an M copy into memory. SnpInvOwn Snoop to obtain data in M ​​or E state. SnpCur Snoop to obtain a non-cacheable snapshot of a cache line. SnpFCode Snoop to receive data in F or S state; Routing layer to distribute to all fanout cmd, addr, Treating peers reqNID, reqTID, homeTID SnpFData Snoop to receive data in E, F, or S state; Routing layer to handle distribution to all fanout peers SnpFMigr Snoop to receive data in M, E, F, or S state; Routing layer to handle distribution to all fanout peers SnpFInvOwn Snoop to obtain data in M ​​or E state; routing layer to handle distribution to all fanout peers. SnpFInv Snoop to invalidate the peer's cache, flushing an M copy into memory; routing layer to handle distribution to all fanout peers. SnpCur Snoop to obtain a non-cacheable snapshot of a cache line; routing layer to handle distribution to all fanout peers.

[0054] The HPI can also support non-snoop requests that it can issue to an address, such as those implemented as incoherent requests. Examples of such requests may include, among other potential examples, a non-snoop read to request a read-only row-form memory, a non-snoop write to write a row to memory, and writing a row to memory according to a mask.

[0055] In one example, four general types of response messages can be defined in the HPI coherence protocol: data, completion, snoop, and forward. Certain data messages can carry an additional completion indication, and certain snoop responses can carry data. Response messages can use the RSP virtual channel, and the communication fabric can manage appropriate message ordering among ordered completion responses and forward responses.

[0056] Table 4 includes a list of at least some potential response messages supported by an example HPI coherence protocol: TABLE 4 name semantics fields Data_M Data is M-state. cmd, destNID, reqTID, Data_E Data is E-state. Data_F Data is F-state. data Data_SI Depending on the request, data in S state or non-cacheable "snapshot" data. Data_M Data is M-state with an ordered final response. Data_E Data is E-state with an ordered final response. Data_F Data is F-state with an ordered final response. Data_SI Depending on the request, data in S state or non-cacheable "snapshot" data with an ordered final response. CmpU Final message without order requirements. cmd, destNID, reqTID CmpO Closing message to be sorted with forward replies. Rspl Cache is in I state. cmd, destNID, homeTID RspS Cache is in S state. RspFwd The copy of the cache line was sent to the requesting agent, cache state has not changed. RspFwdl The copy of the cache line has been sent to the requesting agent, cache goes into I state. RspFwdS The copy of the cache line has been sent to the requesting agent, cache goes into S state. RsplWb Modified line is implicitly written back to memory, cache was allowed to transition to I state. cmd, destNID, homeTID, data RspSWb Modified line is implicitly written back to memory, cache was allowed to transition to S state. RspFwdlWb Modified line is implicitly written back to memory, the copy of the cache line was sent to the requesting agent, cache was allowed to transition to I state. RspFwdSWb Modified line is implicitly written back to memory, the copy of the cache line was sent to the requesting agent, cache was transitioned to S state. RspCnfit Peer has a pending request to the same address, requests an ordered forwarding response, and has allocated a resource for forwarding. cmd, destNID, homeTID, peerNID

[0057] In one example, data responses may address a requesting coherence agent. A home agent may send some of the data responses. A coherence agent may only send data responses that do not contain an ordered completion indication. Additionally, coherence agents may be limited to sending data responses only as a result of processing a snoop request. Combined data and completion responses can always be of the ordered completion type and can be kept ordered with forward responses through the communication fabric.

[0058] The HPI coherence protocol can use the general unordered completion message and a coherence-specific ordered completion message. A home agent can send completion responses to coherence requests, and completion responses can typically be intended for a coherence agent. The ordered completion response can be kept in order with forward responses across the communication fabric.

[0059] Snoop responses can be sent by coherence agents specifically in response to processing a snoop request and address the home agent that will handle the snoop request. The destNID is usually a home agent (determined from the address in the snoop request), and the included TID is for the home agent's resource allocated to process the request. Snoop responses with "Wb" in the command are for implicit writebacks of modified cache lines, and they carry the cache line data. (Implicit writebacks can include those made by a coherence agent in response to another agent's request, while the other requests are made explicitly by the coherence agent using its request resources.)

[0060] Coherence agents can generate a forward request when a snoop request conflicts with a pending request. Forward requests address the home agent that generated the snoop, which is determined by the address in the snoop request. Therefore, the destNID is a home agent. The forward request can also include the TID for the home agent's resource allocated to process the original request and the NID of the coherence agent that generated the forward request.

[0061] The HPI coherence protocol can support a single forward response, FwdCnfltO. Home agents can receive a forward response for each forward request and send it to the coherence agent in the peerNID field of the forward request. Forward responses carry the cache line address so that the coherence agent can match the message with the forward resource it has allocated. A forward response message can carry the requesting agent's NID but, in some cases, not the requesting agent's TID. If a coherence agent wants to support cache-to-cache transfers for forward responses, it can store the requesting agent's TID when processing the snoop and send a forward request.To support conflict resolution, the communication fabric can maintain order between the forward response and all ordered completions previously sent to the same target coherence agent.

[0062] In some systems, home agent resources are pre-allocated using "RTIDs" representing resources in the home agents, and the caching agents allocate RTIDs from system-configured pools when generating new coherence requests. Such schemes can limit the number of active requests any particular caching agent has to a home agent to the number of RTIDs given to it by the system, effectively statically partitioning home resources among caching agents. Such schemes can result in inefficient resource allocation, among other potential problems, and proper sizing of a home agent to support request throughput can become impractical for large systems. For example, such schemes can enforce RTID pool management on caching agents. Additionally, in some systems, a caching agent may not reuse the RTID until the home agent has completely processed the transaction.However, waiting for a home agent to complete all processing can unnecessarily throttle caching agents. Additionally, certain flows in the protocol can involve caching agents holding onto RTIDs beyond the home agent release notification, further throttling their performance, among other issues.

[0063] In one implementation, home agents may be allowed to allocate their resources while requests arrive from cache agents. In such cases, home agent resource management may be kept separate from coherence agent logic. In some implementations, home resource management and coherence agent logic may be at least partially intermixed. In some cases, coherence agents may have more outstanding requests to a home agent than the home agent can handle simultaneously. For example, the HPI may allow requests to queue in the communication fabric. To avoid deadlocks caused by the home agent blocking incoming requests until resources become available, the HPI coherence protocol may be further configured to ensure that other messages can progress around blocked requests, ensuring that active transactions reach completion.

[0064] In one example, resource management may be supported by allowing an agent to receive a request to allocate resources and process it, with the agent sending the request allocating appropriate resources for all responses to the request. The HTID may represent the resource a home agent allocates for a given request included in some protocol message. The HTID (together with RNID / RTID) can be used in snoop requests and forward responses to support responses to a home agent and data forwarding to a requesting agent, among other examples. Further, the HPI may support the ability of an agent to send an orderly completion (CmpO) early, that is, before the home agent has finished processing the request, when it has been determined that it is safe for a requesting agent to reuse its RTID resource.The general handling of snoops with similar RNID / RTID can also be defined by the protocol.

[0065] In an illustrative example, a directory state can be used to determine when the home agent can send a response when a tracker state of a particular request is busy. For example, an Invalid Directory state can allow a response to be sent except for RdCur requests, indicating that there are no outstanding snoop responses. An Unknown Directory state can dictate that all peer agents have been snooped and all their responses collected before a response can be sent. The Exclusive Directory state can dictate that the owner has been snooped and all responses collected before a response is sent, or, if the requesting agent is the owner, a response can be sent immediately. The Shared Directory state can specify that an Invalidate request (e.g.,Rdlnv* or Inv*) has snooped all peer agents and collected all snoop responses. If a given request's tracker state is writeback buffered (WbBuffered), the home agent can send a data response. If the request's tracker state is DataSent (indicating that the home agent has already sent a data response) or DataXfrd (indicating that a peer has transferred a copy of the row), the home agent can send the completion response.

[0066] In cases like those described above, a home agent may send data and completion replies before all snoop replies have been collected. The HPI interface allows these "early" replies. When sending early data and completion, the home agent may collect any outstanding snoop replies before releasing the resource it allocated for the request. The home agent may also continue to block standard requests to the same address until all snoop replies have been collected and then release the resource. A home agent sending a reply message from a Busy or WbBuffered state may, among other examples, use a subaction table (e.g., enclosed in a set of protocol tables embodying the formal specification of the HPI Coherence Protocol) to specify which message to send and use a subaction table for, and how to update the directory state.In some cases, early completion can be performed without pre-allocation by a home node.

[0067] In one embodiment, the HPI coherence protocol may omit the use of both, or either, pre-allocated home resources and ordered request channels. In such implementations, certain messages may be ordered on the HPI RSP communication channel. For example, specific "ordered completion" and "forward-response" messages may be provided, which may be sent from the home agent to the coherence agent. Home agents may send an ordered completion (CmpO or Data_*_CmpO) for all coherent read and invalidate requests (as well as other requests such as NonSnpRd requests that are not involved in cache coherence contention).

[0068] Home agents can send forward responses (FwdCnfltO) to coherence agents that send forward requests (RspCnflt) to indicate a conflict. A coherence agent can generate a forward request whenever it has a pending read or invalidate request and detects an incoming snoop request to the same cache line as the request. When the coherence agent receives the forward response, it checks the current state of the pending request to determine how to process the original snoop. The home agent can send the forward response, which should be preceded by a Completed (for example, CmpO or Data_*_CmpO). The coherence agent can use information included in the snoop to assist the coherence agent in processing a forward response. For example, a forward response cannot include "Type" information and an RTID.The type of forwarding response can be inferred from information obtained from the previous snoop(s). Furthermore, a coherence agent can block pending snoop requests if all of its forwarding resources are waiting for forwarding responses. In some implementations, each coherence agent can be designed to have at least one forwarding resource.

[0069] In some implementations, communication fabric requirements may be located at the routing layer. In one embodiment, the HPI coherence protocol includes a communication fabric requirement specific to the routing layer. The coherence protocol may depend on the routing layer to convert a fanout snoop (SnpF* command codes - Snoop (SNP) channel messages) into the appropriate snoops for all peers of the request in the fanout set of coherence agents. The fanout set is a routing layer configuration parameter shared by the protocol layer. In this coherence protocol specification, it is described as a home agent configuration parameter.

[0070] In some implementations mentioned above, the HPI coherence protocol may use four virtual channels: REQ, WB, SNP, and RSP. The virtual channels may be used to manage dependency cycles and avoid deadlocks. In one embodiment, each message may be delivered without duplication on all virtual channels and an ordering requirement on the RSP virtual channel.

[0071] In some implementations, the communication fabric may be configured to maintain an order among certain completion messages and the FwdCnfltO message. The completion messages are the CmpO message and any data message with CmpO appended (Data_*_CmpO). Together, all of these messages are the "ordered completion responses." The conceptual requirement between ordered completion responses and the FwdCnfltO message is that an FwdCnfltO does not "propagate" an ordered completion. If a home agent sends an ordered completion response followed by an FwdCnfltO message, and both messages are intended for the same coherence agent, then the communication fabric delivers the ordered completion response before the FwdCnfltO, among other potential examples.

[0072] It should be understood that while some examples of protocol flow are disclosed here, the examples described are merely intended to provide an intuitive feel for the protocol and do not necessarily cover all possible scenarios and behaviors that the protocol may exhibit.

[0073] A conflict can occur when requests to the same cache line address are made by more than one coherence agent around the same time. As a specific example, a conflict can occur when a snoop for a standard request from one coherence agent arrives at a peer coherence agent with a pending request to the same address. Because any snoop can result in a conflict, a single request can have multiple conflicts. Resolving conflicts can be a coordinated effort between the home agent, the coherence agents, and the communication fabric. However, the primary responsibility lies with the coherence agents, which detect conflicting snoops.

[0074] In one embodiment, home agents, coherence agents, and communication fabric may be configured to assist in the successful resolution of conflicts. For example, home agents may have outstanding snoops for only one request per address at a time, so that for a given address, a home agent may have outstanding snoops for only one request. This may serve to eliminate the possibility of concurrency conditions involving two requests that conflict with each other. This may also ensure that a coherence agent does not see another snoop to the same address after detecting a conflict but not yet resolving it.

[0075] In another example, the coherence agent may allocate a forwarding resource and send a forwarding request to the home agent when processing a snoop with an address that matches an active standard request. A coherence agent with a pending standard request that receives a snoop to the same address may respond with an RspCnflt snoop reply. This reply may be a forwarding request to the home agent. Because the message is a request, the coherence agent may allocate a resource before sending it to handle the reply that the home agent will send. (The coherence protocol allows conflicting snoops to be blocked if, in some cases, the coherence agent runs out of forwarding resources.) The coherence agent may store information about the conflicting snoop for use when processing the forwarding reply.After detecting a conflict and until processing the forwarding response, a coherence agent can be guaranteed not to see another snoop to the same address.

[0076] In some examples, the home agent does not record the snoop response when it receives a forward request. Instead, the home agent can send a forward response to the conflicting coherence agent. A forward request (RspCnflt) in one example looks like a snoop response, but the home agent does not treat it as one. It does not record the message as a snoop response but instead sends a forward response. Specifically, for each forward request (RspCnflt) it receives, a home agent sends a forward response (FwdCnfltO) to the requesting coherence agent.

[0077] The HPI communication fabric orders forwarding responses and ordered completions between the home agent and the addressed coherence agent. The fabric can thus serve to distinguish early conflict from late conflict at the conflicting coherence agent. From a system-level perspective, an early conflict occurs when a snoop encounters a request that the home agent has not yet processed, and a late conflict occurs when a snoop encounters a request that the home agent has already processed. From a home agent perspective, an early conflict is when a snoop encounters a request for the currently active request that the home agent has not yet received or begun processing, and a late conflict is when the snoop encounters a request that it has already processed.In other words, a late conflict occurs for a request to which the home agent has already sent a completion response. Therefore, when a home agent receives a forward request for a late conflict, it has already sent the completion response to the pending request from the conflicting agent. By ordering the forward responses and ordered completion responses from the home agent to the coherence agent, the coherence agent can determine whether the conflict was early or late based on the processing state of its conflicting request.

[0078] When a coherence agent receives a forward response, it uses the state of its conflicting request to determine whether the conflict was early or late and when to process the original snoop. Due to the ordering requirement of the communication fabric, the state of the conflicting request indicates whether the conflict was early or late. If the request state indicates that the completion was received, then it was a late conflict; otherwise, it was an early conflict. Alternatively, if the request state indicates that the request is still awaiting its response(s), then it was an early conflict; otherwise, it was a late conflict.The type of conflict determines when the snoop should be processed: From a coherence agent perspective, an early conflict means that the snoop is for a request processed before the agent's conflicting request, and a late conflict means that the snoop is for a request processed after the agent's conflicting request. In this ordering, for an early conflict, the coherence agent immediately processes the original snoop; and for a late conflict, the coherence agent waits until the conflicting request has received its data (for reads) and its processor has had a chance to execute the completed request before processing the snoop. When the conflicting snoop is processed, the coherence agent generates a snoop response for the home agent for eventual record.

[0079] All conflicts involving writeback requests can be late conflicts. From the coherence agent perspective, a late conflict occurs when the agent's request is processed before the snoop's request. This definition allows all conflicts involving writeback requests to be treated as late conflicts, since the writeback is processed first. Otherwise, data consistency and coherence could be violated if the home agent were to process the request before the writeback commits to memory. Because all conflicts involving writeback operations are considered late conflicts, coherence agents can be configured to block conflicting snoops until a pending writeback request is completed. Furthermore, writeback operations can also block the processing of forwarders.Blocking forwarders with an active writeback can also be implemented as a protocol requirement to support non-cacheable storage, among other examples.

[0080] When a coherence agent receives a request to snoop its cache, it can first check whether the coherence protocol allows it, and then process the snoop and generate a response. One or more state tables can be defined within a set of state tables that defines the protocol specification; one or more state tables can specify when a coherence agent can process a snoop, and whether it snoops the cache or generates a conflict forwarding request instead. In one example, there are two conditions under which a coherence agent processes a snoop. The first condition is that the coherence agent has a REQ request (Rd* or Inv*) to the snoop address and an available forwarding resource. In this case, the coherence agent must generate a forwarding request (RspCnflt).The second condition is when the coherence agent has no REQ, Wb*, or EvctCln request for the snoop address. A state table can define how a coherence agent should process the snoop according to such corresponding conditions. In one example, among other conditions, the coherence agent can block the snoop until either a forwarding resource becomes available (first condition) or the blocking Wb* or EvctCln receives its CmpU response (second condition). Note that NonSnp* requests must not interfere with snoop processing, and a coherence agent cannot consider NonSnp* entries when determining how to process or block a snoop.

[0081] When generating a forward request, a coherence agent can reserve a resource for the forward response. The HPI coherence protocol, in one example, may not require a minimum number of forward response resources (beyond having at least one) and may allow a coherence agent to block snoops if it has no forward response resources available.

[0082] How a coherence agent handles a snoop in its cache can depend on the snoop type and the current cache state. However, for a given snoop type and cache state, there can be many permissible responses. For example, a coherence agent with a fully modified line that receives a non-conflicting SnpMigr (or processes a forward response after an SnpMigr) can do any of the following: demote to S, send an implicit writeback to Home, and send Data_F to the requesting agent; demote to S, send an implicit writeback to Home; demote to I, send Data_M to the requesting agent; demote to I, send an implicit writeback to Home and Data_E to the requesting agent; demote to I, send an implicit writeback to Home; among potentially other examples.

[0083] The HPI coherence protocol allows a coherence agent to cache modified rows with partial masks. However, all rows for M copies may require a full or empty mask. The HPI coherence protocol can, in one example, restrict implicit writebacks of partial rows. A coherence agent that wants to remove a partial M row due to a snoop request (or forward response) can first initiate an explicit writeback and then block (or forward) the snoop until the explicit writeback is complete.

[0084] Storing Information for Forward Responses: The HPI Coherence Protocol, in one embodiment, allows a coherence agent to store forward response information separately from the outgoing request buffer (ORB). Separating the information allows the ORB to release ORB resources and RTIDs when all responses, regardless of entry, involved in a conflict have been collected. State tables can be used to specify what information to store for forward responses and under what conditions.

[0085] Forward responses in the HPI coherence protocol can contain the address, the requesting agent's NID, and the home TID. They do not contain the original snoop type or RTID. A coherence agent can store the forward type and RTID if it wants to use them with the forward response, and it can use the address to match the incoming forward response with the correct forward entry (and generate the home NID). Storing the forward type can be optional. If no type is stored, the coherence agent can treat a forward response as if it had Fwdlnv. Likewise, storing the RTID can be optional and can only occur if the coherence agent is to support cache-to-cache transfers when processing forward responses.

[0086] As noted above, coherence agents can generate a forward request when a snoop request conflicts with a pending request. Forward requests address the home agent that generated the snoop, which can be determined from the address in the snoop request. Thus, the destNID can identify a home agent. The forward request can also include the TID for the home agent's resource allocated to process the original request and the NID of the coherence agent generating the forward request.

[0087] In one embodiment, a coherence agent may block forwarders for writeback requests to maintain data consistency. Coherence agents may also use a writeback request to pass non-cacheable (UC) data before processing a forwarder and may allow the coherence agent to writeback partial cache lines instead of a protocol that supports partial implicit writeback for forwarders. Indeed, in one embodiment, a coherence agent may be allowed to store modified lines with partial masks in its cache (although M copies must include a full or empty mask).

[0088] In one example, early conflicts can be resolved by a forward-response encountering a pending standard request before it has received any response. A corresponding protocol state table, in one example, can specify that a forward-response can be processed as long as the standard request entry is still in the ReqSent state. Late conflicts can be resolved by a forward-response arriving after the pending request has received its completion response. When this occurs, the request is either complete (has already received its data or was an Inv* request) or the entry is in its RcvdCmp state. If the request is still waiting for its data, then the coherence agent must block forwarding until the data has been received (and used).When the conflicting Rd* or Inv* request completes, the forward response can be processed as long as the coherence agent has not initiated an explicit writeback of the cache line. It may be permissible for a coherence agent to initiate an explicit writeback while having a forward response (or snoop request) to the same address, thus allowing partial lines (e.g., snoop requests to partially modified lines) or non-cacheable stores to be properly committed to memory.

[0089] With reference now to Fig.Figure 7 illustrates a first example of an exemplary conflict management scheme. A first cache (or coherency) agent 705 may send a read request for a particular data line to the home agent 710, resulting in a read of memory 715. Shortly after the read request by the cache agent 705, another cache agent 720 issues a request for ownership (RFO) of the same line. However, the home agent 710 sent the Data_S_CmpO to the first cache agent 705 before receiving the RFO from the cache agent 720. The RFO may result in a snoop (SnpFO) being sent to the cache agent 705 (as well as to other cache agents), and the snoop being received by the first cache agent 705 before receiving the completed Data_S_CmpO.After receiving the snoop SnpO, cache agent 705 may determine a potential conflict involving the memory line requested in its original read request and may notify home agent 710 of the conflict by responding to the SnpO with a forward-response-conflict message (RspCnflt). Home agent 710 may respond to the forward-response RspCnflt by sending a forward-response (FwdCnfltO). Cache agent 705 may then receive the shared-data-completed Data_S_CmpO and transition from an I state to the S state. The forward response FwdCnfltO may then be received by the cache agent 705, and the cache agent 705 may determine how to respond to the forward response message FwdClfltO based on the snoop SnpFO that triggered the sending of the forward response RspCnflt.In this example, the cache agent 705 may consult a protocol state table to determine, for example, a response to the forward response message FwdClfltO. In the specific example of . Fig. 7, the cache agent 705 may transition to an F state and send the S copy of the data it received from the home agent 710 in the Data_S_CmpO message to the second cache agent 720 in a Data_F message. The first cache agent 705 may also send a response message RspFwdS to the home agent 710, notifying the home agent 710 that the first cache agent has shared its copy of the data with the second cache agent.

[0090] In another illustrative example, shown in the simplified flow chart of Fig.8, the first cache agent 705 may send a request for ownership (RFO) of a particular memory line to the home agent 710. Shortly thereafter, a second cache agent may send an RdInvOwn message to the home agent 710 as a request for the same memory line in an M state. In conjunction with the RFO message from the first cache agent 705, the home agent 710 may send a snoop (SnpFO) to the second cache agent 720, which the second cache agent 720 may identify as a potential conflict involving the memory line subject to both the RFO and the RdlnvOwn request. Accordingly, the second cache agent 720 may send a forward request RspCnflt to the home agent 720. The home agent 720 responds to the forward request of the second cache agent 720 with a forward response.The second cache agent 720 determines a response to the forward response based on information contained in the original snoop SnpFO. In this example, the second cache agent 720 responds with a snoop response Rspl, indicating that the second cache agent 720 is in an I state. The home agent 710 receives the snoop response Rspl and determines that it is appropriate to send the data complete exclusive (Data_E_CmpO) to the first cache agent 705, causing the first cache agent to transition to an E state. Once the complete has been sent, the home agent 710 may then begin responding to the second cache agent's RdInvOwn request with a snoop request SnplnvO from the first cache agent 705. The first cache agent 705 may determine that the snoop results in a request by the second cache agent 720 to obtain an exclusive M-state copy of the line.Therefore, the first cache agent 705 transitions to the M state to send its copy of the line as an M-state copy (with a Data_M message) to the second cache agent 720. In addition, the first cache agent 705 also sends a response message RspFwdl to indicate that the copy of the line has been sent to the second cache agent 720 and that the first cache agent has transitioned to an I state (has relinquished ownership of the copy to the second cache agent 720).

[0091] With reference now to the example of Fig.9, another simplified flow diagram is shown. In this example, a cache agent 720 attempts to request exclusive ownership of a non-cacheable (UC) line without receiving any data (e.g., through an InvltoE message). A first cache agent 705 sends a contention message (RdInv) for the cache line in the E state. The HPI coherence protocol may specify that if the requested line was previously cached in the M state, the line is written to memory before delivering E data in response to the first cache agent's 705 Rdlnv. The home agent 710 may send a complete (CmpO) to the InvltoE request and send a snoop (SnpInv) to the cache agent 720 based on the Rdlnv request.If the cache agent 720 receives the snoop before completion, the cache agent 720 may determine that the snoop belongs to the same cache line as its exclusive ownership request and may indicate a conflict through a forward request RspCnflt. As in previous examples, the home agent 710 may be configured to respond to the forward request with a forward response (FwdCnfltO). Multiple permissible responses may be allowed with respect to the forward response. For example, the cache agent 720 may initiate an explicit writeback (e.g., WbMtol) and block (or forward) the snoop until the explicit writeback (e.g., CmpU) is complete, as shown in the example of FIG. Fig. 9 is completed. The cache agent may then complete the snoop response (Rspl). The home agent 710 may then, among other examples, process the Rdlnv request from the first cache agent 705 and return a Completed Data_E_CmpO.

[0092] In examples such as the example of Fig.9, where a cache agent receives a snoop when the agent has a pending read or invalidate request to the same address and it has a partially modified line (often referred to as a "buried M"), the HPI coherence protocol, in one implementation, allows the agent to either 1) perform an explicit writeback (partial) of the line while blocking the snoop or 2) send a forward request (RspCnflt) to the home agent. If (1) is selected, the agent processes the snoop after receiving the writeback completion. If (2) is selected, it is possible for the agent to receive the forward response (FwdCnfltO) while its pending read or invalidate request is still awaiting responses and the agent still has a partially modified line.If this is the case, the protocol allows the agent to block the forwarding while performing an explicit (partial) writeback of the row. During the writeback, the protocol guarantees that the agent will not receive any responses for the pending read or invalidate request. The mechanism described above (allowing coherence agents to issue explicit writebacks and blocking snoops and forwardings even if the agent has a pending read or invalidate request) is also used to ensure that partial or UC writes are posted to memory before the writer acquires global observability.

[0093] Coherence agents use a two-step process for partial / UC writes. First, they check if they have ownership of the cache line and, if not, issue an ownership (invalidation) request in the log. Second, they execute the write. In the first step, if they have executed an ownership request, it is possible that the request conflicts with other agents' requests for the line, meaning the agent could receive a snoop while the ownership request is pending. For each coherence protocol request, the agent issues a forward request for the conflicting snoop. While waiting for the forward response, the agent can receive the completion of the ownership request, which grants the agent ownership of the line and allows the agent to initiate the writeback for the partial / UC write.While this is occurring, the agent may receive the forward response, which it is also obligated to process. The coherence agent may not combine the two activities. Instead, the coherence agent must process the partial / UC write data separately from processing the forward and perform the writeback first. For example, among other examples and features, a cache agent may use a writeback request to pass UC data before processing the forward and writeback of partial cache lines.

[0094] In one embodiment, the HPI coherence protocol may support a read invalidate (Rdlnv) request that accepts exclusive state data. Semantics of non-cacheable (UC) reads include flushing modified data into memory. However, some architectures allow forwarding M data to invalidate reads, which would have forced the requesting agent to evict the row if it received M data. The Rdlnv simplifies the flow and does not allow E data to be forwarded. For example, as shown in the example of Fig. 10, the directory state of a home agent 710 may indicate that no agent (e.g., 705, 710) has a copy of the row. In such cases, the home agent 710 may send the data and completion response(s) immediately. The HPI allows the same if the effective directory state indicates that no peer can have a copy of the row.

[0095] As shown in the example of Fig. 10, in some implementations, an agent may respond to a snoop with an RsplWb message indicating that the caching agent (e.g., 705) is in (or has transitioned to) an I state while requesting a memory write. An RsplWb may set the effective directory state to Invalid, allowing a home agent 710 to send a response without snooping all peers. In the example of Fig.10, a second cache agent 720 sends an Rdlnv request while the home agent directory is in an Unknown state. In response, the home agent 710 initially snoops only the first cache agent 705. In this example, the cache agent 705 has a modified copy of the line and responds with an implicit writeback (e.g., RsplWb). When Home receives the RsplWb message, it can determine that no other agent could have had a copy of the line and can further identify that the cache agent 705 invalidated its cache with the RsplWb. In response, the home agent 710 can set the directory state to Invalid. Since the directory state is invalid, the home agent 710 waits until the write to memory 715 is complete and then sends the data and completion response(s) (e.g., Data_E_CmpO) and releases the resource it allocated for the request from the cache agent 720.In this example, the home agent can skip snooping other cache agents in the system. Indeed, in such examples, a home agent (e.g., 710) can send data and a completion response before receiving all snoop responses (e.g., due to the identification of an M copy at agent 705), as illustrated in the example of . Fig. 11 (with cache agent 1105).

[0096] In the examples of Fig. 10 and Fig.11, when the second cache agent 720 receives the Data_E_CmpO response from the home agent 710, the second cache agent 720 may load the data into its cache, set its cache state to E, and release the resource RTID it allocated for the request. After releasing the RTID, the cache agent 720 may reuse it for a new request. Meanwhile, the home agent 710 may wait for snoop responses for snoops on the request that originally used the RTID. Snoop messages may include the request's RTID and the requesting agent's NID. Therefore, since the cache agent 720 may reuse the RTID for a new request to the same or a different home agent, and that home agent may generate snoops for the new request while snoops for the original request are pending, it is possible that the same "unique" transaction ID exists in snoops to the same coherence agents.From a coherence perspective, this duplication of the transaction ID (TID) may still be permissible, since snoops will only find I-states for the original request.

[0097] A home agent can generate a snoop if the request's tracker state is Waiting, Busy, or DataXfrd, which means that the home agent has not yet sent a data response or a snoop response indicating that a peer forwarded the data to the requesting agent. A home agent can also check the request's snoop field to ensure it has not yet sent a snoop to a peer. When a home agent sends a snoop, it can add the peer (or all fanout peers) to the snoop (to prevent it from sending a second snoop) and track pending snoop responses.

[0098] As noted above, some implementations of the HPI may support fanout snoops. Additionally, in some examples, the HPI may support an explicit fanout snoop operation, SnpF, for fanout snoops generated by the routing layer. An HPI home agent (e.g., 710) may use SnpF to generate a single fanout snoop request (e.g., with a single command and message), and in response, the routing layer may generate snoops to all peer agents in the corresponding fanout cone based on the SnpF request. The home agent may accordingly expect snoop responses from each of the agent sections. While other snoop messages may include a destination node ID, fanout snoops may omit a destination NID because the routing layer is responsible for generating the appropriate snoop messages to all peers in the fanout region.

[0099] Because the routing layer is immediately below the protocol layer, in some implementations, the communication fabric requirements are located at the routing layer. In one embodiment, the HPI coherence protocol may have a communication fabric requirement specific to the routing layer. For example, the coherence protocol may depend on the routing layer to convert a fanout snoop (SnpF* opcodes - Snoop (SNP) channel messages) into the appropriate snoops for all peers of the request in the fanout set of cache agents. The fanout set is a routing layer configuration parameter shared by the protocol layer, or a home agent configuration parameter.

[0100] In some implementations, a home agent can send a fanout snoop for an active standard request. The HPI routing layer can convert the home agent's fanout snoop request into normal snoops to each of the peers in the fanout cone defined by the routing layer. The HPI coherence protocol home agent is made aware of which coherence agents are covered by the routing layer fanout via a HAFanoutAgent configuration parameter, which identifies the corresponding cache agents included in the fanout cone by address. The routing layer can receive the fanout snoop (SnpF) and convert it into snoops of each cache agent included in the fanout cone (excluding the requesting agent). In one implementation, the routing layer may, among other examples, convert the fanout snoop into corresponding non-fanout snoops (with appropriate non-fanout opcodes, such as those in Table 3).

[0101] Similar to normal snoops, a home agent may be restricted to sending only one fanout snoop before sending a completion response to a coherence protocol request by a cache agent. Further, additional conditions may be imposed on fanout snoops. For example, a home agent may send a fanout snoop if it has not individually snooped any of the peers in the fanout cone. In other words, in some implementations, among other examples, a home agent may not initiate a fanout snoop if the fanout cone is empty, or if the requesting cache agent is the only agent in the fanout cone.

[0102] In one embodiment, the HPI may support explicit writeback with cache push hint (WbPushMtol). In some examples, modified data may generally be transferred either by explicitly writing the data back to memory or by transferring the modified data in response to a snoop request. Transferring modified data in conjunction with a snoop response may be considered a "pull" transfer. In some implementations, a "push" mechanism may also be supported, where a caching agent with the modified data sends the modified data directly to another caching agent for storage in the target agent's cache (along with the modified cache state).

[0103] In one embodiment, a cache agent may write modified data back to the home agent with an indication that it can push the modified data to a "local" cache and store the data in the M state in the local cache without writing the data to memory. In one implementation, a home agent 710 may receive a WbPushMtol message from a cache agent 705 and identify the indication that another cache agent (e.g., 720) is likely to use or desire ownership of a particular line in the near future, as shown in the example of Fig.12. The home agent 710 may process the WbPushMtol message and effectively accept the hint and push the written-back data to the other cache agent 720 without writing the data to memory 715, thereby causing the other cache agent 720 to transition to an M state. In some implementations, the home agent 710 may alternatively process the WbPushMtol message and choose to write the data back to memory as with a WbMtol request (as illustrated in Fig. 13) and do not push the written-back data directly to the other cache agent 720.

[0104] In an example implementation, a home agent (e.g., 710) may process a WbPushMtol message by checking that the tracker state is WbBuffered, which may indicate that the home agent has not yet processed the data. In some cases, pushing the data may be made dependent on the home agent by determining that the home agent is not already processing a standard request to the same address. In some implementations, pushing may be further made dependent on the home agent by determining that the addressed cache agent (e.g., 720 in the example of Fig.12) is "local." If the addressed cache agent is not covered by the home agent directory, the home agent may transfer the data to the target cache agent's cache and update the directory to invalid. If the addressed cache agent is covered by the directory, data transfer to the cache agent's cache may only be allowed if the addressed cache agent does not have an active InvXtol, and after the transfer, the home agent may update the directory to exclusive with the target cache agent as the owner. Other conditions may exist, among other potential examples for a home agent (e.g.,in a corresponding protocol state table) to determine whether the WbPushMtol message hint should be accepted and data pushed to an addressed cache agent, or whether instead the WbPushMtol message should be processed as a WbMtol request by first writing the data to memory.

[0105] In some implementations, the HPI may support an InvltoM message to preallocate a directory cache of a home agent, such as an I / O directory cache (IODC). An InvItoM may request exclusive ownership of a cache line without receiving any data, while indicating the intent to perform a writeback soon afterward. A required cache state may be an M state and an E state, or both. A home agent may process an InvltoM message to preallocate a resource for the writeback, indicated by the InvltoM message (which includes the InvltoM opcode).

[0106] In some implementations, an instruction code may be provided by the HPI coherence protocol to trigger a memory flush of a memory controller with which one or more home agents interact. For example, an instruction code, WbFlush, may be defined for a permanent memory flush. As shown in the example of Fig.14, a host (e.g., 1405) may send a WbFlush message directed to a specific memory controller 1410. In some cases, the WbFlush may indicate a specific address, and the WbFlush command may be sent to the specific memory controller addressed by the address. In another example, a WbFlush message may be transmitted to multiple memory controllers. In one example, the t may be sent as a result of a permanent commit in a CPU. Each corresponding memory controller (e.g., 1410) receiving a WbFlush command may process the message to flush all pending writes at the memory controller to a permanent storage device (or location) managed by the memory controller. The purpose of the command may be to commit all previous writes to permanent storage.For example, a WbFlush command can be triggered in conjunction with a power-failure management controller or process to ensure that pending writes to non-volatile memory are flushed and preserved in the event of a system power failure. Further, as shown in the example of . Fig. 14 After flushing (or initiating the flushing) of all pending memory writes (e.g., 1415), memory controller 1410 should respond to the requesting host (or agent) (e.g., 1405) with a completion indicating the flush. The completion should not be sent to the host until the memory controller has ensured that the data makes it to persistent storage. The WbFlush message or corresponding completion, among other uses and examples, can serve as a checkpoint for other processes and controllers that depend on it or direct the flushing of pending memory writes.

[0107] Some traditional architectures may require that Data_M and corresponding Completed be sent separately. The HPI can be extended to allow coherence agents to support accepting a combined Data_M_CmpO. Further, home agents can be configured to generate a combined Data_M_CmpO message by buffering implicit writeback data. Indeed, in some implementations, an agent may be equipped with logic that combines cache and home agent behavior, so that when the agent receives a request and finds M data in its cache, it can directly generate the Data_M_CmpO. In such cases, among other examples, the Data_M_CmpO response can be generated without generating an RsplWb or buffering writeback data.

[0108] In another example, as shown in the example protocol state table 1500 shown in Fig.15, a state machine (embodied by a machine-readable state table (e.g., 1500)) may define a plurality of potential response messages that a home agent may send when the tracker entry of the standard request is identified as a Busy or WbBuffered state. As shown in table 1500, in one example, a home agent may not be allowed to send a CmpO completion message to a Read Rd* request from any state, effectively meaning that a home agent must send a data response before or with a completion response. In cases where a Data_X response can be sent in the home agent response message, the home agent may combine the data response with a completion and send it instead.

[0109] The state of the data response can be fixed for invalid requests and RdCur. For RdMigr and RdData, non-shared directory state can allow E data to be sent. For RdMigr, RdData, and RdCode, shared directory state can involve checking whether all peers that might have F state have been snooped. If so, then the data can be sent with the F state; otherwise, the data can be sent in the S state in the case of an unsnooped peer having an F copy, among other potential examples. Furthermore, in some implementations, a home agent can only send a Data_M or Data_M_CmpO response if it has buffered the data from an RsplWb snoop response. When a home agent buffers RsplWb data, it can store the data in the tracker entry and change the entry's state to WbBuffered.Note that if a home agent buffers the RsplWb data instead of writing it to memory, it sends a Data_M or Data_M_CmpO response in this example.

[0110] In one embodiment, as noted above, the HPI coherence protocol may support an F-state, allowing a cache agent to hold the F-state while forwarding shared data. In some systems or cases, the F- (forward) cache state may itself be forwardable. If a cache holds a line in the F-state and receives a snoop that allows shared data to be transferred, the cache may forward the data. If it does, it may send the F-state with the data and transition its cache state to S (or I).

[0111] Under certain circumstances, it is desirable for the cache to instead hold the F state while forwarding data, and then send the S state with the forwarded data.

[0112] In one example, the ability of a cache agent to retain or pass F-state during a shared transfer may be controllable. In one example, a per-coherence agent configuration parameter may indicate whether a coherence agent is transmitting or persisting to F-state. Regardless of the parameter setting, the coherence agent may use the same snoop response (e.g., RspFwdS). In the additional case of an agent that has the row in E-state when the snoop arrives, the cache agent may transition its cache state to F when forwarding the S-data and send the RspFwdS response (if the parameter is set to retain F-state).In the additional case of an agent that has the row in the M (Full) state when the snoop arrives, the cache agent may downgrade its cache state to F upon forwarding the S data, write the data back to memory, and send the RspFwdSWb response (if the parameter is set to hold the F state). Further, an F-state coherence agent that receives a "sharing" snoop or forwards after such a snoop may hold the F state while sending the S state to the requesting agent. In other cases, the configuration parameter may be toggled to allow the F state to be transferred in a shared data transfer and transition to an S (or I) state, among other examples. Indeed, as shown in the example state table 1600 of FIG. Fig.16 A cache agent in the F state may respond in a variety of ways including SnpMigr / FwdMigr, F, F, RspFwdS, Data_S, among other examples.

[0113] As stated above, in some implementations, state transitions of a cache line and agents may be managed using a state machine. In one implementation, the state machine may be further embodied by a set or library of state tables defined to represent all the various combinations of instructions, attributes, preceding states, and other conditions that may affect how state transitions should occur, as well as the types of messages, data operations, masks, and so on that may be associated with the state transition (as illustrated in the specific examples of the Fig. 15 and Fig.16). Each state table can correspond to a specific action or category of actions or states. The set of tables can include multiple tables, each corresponding to a specific action or subaction. The set of tables can embody a formal specification of a protocol, such as the Coherence Protocol or another protocol (at any of the stack layers) of the HPI.

[0114] State tables can be human-readable files, such as table structures, that can be readily interpreted and modified and developed by a human user interacting with the state table structure using an endpoint computing device. Other users can use the state table to readily interpret state transitions within the Coherence Protocol (or any other HPI protocol). Furthermore, state tables can be machine-readable and processable structures that can be read and interpreted by a computer to determine how to transition states according to a particular protocol specification.

[0115] Fig.Figure 17 illustrates a simplified representation of a generalized state table for an action "Action A." A protocol state table 1700, in one example, may include columns (e.g., 1705) pertaining to current states (or the states to transition from) and other columns (e.g., 1710) pertaining to next states (or the states to transition to). Columns in the current state columns may correspond to various properties of the state, such as commands received in a reply message, snoop message, or other message, a cache line state, an outbound fetch buffer (ORB) condition, credits or resources to be applied / reserved, whether the cache line is partially modified, a forwarding condition, and so on. Each row in the table 1700 may correspond to a recognized set of conditions for a cache line in a particular state.Further, the cells in the row within the next state columns (e.g., 1710) may indicate the next state and the conditions of the next state to be entered based on the conditions of the current state specified in the row cells in the current state columns (e.g., 1705). The next state columns (e.g., 1710) may correspond to conditions in the next state, such as the messages to be sent (e.g., to a corresponding home node (HNID), requesting node (RNID), peer node, etc.), the next cache line state, forwarding state, and so on.

[0116] In one embodiment, protocol state tables may use row spanning to indicate that multiple behaviors or states (rows) are equally valid for a given set of current state conditions. In the example of Fig.17 For example, if the command is Cmd1, a first condition is false, the cache line is in a second state, and a second condition is also false (as indicated by rows 1715), multiple potential next-state conditions are possible and may be equally permissible, each indicated by a corresponding row. In other words, any such equally permissible transitions may be triggered based on the corresponding current-state conditions. In some implementations, among other example implementations, additional agent logic may select which of the multiple next states to select.In one illustrative example, a current state section of a state table corresponding to Home-Agent-Send-Request-Response may include multiple conditions (or input and state guards), including all valid behaviors for a coherence agent to execute when the agent holds a full M line in its cache and is processing an SnpMigr to the same cache line. The table rows may correspond, among other examples, to five different and equally valid next-state behaviors that the coherence agent can take in response to the current state conditions.

[0117] In other systems, a bias bit may be included in protocol state tables where multiple potential next states or conditions for a particular current state are possible. In QPI, for example, a "bias" bit is included in tables as a mechanism to select among behaviors. Such bits may be used primarily during the validation of a protocol's state machine, but such bits introduce additional complexity and, in some cases, confusion for those unfamiliar with the bias bit's utility. In some respects, a bias bit may be nothing more than an artifact of a validation exercise. In one HPI example, protocol tables that use rows that potentially span multiple rows can exclude bits and other features. In such cases, HPI protocol tables can emphasize explicit nondeterminism.

[0118] With reference now to the example of Fig. 18, in one embodiment, log tables may be nested by having one table refer to another subtable in the “next state” columns, and the nested table may have additional or finer-grained safeguards to specify which rows (behaviors) are allowed. As shown in Fig.18, an exemplary protocol state table 1700 may include an embedded reference 1805 to another table 1800 included in the set of tables embodying a protocol specification, such as a state table corresponding to a sub-action related to the action or behavior included in the next state provided for particular rows of the table 1700. Multiple tables (e.g., 1700, 1810) may reference a nested table (e.g., 1800). As an example, an agent processing incoming protocol responses may follow an action table (e.g., 1700, 1810) and a sub-action table 1800. Here, the action table 1700 may include a next state with a sub-action table nested under one or more other protocol tables.This type of nesting can be applied beyond coherence protocol and protocol layer state tables, but can also be applied to any known or future protocol response / state tables.

[0119] In one example, an agent may utilize history tables (or other processable structure constructed from history tables) and may identify a particular state table corresponding to a particular action or event. Further, the agent may identify the row that applies to the cache line being handled or addressed by the agent and may identify from the table the next state information for the cache line. This determination may include identifying a reference to a nested table of a sub-action. Accordingly, the agent may determine the appropriate structure of the associated nested table and further reference the nested table to determine the state transition.

[0120] In a specific example, a common set of protocol tables can be defined and represent all possible, defined state transitions in a protocol. Further, each table can specify a set of transitions that cover a set of related behaviors within the protocol (e.g., one table covers all behaviors involved in snooping and updating cache state, one covers all behaviors that generate new requests, etc.). When an agent needs to execute a behavior, process an event, or check whether another action should be taken, the agent can identify the specific state table that covers the specific behavior within the set of state tables.The agent can then determine the current state of the system and reference the selected state table to identify the row or group of rows that match the current state, if any. If no rows match, the agent may, in some cases, refrain from taking any action for the given current state and wait for another event / behavior to change the state before attempting again. Further, in some cases, as introduced above, if more than one row matches the identified system state, the agent may choose any one of them to execute, since all can be considered equally valid.Furthermore, in the case of nesting, the agent can access the nested table and use the identified current state of the system to search for allowed rows in the nested table if a row references a nested table.

[0121] In some examples, after traversing any primary and nested tables to determine a response to a particular identified system (or protocol) state, the agent may initiate the appropriate actions to be performed and update the state of the system according to the "next states" provided in the corresponding state tables.

[0122] In some cases, it may be possible for more than one state table to relate to or cover a set of behaviors. For example, as an illustrative example, two tables may be provided to handle snoops: the first in case there was a conflicting active request, the second in case there was none. Accordingly, in some implementations, an agent may query multiple tables to determine which table includes rows relevant to the particular conditions and states identified by the agent. Further, in some cases, an agent may handle two unrelated or distinct events occurring simultaneously, such as when a home agent receives a snoop response and a new request at the same time.In cases where multiple events are processes, an agent can simultaneously identify and use multiple corresponding tables to determine how to process the events.

[0123] With reference now to the Fig. 19 and Fig. 20 are simplified block diagrams 1900, 2000 of examples of a test or validation environment for use in validating at least a portion of a protocol. In the example of Fig.19, for example, a test engine 1900 is provided that is adapted to validate a state machine of a protocol. In one example, the test engine 1900 may, for example, include or be based on principles of a Murphi tool or other enumerative (explicit state) model checker, among other examples. For example, other specification languages ​​may be used instead of the described Murphi examples, including, as another example, TLA+ or another suitable language or format. In traditional systems, state model checkers have been designed by human developers attempting to translate state machines (e.g., from accompanying state tables, etc.) into a set of requirements that are then used to generate a checker capable of verifying the state machine.Not only is this typically a labor- and resource-intensive process, but it also introduces human error as the states and state transitions of a state table are transcribed and interpreted by human users.

[0124] In one implementation, a test engine 1900 may use a set of state tables (e.g., 1905) to automatically generate resources from the set of state tables to model agent behavior in a test environment. In the example of Fig.19, a test engine 1900 may, for example, use the state tables 1905 as a functionality engine to model a cache agent or other agent (e.g., 1910) that may be used to validate various state transitions by simulating requests and responses (e.g., 1912) with other real or simulated agents, including a home agent 1915. Similarly, as shown in the example of Fig.20 The test engine 1900 may use the state tables 1905 to simulate requests and responses (e.g., 1918) from a home agent (e.g., 1920) and connect them to other real or simulated agents (e.g., 1925) to further validate and enumerate states of the protocol. As an example, the test engine 1900 may model an agent and receive real or modeled protocol messages, such as HPI coherence protocol messages, and reference state tables 1905 (or another processable structure generated from the state tables 1905) to automatically generate an appropriate response, perform appropriate state transitions, etc., based on the state tables 1905.

[0125] In a specific implementation, a test engine or other software- or hardware-based utility can be used to use state tables (e.g., 1905) and generate code to drive and respond to designs that employ a specific protocol, such as an HPI coherence protocol. In this particular example, state tables can be used as an input to the test engine by converting tables or enclosed pseudocode, along with Murphi mappings for table values ​​and pseudocode elements, into an appropriate Murphi rule and procedure format. The test engine can be used to further generate Murphi code for type definitions and supporting functionality. Murphi rule, procedure, type, and support code can be used to generate a Murphi model.The Murphi model, for example, can be translated using a converter for a C++ or other class definition. Indeed, any suitable programming language can be used. Further subclasses of the model class can be generated, and these modules can be used to behave like a simulated or testbench version of an agent deployed and aligned with the protocol specification embodied in the state tables. Further, an internal API can be generated or otherwise provided that is aligned with message generation and message reception as defined in the protocol state tables. For example, a message generation API can be bound to link packet types, and message reception can be unified under a single interface point.In this example, an entire formal protocol specification can be converted into a C++ class (or another object-oriented programming language). Inheritance can be used to interrupt generated messages, and instances of the inheriting class can be created as functional testbench agents. In general, formal specification tables can be used as a functionality engine for a validation or testbench tool, rather than developers separately creating their own tool based on their interpretation of the specification.

[0126] The HPI may be incorporated into any variety of computing devices and systems, including, among other examples, mainframes, server systems, personal computers, portable computers (such as tablets, smartphones, personal digital systems, etc.), smart devices, gaming or entertainment consoles, and set-top boxes. With reference to Fig.For example, Figure 21 illustrates one embodiment of a block diagram for a computer system including a multi-core processor. Processor 2100 includes any processor or processing device, such as a microprocessor, an integrated circuit processor, a digital signal processor (DSP), a network processor, a handheld processor, an application processor, a coprocessor, a system on a chip (SOC), or other device, to execute code. Processor 2100, in one embodiment, includes at least two cores—cores 2101 and 2102—which may include asymmetric cores or symmetric cores (the illustrated embodiment). However, processor 2100 may include any number of processing elements, which may be symmetric or asymmetric.

[0127] In one embodiment, a processing element refers to hardware or logic to support a software thread. Examples of hardware processing elements include: a thread unit, a thread slot, a thread, a process unit, a context, a context unit, a logical processor, a hardware thread, a core, and / or any other element that can hold state for a processor, such as execution state or architectural state. In other words, a processing element, in one embodiment, refers to any hardware that can be independently connected to code, such as a software thread, operating system, application, or other code. A physical processor (or processor socket) typically refers to an integrated circuit that potentially includes any number of other processing elements, such as cores or hardware threads.

[0128] A core often refers to logic on an integrated circuit capable of maintaining an independent architectural state, where each independently maintained architectural state is associated with at least some dedicated execution resources. In contrast to cores, a hardware thread typically refers to any logic residing on an integrated circuit capable of maintaining an independent architectural state, where the independently maintained architectural states share access to execution resources. It can be seen that the line between the nomenclature of a hardware thread and core overlaps when certain resources are shared and others are dedicated to an architectural state.However, a core and a hardware thread are often viewed by an operating system as individual logical processors, and the operating system can schedule operations on each logical processor individually.

[0129] The physical processor 2100 closes as in Fig.Figure 21 illustrates two cores, core 2101 and 2102. Here, cores 2101 and 2102 are considered symmetric cores, that is, cores with the same configurations, functional units, and / or logic. In another embodiment, core 2101 includes an out-of-order processor core, while core 2102 includes an in-order processor core. Cores 2101 and 2102 may be individually selected from any type of core, such as a native core, a software-managed core, a core adapted to execute a native instruction set architecture (ISA), a core adapted to execute a translated instruction set architecture (ISA), a co-designed core, or other known core. In a heterogeneous core environment (i.e., asymmetric cores), a form of translation, such as binary translation, can be used to schedule or execute code on one or both cores.To further the discussion, the functional units illustrated in core 2101 are described in further detail below, since the units in core 2102 operate in a similar manner in the illustrated embodiment.

[0130] As illustrated, core 2101 includes two hardware threads 2101a and 2101b, which may also be referred to as hardware thread slots 2101a and 2101b. Therefore, in one embodiment, software entities, such as an operating system, potentially view processor 2100 as four separate processors, i.e., four logical processors or processing elements capable of executing four software threads simultaneously. As noted above, a first thread is connected to architectural state registers 2101a, a second thread is connected to architectural state registers 2101b, a third thread may be connected to architectural state registers 2102a, and a fourth thread may be connected to architectural state registers 2102b. Here, each of the architectural state registers (2101a, 2101b, 2102a and 2102b) may be referred to as processing elements, thread slots or thread units as described above.As illustrated, the architectural state registers 2101a are repeated in the architectural state registers 2101b so that individual architectural states / contexts can be stored for the logical processor 2101a and the logical processor 2101b. In core 2101, other smaller resources, such as instruction pointers and rename logic in the allocator and rename block 2130, may also be repeated for the threads 2101a and 2101b. Some resources, such as reorder buffers in the reorder / restore unit 2135, ILTB 2120, load / store buffers, and queues, may be shared through partitioning. Other resources, such as B. Internal general purpose registers, page table base registers, subordinate data cache and data TLB 2151, execution unit(s) 2140 and portions of out-of-order unit 2135 are potentially fully shared.

[0131] The processor 2100 often includes additional resources that may be fully shared, shared through partitioning, or dedicated to / by processing elements. Fig.21 illustrates one embodiment of a purely exemplary processor with illustrative logical units / resources of a processor. It should be noted that a processor may include or omit any of these functional units, as well as include any other known functional units, logic, or firmware not shown. As illustrated, core 2101 includes a simplified, representative out-of-order (OOO) processor core. However, in various embodiments, an in-order processor may be used. The OOO core includes a branch target buffer 2120 to predict branches to be executed / taken and an instruction translation buffer (I-TLB) 2120 to store address translation entries for instructions.

[0132] Core 2101 further includes decode module 2125, which is coupled to fetch unit 2120 to decode fetched elements. The fetch logic, in one embodiment, includes individual sequencers connected to thread slots 2101a, 2101b, respectively. Typically, core 2101 is connected to a first ISA that specifies / defines instructions executable at processor 2100. Often, machine instructions that are part of the first ISA include a portion of the instruction (referred to as an instruction code) that references / specifies an instruction or operation to be performed. Decode logic 2125 includes circuitry that recognizes these instructions from their instruction codes and passes the decoded instructions into the pipeline for processing, as defined by the first ISA.For example, as described in more detail below, in one embodiment, decoders 2125 may include logic designed or adapted to recognize special instructions, such as a transaction instruction. As a result of the recognition by decoder 2125, architecture or core 2101 takes special, predefined actions to perform tasks associated with the corresponding instruction. Importantly, some of the tasks, blocks, operations, and methods described herein may be executed in response to single or multiple instructions, some of which may be new or old instructions. Note that in one embodiment, decoders 2126 recognize the same ISA (or a subset thereof). Alternatively, in a heterogeneous core environment, decoders 2126 recognize a second ISA (either a subset of the first ISA or a different ISA).

[0133] In one example, the allocator and renamer block 2130 includes an allocator to reserve resources such as register files and to store instruction processing results. However, threads 2101a and 2101b are potentially capable of out-of-order execution, with the allocator and renamer block 2130 also reserving other resources, such as reorder buffers, to track instruction results. Unit 2130 may also include a register renamer to rename program / instruction reference registers to other registers within processor 2100. The reorder / reorder unit 2135 includes components, such as the aforementioned reorder buffers, load buffers, and store buffers, to support out-of-order execution and later in-order reordering of out-of-order instructions.

[0134] The scheduler and execution unit(s) block 2140, in one embodiment, includes a scheduler unit for scheduling instructions / operations at execution units. For example, a floating-point instruction is scheduled at a port of an execution unit that has an available floating-point execution unit. Register files associated with the execution units are also included to store processing results of information instructions. Example execution units include a floating-point execution unit, an integer execution unit, a jump execution unit, a load execution unit, a store execution unit, and other known execution units.

[0135] The subordinate data cache and data translation buffer (D-TLB) 2150 are coupled to the execution unit(s) 2140. The data cache is intended to store recently used / operated elements, such as data operands, that are potentially held in memory coherence states. The D-TLB stores recent virtual / linear to physical address translations. As a specific example, a processor may include a page table structure to partition physical memory into a plurality of virtual pages.

[0136] Here, cores 2101 and 2102 share access to higher-level or more distant cache, such as a second-level cache connected to on-chip interface 2110. Note that higher-level or more distant refers to cache levels that increase or move further away from the execution unit(s). In one embodiment, the higher-level cache is a last-level data cache—the last cache in the memory hierarchy on processor 2100—such as a second-level or third-level data cache. However, higher-level cache is not limited in this regard, as it may be connected to or include an instruction cache. A trace cache—one type of instruction cache—may instead be coupled behind decoder 2125 to store recently decoded traces. Here, an instruction potentially refers to a macroinstruction (i.e.,, a non-privileged instruction recognized by the decoders) that can decode into a number of microinstructions (micro-operations).

[0137] In the illustrated configuration, processor 2100 also includes the on-chip interface module 2110. Historically, a memory controller, described in more detail below, was included in a computer system that is external to processor 2100. In this scenario, on-chip interface 2110 communicates with devices external to processor 2100, such as system memory 2175, a chipset (which often includes a memory controller hub to connect to memory 2175 and an I / O controller hub to connect peripherals), a memory controller hub, a northbridge, or another integrated circuit. And in this scenario, bus 2105 can include any known interconnection structure, such as a multi-drop bus, a point-to-point interconnection structure, a serial interconnection structure, a parallel bus, a coherent (e.g.,cache-coherent) bus, a layered protocol architecture, a differential bus and a GTL bus.

[0138] Memory 2175 may be dedicated to processor 2100 or shared with other devices in a system. Common examples of memory 2175 types include DRAM, DRAM, non-volatile memory (NV memory), and other known memory devices. Note that device 2180 may include a graphics accelerator, processor or card coupled to a memory controller hub, data storage coupled to an I / O controller hub, a wireless transceiver, a flash memory device, an audio controller, a network controller, or other known device.

[0139] While more logic and components can be integrated on a single die such as a SOC, each of these components can be included with processor 2100. For example, in one embodiment, a memory controller hub is located on the same package and / or die with processor 2100. Here, a portion of core (an on-core portion) 2110 includes one or more controllers to connect to other devices such as memory 2175 or a graphics assembly 2180. The configuration that includes an interconnect and controllers to connect to such devices is often referred to as an on-core (or non-core) configuration. As one example, on-chip interface 2110 includes a ring interconnect for on-chip communication and a high-speed serial point-to-point link 2105 for off-chip communication.However, in the SOC environment, even more devices such as the network interface, coprocessors, memory 2175, graphics processor 2180 and any other known computing devices / interface can be integrated on a single die or integrated circuit to provide a small form factor with high functionality and low power consumption.

[0140] In one embodiment, processor 2100 is capable of executing compiler, optimization, and / or translator code 2177 to compile, translate, and / or optimize application code 2176 to support or interface with the apparatus and methods described herein. A compiler often includes a program or set of programs to translate source text / code into target text / code. Typically, compilation of program / application code with a compiler is performed in multiple phases and passes to convert high-level programming language code into low-level machine or assembly code. However, single-pass compilers may still be used for simple compilation. A compiler may use any known compilation techniques and perform any known compiler operations, such as lexical analysis, preprocessing, parsing, semantic analysis, code generation, code translation, and code optimization.

[0141] Larger compilers often include multiple phases, but most commonly these phases are contained within two general phases: (1) a front-end, i.e., generally where syntactic processing, semantic processing, and some conversion / optimization can occur; and (2) a back-end, i.e., generally where analysis, conversions, optimizations, and code generation can occur. Some compilers reference a middle phase, illustrating the blurring of the distinction between a front-end and a back-end of a compiler. As a result, reference to insertion, linking, generation, or any other operation of a compiler can occur in any of the above-mentioned phases or passes, as well as any other known phases or passes of a compiler. As an illustrative example, a compiler potentially inserts operations, calls, functions, etc.in one or more phases of compilation, such as inserting calls / operations in a front-end phase of compilation and then converting the calls / operations into lower-level code during a conversion phase. Note that during dynamic compilation, compiler code or dynamic optimization code may insert such operations / calls, as well as optimize the code for execution at runtime. As a specific illustrative example, binary code (already compiled code) may be dynamically optimized at runtime. Here, the program code may include the dynamic optimization code, the binary code, or a combination thereof.

[0142] Similar to a compiler, a translator, such as a binary translator, translates code either statically or dynamically to optimize and / or translate code. Therefore, references to code execution, application code, program code, or another software environment may refer to: (1) the execution of a compiler program orby compiler programs, optimizing code optimizers, or translators, either dynamically or statically, to compile program code, to maintain software structures, to perform other operations, to optimize code, or to translate code; (2) the execution of the main program code including operations / calls, such as application code that has been optimized / compiled; (3) the execution of other program code, such as libraries, that are associated with the main program code to maintain software structures, to perform other software-related operations, or to optimize code; or (4) a combination thereof.

[0143] Although the present invention has been described with respect to a limited number of embodiments, those skilled in the art will appreciate that many further modifications and variations are possible. The appended claims are intended to cover all such modifications and variations that fall within the spirit and scope of the present invention.

[0144] A design can go through several stages, from creation to simulation to manufacturing. Data representing a design can represent the design in several ways. As is useful in simulations, the hardware may first be represented using a hardware description language or another functional description language. Additionally, a circuit-level model containing logic and / or transistor gates may be created at some stage of the design process. In addition, most designs eventually reach a data level that represents the physical arrangement of various devices in the hardware model. When using conventional semiconductor manufacturing techniques, the data representing the hardware model may be the data that specifies the presence or absence of various features at different mask layers for masks used to fabricate the integrated circuit.In one representation of the design, the data may be stored in the form of a machine-readable medium. A memory or a magnetic or optical storage medium, such as a disc, may be the machine-readable medium for storing information transmitted by means of an optical or electrical wave that is modulated or otherwise generated to transmit such information. When an electrical carrier wave indicating or carrying the code or design is transmitted, such that copying, buffering, or retransmission of the electrical signal occurs, a new copy is created. Thus, a communications service provider or network service provider may at least temporarily store on a tangible machine-readable medium an item, such as information encoded in a carrier wave embodying techniques of embodiments of the present invention.

[0145] A module, as used herein, refers to any combination of hardware, software, and / or firmware. As an example, a module includes hardware, such as a microcontroller, coupled to a non-transitory medium to store code adapted to be executed by the microcontroller. Therefore, in one embodiment, reference to a module refers to the hardware specifically configured to recognize and / or execute the code stored on a non-transitory medium. In another embodiment, further, the use of a module refers to the non-transitory medium that includes the code specifically adapted to be executed by the microcontroller and perform predetermined operations.And, as can be inferred, in yet another embodiment, the term "module" (in this example) may refer to the combination of microcontroller and non-transitory medium. Module boundaries illustrated as separate will typically vary and potentially overlap.

[0146] For example, a first and a second module may share hardware, software, firmware, or a combination thereof, while potentially maintaining some independent hardware, software, or firmware. In one embodiment, the use of the term logic includes hardware such as transistors, registers, or other hardware such as programmable logic devices.

[0147] The use of the term "is configured" in one embodiment refers to arranging, assembling, manufacturing, offering for sale, importing, and / or constructing a device, hardware, logic, or element to perform an intended or particular task. In this example, a device or element thereof that is not operating is still "configured" to perform an intended task if it is designed, coupled, and / or connected to perform that intended task. As a purely illustrative example, a logic gate may provide a 0 or 1 during operation. However, a logic gate that is "configured" to provide an enable signal to a clock does not include every potential logic gate that can provide a 1 or 0. Instead, the logic gate is one that is coupled in a manner such that, during operation, the 1 or 0 output enables the clock.Once again, it should be noted that the use of the term "is configured" does not require operation, but instead focuses on the latent state of a device, hardware, and / or element, where in the latent state, the device, hardware, and / or element is designed to perform a particular task when the device, hardware, and / or element is operating.

[0148] Furthermore, the use of the terms "to," "capable of," and / or "operable to," in one embodiment, refers to a device, logic, hardware, and / or element that is designed in such a way as to enable use of the device, logic, hardware, and / or element in a specified manner. As above, note that the use of "to," "capable of," and / or "operable to," in one embodiment, refers to the latent state of a device, logic, hardware, and / or element where the device, logic, hardware, and / or element is not operating, but is designed in such a way as to enable use of a device in a specified manner.

[0149] A value, as used herein, includes any known representation of a number, a state, a logical state, or a binary logical state. The use of logic levels, logic values, or logical values ​​is also referred to as 1s and 0s, which simply represent binary logical states. For example, a 1 refers to a high logic level, and a 0 refers to a low logic level. In one embodiment, a memory cell, such as a transistor or flash cell, may be capable of holding a single logical value or multiple logical values. However, other representations of values ​​have been used in computer systems. For example, the decimal number ten can also be represented as a binary value 2110 and a hexadecimal letter A. Therefore, a value includes any representation of information that may be contained in a computer system.

[0150] Additionally, states may be represented by values ​​or portions of values. For example, a first value, such as a logical one, may represent a default or initial state, while a second value, such as a logical zero, may represent a non-default state. Additionally, in one embodiment, the terms reset and set refer to a default and an updated value or state, respectively. For example, a default value potentially includes a logical high value, i.e., reset, while an updated value potentially includes a logical low value, i.e., set. It should be noted that any combination of values ​​may be employed to represent any number of states.

[0151] The embodiments of the above-mentioned methods, hardware, software, firmware, or code may be implemented by instructions or code stored on a machine-accessible, machine-readable, or computer-readable medium and executable by a processing element. A non-transitory machine-accessible / readable medium includes any mechanism that provides (i.e., stores and / or transmits information in a form readable by a machine, such as a computer or electronics). A non-transitory machine-accessible medium includes, for example, random access memory (RAM), such as static RAM (SRAM) or dynamic RAM (DRAM); ROM; a magnetic or optical storage medium; flash memory devices; an electrical storage device, optical storage devices, acoustic storage devices; another form of storage device to store information embodied by transient (propagated) signals (e.g.,carrier waves, infrared signals, digital signals), etc., which are to be distinguished from the non-transitory media that can receive information therefrom.

[0152] Instructions used to program logic to carry out embodiments of the invention may be stored in memory within the system, such as DRAM, cache, flash memory, or other memory. Furthermore, the instructions may be distributed over a network or using other computer-readable media. Thus, a machine-readable medium may include any mechanism for storing or conveying information in a (e.g.,a computer) machine-readable form, but is not limited to, floppy disks, optical drives, CDs, read-only memories (CD-ROMs), magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic or optical cards, flash memory, or any non-transitory, machine-readable memory used in the transmission of information over the Internet using electrical, optical, acoustic, or other forms of propagating signals (e.g., carrier waves, infrared signals, digital signals, etc.). Accordingly, computer-readable medium includes any type of non-transitory, machine-readable medium suitable for storing or conveying electronic instructions or information in a form readable by a machine (e.g., a computer).

[0153] The following examples relate to embodiments according to this specification. One or more embodiments may provide an apparatus, a system, a machine-readable storage, a machine-readable medium, and a method for receiving a request referencing a first agent and requesting a particular memory line to be cached in an exclusive state, sending a snoop request intended for one or more other agents, receiving a snoop response referencing a second agent, the snoop response including writing back a modified cache line to memory corresponding to the particular memory line, and sending a completed message to be addressed to the first agent, the completed message including data of the particular memory line based on the write back.

[0154] In at least one example, the modified cache line is written to the special memory line.

[0155] In at least one example, it is determined that the cache line of the second agent is a modified cache line. The completed may be sent prior to receiving responses to all snoop requests that match the request from the first agent based on determining that the cache line of the second agent is a modified cache line.

[0156] In at least one example, the snoop request includes a snoop invalidate request. The snoop invalidate request may consist of invalidating the receiving other agent's cache corresponding to the particular line or memory. The snoop invalidate request may identify the particular memory line and an instruction included in the request from the first agent.

[0157] In at least one example, a directory state may be changed to indicate that the particular memory line is associated with an exclusive state:

[0158] One or more embodiments may provide an apparatus, a system, a machine-readable memory, a machine-readable medium, and a method for receiving a request consisting of sending a request for a particular cache line in an exclusive state and receiving data corresponding to the particular cache line from memory, wherein the particular data comprises data written back to memory by another agent following the request.

[0159] In at least one example, the particular cache line is in an invalid state before the request.

[0160] In at least one example, the exclusive state is an E state indicating that a copy of the data in the particular cache line matches memory and is an exclusive copy.

[0161] In at least one example, the special data is copied to the special cache line. The special cache line may be transitioned to an exclusive state based on receiving the special data.

[0162] In at least one example, the data written back to memory by another agent includes data returned in response to a snoop corresponding to the request for the particular cache line in an exclusive state.

[0163] In at least one example, the snoop is one of a plurality of snoops, and the special data is received before responses returned for each of the snoop requests.

[0164] One or more embodiments may provide an apparatus, a system, a machine-readable storage, a machine-readable medium, and a method for receiving a request consisting of receiving an explicit writeback request, wherein the explicit writeback request corresponds to a modified cache line corresponding to a specific memory line, and wherein the explicit writeback request includes an indication to indicate that another cache requests the specific memory line, determining whether to push data of the modified cache line to the other cache before writing the data of the modified cache line to the specific memory line, and sending a completed to correspond to the explicit writeback request.

[0165] In at least one example, determining not to shift the data causes the data of the modified cache line to be written to the particular memory line.

[0166] In at least one example, the data of the modified cache line is not pushed to the other cache.

[0167] In at least one example, a directory state corresponding to the particular memory line may be changed from an exclusive state to an invalid state.

[0168] In at least one example, determining to push the data causes the modified cache line data to be sent to a first cache agent corresponding to the other cache to write the modified cache line data to be written to a corresponding cache line of the other cache.

[0169] In at least one example, a directory state corresponding to the particular memory line transitions to a state indicating that the other cache has an exclusive copy of the particular memory line.

[0170] In at least one example, the explicit writeback request comprises a single coherence protocol request from a different, second cache agent corresponding to the modified cache line.

[0171] In at least one example, determining to push the data includes determining whether the other cache is a local cache.

[0172] In at least one example, determining to push the data includes determining whether there are other outstanding requests for the particular memory line.

[0173] One or more embodiments may provide an apparatus, a system, a machine-readable storage, a machine-readable medium, and a method for receiving a request consisting of sending an explicit writeback request to a home agent, the explicit writeback request corresponding to a modified cache line corresponding to a particular memory line, the explicit writeback request including an indication to indicate that another cache is requesting the particular memory line, and receiving completion from the home agent for the explicit writeback request.

[0174] In at least one example, the modified cache line transitions from a modified state to an invalid state following sending the explicit writeback request.

[0175] In at least one example, the explicit writeback request causes data of the modified cache line to be written to the other cache without being written to the specific memory line.

[0176] In at least one example, the explicit writeback request comprises a single coherence protocol request.

[0177] In at least one example, the explicit writeback request identifies the other cache.

[0178] One or more embodiments may provide an apparatus, a system, a machine-readable memory, a machine-readable medium, and a method for receiving a request consisting of receiving a write-back flush message, identifying a set of pending writes of the memory controller to a specific persistent storage, and writing all sets of pending writes to the specific storage based on the write-back flush message.

[0179] In at least one example, the write-back flush message comprises a coherence protocol message.

[0180] In at least one example, the writeback flush message is generated by a cache agent.

[0181] In at least one example, the set of outstanding writes includes all outstanding writes of the memory controller.

[0182] The apparatus of claim 40, wherein the write-back flush message identifies the memory controller.

[0183] In at least one example, the write-back flush message specifies a memory address corresponding to the particular memory.

[0184] In at least one example, the write-back flush message corresponds to a power failure management activity.

[0185] One or more embodiments may provide an apparatus, a system, a machine-readable storage, a machine-readable medium, and a method for receiving a request consisting of determining that a particular line of a cache is in a forwarding state, receiving a request corresponding to the particular line of the cache, determining whether the forwarding state is maintained following a response to the request, and responding to the request.

[0186] In at least one example, determining whether to maintain the forwarding state includes determining a value of a configuration parameter for the agent, wherein a value of the configuration parameter determines whether to maintain the forwarding state.

[0187] In at least one example, the value of the configuration parameter may be changed. Determining whether to maintain the forwarding state may include determining to maintain the forwarding state following the response. Determining whether to maintain the forwarding state may include determining to transition from the forwarding state following the response. In at least one example, the forwarding state transitions from the forwarding state to a shared state. In at least one example, the forwarding state transitions from the forwarding state to the invalid state.

[0188] In at least one example, the request includes a snoop. Responding to the request may include forwarding data from the particular cache line to another agent.

[0189] One or more embodiments may provide an apparatus, a system, a machine-readable storage, a machine-readable medium, and a method for receiving a request, comprising providing an agent including protocol layer logic to generate a fanout snoop request, and routing layer logic to determine a plurality of agents to receive a snoop according to the fanout snoop request and to send snoop requests to each of the plurality of agents.

[0190] In at least one example, the plurality of agents are identified by a configuration parameter that designates each agent in a corresponding Fanouot cone.

[0191] In at least one example, the configuration parameter identifies each agent by address.

[0192] In at least one example, it may be determined whether a fanout snoop may be used to snoop one or more agents.

[0193] In at least one example, the agent is a home agent and the snoop requests may each include a snoop to obtain cache data in any of a forwarding or shared state.

[0194] In at least one example, the snoop requests each include a snoop to obtain cache data in any one of a modified, exclusive, forwarded, or shared state.

[0195] In at least one example, the snoop requests each include a snoop to obtain cache data in any of a modified or exclusive state.

[0196] In at least one example, the snoop requests each comprise a snoop to the cache of the corresponding agent, flushing data in the modified state into memory.

[0197] In at least one example, snoop responses may be received for one or more of the snoop requests.

[0198] One or more examples may further provide an agent including a layered protocol stack including a protocol layer, wherein the protocol layer initiates a read invalidate request that accepts exclusive coherency state data.

[0199] One or more examples may further provide an agent including a layered protocol stack including a protocol layer, wherein the protocol layer initiates an invalidate requesting exclusive ownership of a cache line without receiving data and with an indication to write back the cache line.

[0200] In at least one example, the cache line writeback occurs within a short time window.

[0201] One or more examples may further provide an agent including a layered protocol stack including a protocol layer, wherein the protocol layer initiates a write-back flush request that causes a flush of data to persistent storage.

[0202] One or more examples may further provide an agent including a layered protocol stack including a protocol layer, the protocol layer initiating a single fanout snoop request that causes a snoop request to be generated to peer agents within a fanout cone.

[0203] One or more examples may further provide an agent including a layered protocol stack including a protocol layer, wherein the protocol layer initiates an explicit writeback request with cache push hint to a home agent that a referenced cache line may be pushed to a local cache without writing the data to memory.

[0204] In at least one example, the cache line may store in the M state.

[0205] One or more examples may further provide an agent including a layered protocol stack including a protocol layer, the protocol layer initiating forwarding of shared data while maintaining a forwarding state to be associated with the shared data.

[0206] References in this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the uses of the phrase "in one embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0207] In the foregoing description, a detailed description has been given with reference to specific exemplary embodiments. However, it will be apparent that various modifications and changes may be made therein without departing from the broader spirit and scope of the invention as set forth in the appended claims. The description and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. Furthermore, the above use of embodiment and other exemplary language does not necessarily refer to the same embodiment or example, but can refer to different and various embodiments, as well as potentially the same embodiment.

[0208] According to one aspect of the present invention, an apparatus is provided comprising a special agent including protocol layer logic to receive a request referencing a first agent and requesting a special memory line to be cached in an exclusive state, send a snoop request intended for one or more other agents, receive a snoop response referencing a second agent, the snoop response including a write-back of a modified cache line to memory corresponding to the special memory line, and send a complete to be addressed to the first agent, the complete including data from the special memory line based on the write-back.

[0209] According to one embodiment, the protocol layer logic further causes the modified cache line to be written to the special memory line.

[0210] According to one embodiment, the protocol layer logic further determines that the cache line of the second agent is a modified cache line.

[0211] According to one embodiment, the completed is sent prior to receiving responses to all snoop requests corresponding to the request from the first agent based on determining that the cache line of the second agent is a modified cache line.

[0212] In one embodiment, the snoop request comprises a snoop invalidation request.

[0213] In another embodiment, the snoop invalidate request invalidates the receiving other agent's cache corresponding to the particular line or memory.

[0214] According to one embodiment, the snoop invalidate request identifies the particular memory line and a command included in the request from the first agent.

[0215] In one embodiment, the protocol layer logic further transitions to a directory state to indicate that the particular memory line is associated with an exclusive state.

[0216] According to one aspect of the present invention, an apparatus is defined comprising an agent including protocol layer logic to send a request for a particular cache line in an exclusive state and receive data from memory corresponding to the particular cache line, wherein the particular data comprises data written back to memory by another agent following the request.

[0217] According to one embodiment, the particular cache line is in an invalid state prior to the request.

[0218] According to another embodiment, the exclusive state is an E state indicating that a copy of the data in the particular cache line matches memory and is an exclusive copy.

[0219] In one embodiment, the protocol layer logic further copies the specific data to the specific cache line.

[0220] In another embodiment, the protocol layer logic further transitions the particular cache line to an exclusive state based on receiving the particular data.

[0221] In one embodiment, the data written back to memory by another agent includes data returned in an exclusive state in response to a snoop corresponding to the request for the particular cache line.

[0222] In another embodiment, the snoop is one of a plurality of snoops and the special data is received prior to responses returned for each of the snoop requests.

[0223] According to one aspect of the present invention, a method is provided, comprising receiving a request from a first agent to cache a particular memory line in an exclusive state, sending a snoop request to one or more other agents, receiving a snoop response from a second agent, wherein the snoop response includes a write-back to memory of a modified cache line of the second agent corresponding to the particular memory line, and sending a complete to the first agent, wherein the complete includes data from the particular memory line based on the write-back.

[0224] According to one embodiment, the method further comprises sending a request to a memory controller to write the modified cache line to the specific memory line.

[0225] According to another embodiment, the method further comprises determining that the cache line of the second agent is a modified cache line.

[0226] In one embodiment, the completed is sent before receiving responses to all snoop requests that match the request from the first agent.

[0227] According to another embodiment, the method further comprises skipping a directory state to indicate that the particular memory line is associated with an exclusive state.

[0228] According to one aspect of the present invention, a system is provided comprising a memory controller of a specific memory location, a home agent corresponding to the specific memory location, and a first cache agent of a first device, wherein the home agent receives a request referencing a first agent and requests a specific memory line to be cached in an exclusive state, sends a snoop request intended for one or more other agents, receives a snoop response referencing a second agent, the snoop response including a write-back of a modified cache line to memory corresponding to the specific memory line, and sends a complete to be addressed to the first agent, the complete including data from the specific memory line based on the write-back.

[0229] According to one embodiment, the system further comprises the second cache agent receiving the snoop request, identifying from the snoop request the request from the first cache agent to cache the particular line of the particular memory location in an exclusive state, sending the snoop response to the home agent.

[0230] According to another embodiment, the first cache agent sends the request to the home agent, and upon receiving the completion, writes the data of the particular memory line to a corresponding cache line, and transitions the corresponding cache line to an exclusive state.

[0231] In one embodiment, the request from the first cache agent comprises a coherence protocol request.

[0232] In another embodiment, the completed is sent from the first cache before receiving responses to all snoop requests that match the request.

[0233] According to one aspect of the present invention, an apparatus is defined comprising an agent including protocol layer logic to receive an explicit writeback request, the explicit writeback request corresponding to a modified cache line corresponding to a specific memory line, and the explicit writeback request including an indication to indicate that another cache requests the specific memory line, determine whether to push data of the modified cache line to the other cache before writing the data of the modified cache line to the specific memory line, and send a completed to correspond to the explicit writeback request.

[0234] According to one embodiment, determining not to shift the data causes the data of the modified cache line to be written to the particular memory line.

[0235] According to another embodiment, the data of the modified cache line is not shifted to the other cache.

[0236] In one embodiment, the protocol layer logic further transitions a directory state corresponding to the particular memory line from an exclusive state to an invalid state.

[0237] In another embodiment, determining to push the data causes the modified cache line data to be sent to a first cache agent corresponding to the other cache to write the modified cache line data to be written to a corresponding cache line of the other cache.

[0238] According to one embodiment, the protocol layer logic further transitions a directory state corresponding to the particular memory line to a state indicating that the other cache has an exclusive copy of the particular memory line.

[0239] According to one embodiment, the explicit writeback request comprises a single coherence protocol request from a different, second cache agent corresponding to the modified cache line.

[0240] According to another embodiment, determining to push the data includes determining whether the other cache is a local cache.

[0241] In one embodiment, determining to push the data includes determining whether there are other outstanding requests for the particular memory line.

[0242] According to one aspect of the present invention, an apparatus is provided comprising an agent including protocol layer logic to send an explicit writeback request to a home agent, the explicit writeback request corresponding to a modified cache line corresponding to a particular memory line, the explicit writeback request including a hint to indicate that another cache is requesting the particular memory line, and to receive completion from the home agent for the explicit writeback request.

[0243] According to one embodiment, the protocol layer logic further transitions the modified cache line from a modified state to an invalid state following sending the explicit writeback request.

[0244] According to another embodiment, the explicit writeback request causes data of the modified cache line to be written to the other cache without being written to the specific memory line.

[0245] In one embodiment, the explicit writeback request comprises a single coherence protocol request.

[0246] In another embodiment, the explicit writeback request identifies the other cache.

[0247] According to one aspect of the present invention, an apparatus is defined comprising a memory controller to receive a write-back flush message, identify a set of outstanding writes of the memory controller to a dedicated persistent memory, and write the entire set of outstanding writes to the dedicated memory based on the write-back flush message.

[0248] According to one embodiment, the write-back flush message comprises a coherence protocol message.

[0249] In one embodiment, the write-back flush message is generated by a cache agent.

[0250] According to another embodiment, the set of outstanding writes includes all outstanding writes of the memory controller.

[0251] In one embodiment, the write-back flush message identifies the memory controller.

[0252] According to one embodiment, the write-back flush message identifies a memory address corresponding to the particular memory.

[0253] According to another embodiment, the write-back flush message corresponds to a power failure management activity.

[0254] According to one aspect of the present invention, an apparatus is provided comprising an agent including protocol layer logic to identify that a particular line of a cache is in a forwarding state, receive a request corresponding to the particular line of the cache, determine whether to maintain the forwarding state following a response to the request, and respond to the request.

[0255] According to one embodiment, determining whether to maintain the forwarding state includes determining a value of a configuration parameter for the agent, wherein a value of the configuration parameter identifies whether to maintain the forwarding state.

[0256] According to another embodiment, the protocol layer logic further changes the value of the configuration parameter.

[0257] In one embodiment, determining whether to maintain the forwarding state comprises determining to maintain the forwarding state following the response.

[0258] According to another embodiment, determining whether to maintain the forwarding state comprises determining to transition from the forwarding state following the response.

[0259] In one embodiment, the forwarding state transitions from a forwarding state to a shared state.

[0260] In another embodiment, the forwarding state transitions from the forwarding state to the invalid state.

[0261] According to another embodiment, the request comprises a snoop.

[0262] In one embodiment, responding to the request includes forwarding data from the particular cache line to another agent.

[0263] According to one aspect of the present invention, an apparatus is provided comprising an agent including protocol layer logic to generate a fanout snoop request, and routing layer logic to identify a plurality of agents to receive a snoop according to the fanout snoop request, and to send snoop requests to each of the plurality of agents.

[0264] According to one embodiment, the routing layer logic identifies the plurality of agents from a configuration parameter that identifies each agent in a corresponding fanout cone.

[0265] According to another embodiment, the configuration parameter identifies each agent by address.

[0266] In one embodiment, the protocol layer logic further determines whether a fanout snoop can be used to snoop one or more agents.

[0267] According to one embodiment, the agent is a home agent.

[0268] According to another embodiment, the snoop requests each comprise a snoop to obtain cache data in one of a forwarded and shared state.

[0269] In one embodiment, the snoop requests each include a snoop to obtain cache data in one of a modified, exclusive, forward, or shared state.

[0270] In another embodiment, the snoop requests each include a snoop to obtain cache data in a modified or exclusive state.

[0271] According to one embodiment, the snoop requests each comprise a snoop to the cache of the corresponding agent, flushing data in the modified state into memory.

[0272] According to another embodiment, the protocol layer logic further receives snoop responses for one or more of the snoop requests.

[0273] According to one aspect of the present invention, an apparatus is provided comprising an agent including a layered protocol stack including a protocol layer, wherein the protocol layer initiates a read invalidate request that accepts exclusive coherency state data.

[0274] According to one aspect of the present invention, an apparatus is defined comprising an agent including a layered protocol stack including a protocol layer, wherein the protocol layer initiates an invalidate requesting exclusive ownership of a cache line without receiving data and with an indication to write back the cache line.

[0275] According to one embodiment, the write back of the cache line occurs within a time window.

[0276] According to one aspect of the present invention, an apparatus is defined comprising an agent including a layered protocol stack including a protocol layer, wherein the protocol layer initiates a write-back flush request that causes a flush of data to persistent storage.

[0277] According to one aspect of the present invention, an apparatus is provided comprising an agent including a layered protocol stack including a protocol layer, wherein the protocol layer initiates a single fanout snoop request that causes a snoop request to be generated to peer agents within a fanout cone.

[0278] According to one aspect of the present invention, an apparatus is defined comprising an agent including a layered protocol stack including a protocol layer, wherein the protocol layer initiates an explicit writeback request with cache push indication to a home agent that a referenced cache line may be pushed to a local cache without writing the data to memory.

[0279] According to one embodiment, the cache line may store in the M state.

[0280] According to one aspect of the present invention, an apparatus is defined comprising an agent including a layered protocol stack including a protocol layer, the protocol layer initiating forwarding of shared data while maintaining a forwarding state to be associated with the shared data.

Claims

[1] Device comprising: a node comprising at least one processor, a cache and a first coherence agent (705) to: receive a snoop request, wherein the snoop request is to be received from a home agent (710), wherein the snoop request comprises a snoop invalidation request, wherein the snoop request is to correspond to a request sent to the home agent (710) from a second coherence agent (720), and wherein the snoop request comprises an address field encoded with a node identifier (NID) of the home agent (710); generate a snoop response, the snoop response comprising a command field, a target NID field, a home transaction identifier (TID) field, and data, the command field being encoded to indicate that the snoop response comprises an implicit write-back, the target NID field being encoded with the NID of the home agent (710), the home TID field being encoded to identify a resource allocated by the home agent (710) to process the snoop request, and the data being written back to a memory (715); send the snoop response to the home agent (710); and to change a cache state based on the snoop response. [2] The apparatus of claim 1, wherein the state of the cache is one of a set of defined states. [3] The apparatus of claim 2, wherein the set of defined states comprises a modified state, an exclusive state, a shared state, and an invalid state. [4] The apparatus of claim 1, wherein the state of the cache is changed to an invalid state. [5] The apparatus of claim 4, wherein the state of the cache is changed from a modified state to the invalid state. [6] The apparatus of claim 5, wherein the snoop response is to include the implicit writeback based on the modified state. [7] The apparatus of claim 1, wherein the snoop invalidation request is based on an exclusive directory state associated with the home agent (710). [8] The apparatus of claim 1, wherein the data is written back to memory (710) based on the implicit write back. [9] The apparatus of claim 8, wherein the data is flushed from the cache concurrently with the writing of the data back to the memory (710). [10] Procedure comprising the following steps: Receiving, in a home agent (710) of a memory, a read request from a first caching agent (720), the read request corresponding to a particular line of the memory; Sending a snoop request to a second caching agent (705) in response to the read request, the snoop request comprising a command field, an address field, a destination node identifier (NID) field, a request (NID) field, a requester transaction identifier (TID) field, and a home TID field, the command field of the snoop request being encoded to indicate that the snoop request includes a snoop invalidation, and the home TID field of the snoop request identifying a resource allocated by the home agent (710) to service the snoop request; Receiving a snoop response from the second caching agent (705) in response to the snoop request, the snoop response comprising a command field, a target NID field, a home TID field, and data from a cache corresponding to the second caching agent (705), the command field of the snoop response being encoded to indicate that the snoop response includes an implicit writeback; Writing the data to the specific line of memory (715); Changing a state of the particular row of the memory (715) to an invalid state based on writing the data to the particular row; Sending a completion to the first caching agent (720) in response to the read request; and Changing the state of the particular line of memory (715) to an exclusive state based on the completion. [11] The method of claim 10, wherein the state of the particular line of the memory (715) is changed from an exclusive state to an invalid state. [12] The method of claim 10, wherein the read request comprises a request to cause the first caching agent (720) to acquire ownership of the particular line of memory in an exclusive state. [13] The method of claim 12, wherein the closure does not include any data. [14] The method of claim 10, wherein the closure comprises the data. [15] The method of claim 10, wherein the read request comprises a read invalidation (RdInv) request. [16] The method of claim 15, wherein the invalid read request corresponds to an unknown directory condition corresponding to the particular line of memory (715). [17] The method of claim 15, wherein the invalidate read request indicates that an exclusive copy of the particular row is requested. [18] At least one non-transitory machine-accessible storage medium having instructions stored thereon, which instructions, when executed on a machine, cause the machine to: receive a snoop request, wherein the snoop request is to be received from a home agent (710), wherein the snoop request comprises a snoop invalidation request, wherein the snoop request is to correspond to a request sent to the home agent (710) from a second coherence agent (720), and wherein the snoop request comprises an address field encoded with a node identifier (NID) of the home agent (710); generate a snoop response, the snoop response comprising a command field, a target NID field, a home transaction identifier (TID) field, and data, the command field being encoded to indicate that the snoop response comprises an implicit write-back, the target NID field being encoded with the NID of the home agent (710), the home TID field being encoded to identify a resource allocated by the home agent (710) to process the snoop request, and the data being written back to a memory (715); send the snoop response to the home agent (710); and to change a cache state based on the snoop response. [19] At least one non-transitory machine-accessible storage medium having stored thereon instructions which, when executed on a machine, cause the machine to: send a snoop request to a second caching agent (705) in response to a read request from a first caching agent (720), the snoop request comprising a command field, an address field, a destination node identifier (NID) field, a request (NID) field, a requester transaction identifier (TID) field, and a home TID field, the command field of the snoop request being encoded to indicate that the snoop request includes a snoop invalidation, and the home TID field of the snoop request identifying a resource allocated by a home agent (710) to service the snoop request; receive a snoop response from the second caching agent (705) in response to the snoop request, the snoop response comprising a command field, a target NID field, a home TID field, and data from a cache corresponding to the second caching agent (705), the command field of the snoop response being encoded to indicate that the snoop response includes an implicit writeback; to write the data into the specific line of the memory (715); change a state of the particular line of memory to an invalid state based on writing the data to the particular line; send a completion to the first caching agent (720) in response to the read request; and to change the state of the specific line of memory to an exclusive state based on completion. [20] System comprising: a first node comprising a first processor, a first cache, and a first caching agent (720); a second node comprising a second processor, a second cache, and a second caching agent (705); and a third node comprising a third processor, a memory (715), and a third caching agent, the third caching agent comprising protocol logic to: receive a read request from the first caching agent (720) for data from a particular line of memory; send a snoop request to the second caching agent (705) in response to the read request, the snoop request comprising a command field encoded to indicate that the snoop request comprises a snoop invalidation; receive a snoop response from the second caching agent (705) in response to the snoop request, the snoop response comprising data from the second cache and a command field encoded to indicate that the snoop response includes an implicit writeback; write the data to the particular row of memory (715) in response to the snoop response; change a directory state of the specified line to an invalid state based on the implicit writeback; send a completion to the first caching agent (720) in response to the read request, the completion comprising the data; and change the directory state of the specific row to an exclusive state based on sending the completion. [21] The system of claim 20, wherein the third caching agent comprises a home agent (710). [22] The system of claim 20, wherein the first, second and third nodes are connected together by a plurality of links in the system. [23] The system of claim 22, wherein the plurality of connections are configured according to a cache-coherent multilayer interconnect protocol. [24] System comprising: Means for receiving a snoop request, wherein the snoop request is to be received from a home agent (710), wherein the snoop request comprises a snoop invalidation request, wherein the snoop request is to correspond to a request sent to the home agent (710) from a second coherence agent (720), and wherein the snoop request comprises an address field encoded with a node identifier (NID) of the home agent; Means for generating a snoop response, the snoop response comprising a command field, a target NID field, a home transaction identifier (TID) field, and data, the command field to be encoded to indicate that the snoop response comprises an implicit writeback, the target NID field being encoded with the NID of the home agent (710), the home TID field being encoded to identify a resource allocated by the home agent (710) to service the snoop request, and the data to be written back to a memory (715); means for sending the snoop response to the home agent (710); and Means to change a cache state based on the snoop response.

Citation Information

Patent Citations

  • PCI express enhancements and extensions

    US20080109565A1

  • Coherency for write-back cache in a system designed for write-through cache including write-back latency control

    US5524234A

  • Fast, dual ported cache controller for data processors in a packet switched cache coherent multiprocessor system

    US5644753A