Implicit Directory Status Updates
Through the serial point-to-point interconnect architecture and buffer memory protocol, the performance bottlenecks of the interconnect architecture under high-speed requirements in the prior art are solved, and efficient processor communication and bandwidth utilization are achieved, suitable for high-performance computing and mobile devices.
Patent Information
- Application Number
- CN202011523261.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2015-03-27
- Filing Date
- 2016-02-26
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2036-02-26
AI Technical Summary
The interconnect architecture in existing computing systems is difficult to meet the bandwidth and communication efficiency of high-performance computing when the processor needs to increase at a high-speed, resulting in an increase in communication complexity between slots and devices, affecting system performance.
Adopting serial point-to-point interconnect architecture and buffer memory protocol, the design of the layered protocol stack and physical layer can realize high-performance interconnection (HPI) system, supports multi-core processors and multi-slot configurations, and uses microchips and slotted microchips. Format optimizes link width and data transmission efficiency.
Improves communication efficiency and bandwidth utilization among processors, reduces power consumption, is suitable for high-performance computing and mobile devices, and enhances system performance and flexibility.
Smart Images

Figure CN112433683B_ABST
Abstract
Description
[0001] This application is a divisional application, the parent application number is 201680018780X, the application date is February 26, 2016, and the name of the invention is “Implicit Directory Status Update”.
[0002] CROSS-REFERENCE TO RELATED APPLICATIONS
[0003] This application claims the benefit of and priority to U.S. Non-Provisional Patent Application No. 14 / 670,526, filed on March 27, 2015, entitled “IMPLIEDDIRECTORY STATE UPDATES,” which is incorporated herein by reference in its entirety. Technical Field
[0004] The present disclosure relates generally to the field of computing architectures, and more particularly to buffer memory protocols. Background Art
[0005] Advances in semiconductor processing and logic design have allowed for an increase in the amount of logic that can be present on an integrated circuit device. As a corollary, computer system configurations have evolved from a single or multiple integrated circuits in a system to multiple cores, multiple hardware threads, and multiple logical processors present on each integrated circuit and other interfaces integrated within such processors. A processor or integrated circuit typically includes a single physical processor die, where the processor die may include any number of cores, hardware threads, logical processors, interfaces, memory, controller hubs, etc.
[0006] Smaller computing devices are becoming more popular due to the greater ability to fit more processing power in a smaller package. Smart phones, tablets, ultra-thin notebooks, and other user devices have grown exponentially. However, these smaller devices rely on servers for both complex processing and data storage that exceed the form factor. Therefore, the demand in the high-performance computing market (i.e., server space) has also increased. For example, in modern servers, there is usually not only a single processor with multiple cores, but also multiple physical processors (also known as multiple sockets) to increase computing power. But as processing power grows along with the number of devices in a computing system, communication between sockets and other devices becomes more critical.
[0007] In fact, interconnects have grown from more traditional multi-drop buses that originally handled electrical communications to fully developed interconnect architectures that facilitate fast communications. Unfortunately, with the need for future processors to consume at even higher rates, corresponding demands are placed on the capabilities of existing interconnect architectures. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] Figure 1The figure shows a simplified block diagram of a system including a serial point-to-point interconnect to connect I / O devices in a computer system according to one embodiment.
[0009] Figure 2 The figure shows a simplified block diagram of a layered protocol stack according to one embodiment.
[0010] Figure 3 An embodiment of a transaction descriptor is illustrated.
[0011] Figure 4 The figure shows an embodiment of a serial point-to-point link.
[0012] Figure 5 An embodiment of a potential high performance interconnect (HPI) system configuration is illustrated.
[0013] Figure 6 An embodiment of a layered protocol stack associated with HPI is illustrated.
[0014] Figure 7 The figure shows a representation of an example state machine.
[0015] Figure 8 A representation of an example flit sent over an example 20-lane data link is shown.
[0016] Fig. 9 A representation of an example flits sent over an example 8-lane data link is shown.
[0017] Fig.10 A representation of an example multi-slot microchip is shown.
[0018] Fig.11 A representation of an example flits sent over an example 8-lane data link is shown.
[0019] Fig.12 A representation of an example multi-slot flit including debug messages is shown.
[0020] Fig.13 The figure shows a representation of a system employing a buffer memory protocol.
[0021] Fig.14 The figure shows a representation of a buffer device supporting a buffer memory protocol.
[0022] Figures 15A-15C The figure shows a representation of an example flits in a buffer memory protocol.
[0023] Figures 16A-16C The diagram illustrates example communications between a buffer device and a host device according to some implementations.
[0024] Fig.17A representation of an example embodiment of a cache line is shown in accordance with some implementations.
[0025] Fig.18 An embodiment of a block diagram of a computing system including multiple processor sockets is illustrated.
[0026] Fig.19 Another embodiment of a block diagram of a computing system including a multi-core processor is illustrated.
[0027] Like reference numbers and designations in the various drawings indicate like elements. DETAILED DESCRIPTION
[0028] In the following description, numerous specific details are set forth, such as the following examples: specific processor types and system configurations, specific hardware structures, specific architectures and microarchitecture details, specific register configurations, specific instruction types, specific system components, specific processor pipeline levels, specific interconnect layers, specific grouping / transaction configurations, specific transaction names, specific protocol exchanges, specific link widths, specific implementations, and operations, etc., in order to provide a thorough understanding of the present invention. However, it will be apparent to those skilled in the art that these specific details are not necessarily required to practice the subject matter of the present disclosure. In other instances, very detailed descriptions of known components or methods (such as specific and alternative processor architectures, specific logic circuits / codes for the described algorithms, specific firmware codes, low-level interconnect operations, specific logic configurations, specific manufacturing techniques and materials, specific compiler implementations, specific expressions of algorithms in code, specific power-down and gating techniques / logic, and other specific operational details of computer systems) have been avoided in order to avoid unnecessarily obscuring the present disclosure.
[0029] Although the following embodiments may be described with reference to energy conservation, energy efficiency, processing efficiency, etc. in a specific integrated circuit (such as in a computing platform or microprocessor), other embodiments may be applied to other types of integrated circuits and logic devices. Similar techniques and teachings of the embodiments described herein may be applied to other types of circuits or semiconductor devices that may also benefit from such features. For example, the disclosed embodiments are not limited to server computer systems, desktop computer systems, laptop computers, Ultrabooks, etc. TM(Ultrabook), but can also be used in other devices such as handheld devices, smart phones, tablets, other thin notebooks, system-on-chip (SOC) devices, and embedded applications. Some examples of handheld devices include cellular phones, Internet protocol devices, digital cameras, personal digital assistants (PDAs), and handheld PCs. Here, similar techniques for high-performance interconnects can be applied to increase performance (or even save power) in low-power interconnects. Embedded applications typically include microcontrollers, digital signal processors (DSPs), systems-on-chip, network computers (NetPCs), set-top boxes, network hubs, wide area network (WAN) switches, or any other system that can perform the functions and operations taught below. In addition, the devices, methods, and systems described herein are not limited to physical computing devices, but may also involve software optimization for energy saving and energy efficiency. As can be easily apparent in the following description, embodiments of the methods, devices, and systems described herein (whether or not with reference to hardware, firmware, software, or a combination thereof) can be considered to be critical to the future of "green technology" that is weighed against performance considerations.
[0030] As computing systems continue to advance, the components therein are becoming more complex. The interconnect architectures that couple components and communicate between components are also increasing in complexity to ensure that bandwidth requirements are met for optimal component operation. In addition, different market segments require different aspects of the interconnect architecture to suit the respective markets. For example, servers require higher performance, while the mobile ecosystem can sometimes sacrifice overall performance to save power. However, the sole purpose of most configurations is to provide the highest possible performance with maximum power savings. In addition, a variety of different interconnects can potentially benefit from the subject matter described herein.
[0031] Peripheral Component Interconnect Express (PCI) (PCIe) interconnect fabric architecture and Quick Path Interconnect (QPI) fabric architecture and other examples can potentially be improved according to one or more principles described herein and other examples. For example, the primary goal of PCIe is to enable components and devices from different vendors to interoperate in an open architecture, across multiple market segments; clients (desktop and mobile), servers (standard and enterprise), and embedded and communication devices. PCI Express is a high-performance, general-purpose I / O interconnect that is defined for a wide variety of future computing and communication platforms. Some PCI attributes, such as its usage model, load-store architecture, and software interface, have been maintained through its revisions, and previous parallel bus implementations have been replaced by highly scalable, all-serial interfaces. Recent versions of PCI Express utilize point-to-point interconnects, switch-based technologies, and advances in packetized protocols to deliver new performance levels and features. Power management, quality of service (QoS), hot-plug / hot-swap support, data integrity, and error handling are among some of the advanced features supported by PCI Express. Although the primary discussion herein refers to a new high performance interconnect (HPI) architecture, aspects of the invention described herein may be applied to other interconnect architectures, such as a PCIe compliant architecture, a QPI compliant architecture, a MIPI compliant architecture, a high performance architecture, or other known interconnect architectures.
[0032] See also Figure 1 , illustrates an embodiment of a construction consisting of point-to-point links interconnecting a set of components. System 100 includes a processor 105 and a system memory 110 coupled to a controller hub 115. Processor 105 may include any processing element, such as a microprocessor, a main processor, an embedded processor, a coprocessor, or other processor. Processor 105 is coupled to controller hub 115 via a front side bus (FSB) 106. In one embodiment, FSB 106 is a serial point-to-point interconnect as described below. In another embodiment, link 106 includes a serial, differential interconnect architecture that complies with different interconnect standards.
[0033] System memory 110 includes any memory device, such as random access memory (RAM), non-volatile (NV) memory, or other memory accessible by devices in system 100. System memory 110 is coupled to controller hub 115 via memory interface 116. Examples of memory interfaces include double data rate (DDR) memory interfaces, dual channel DDR memory interfaces, and dynamic RAM (DRAM) memory interfaces.
[0034] In one embodiment, the controller hub 115 may include a root hub, root complex, or root controller, such as in a PCIe interconnect hierarchy. Examples of controller hub 115 include chipsets, memory controller hubs (MCHs), north bridges, interconnect controller hubs (ICHs), south bridges, and root controller / hubs. The term chipset often refers to two physically separate controller hubs, for example, a memory controller hub (MCH) coupled to an interconnect controller hub (ICH). Note that current systems often include an MCH integrated with the processor 105, and the controller 115 will communicate with I / O devices in a similar manner as described below. In some embodiments, peer-to-peer routing is optionally supported through the root complex 115.
[0035] Here, controller hub 115 is coupled to switch / bridge 120 via serial link 119. Input / output modules 117 and 121, which may also be referred to as interfaces / ports 117 and 121, may include / implement a layered protocol stack to provide communication between controller hub 115 and switch 120. In one embodiment, multiple devices can be coupled to switch 120.
[0036] The switch / bridge 120 routes packets / messages from the device 125 upstream (i.e., above the hierarchy) toward the root complex to the controller hub 115 and downstream (i.e., below the hierarchy away from the root controller), from the processor 105 or the system memory 110 to the device 125. In one embodiment, the switch 120 is referred to as a logical assembly of multiple virtual PCI to PCI bridge devices. The device 125 includes any internal or external device or component to be coupled to the electronic system, such as an I / O device, a network interface controller (NIC), an expansion card, an audio processor, a network processor, a hard drive, a storage device, a CD / DVD ROM, a monitor, a printer, a mouse, a keyboard, a router, a portable storage device, a FireWire device, a Universal Serial Bus (USB) device, a scanner, and other input / output devices. Usually in PCIe lingo, such as a device, is referred to as an endpoint. Although not specifically shown, the device 125 may include a bridge (e.g., a PCIe to PCI / PCI-X bridge) to support legacy or other versions of the device or interconnection structure supported by such a device.
[0037] The graphics accelerator 130 may also be coupled to the controller hub 115 via a serial link 132. In one embodiment, the graphics accelerator 130 is coupled to the MCH, which is coupled to the ICH. The switch 120, and accordingly the I / O devices 125, are then coupled to the ICH. The I / O modules 131 and 118 will also implement a layered protocol stack to communicate between the graphics accelerator 130 and the controller hub 115. Similar to the MCH discussion above, the graphics controller or graphics accelerator 130 itself may be integrated into the processor 105.
[0038] Go to Figure 2 , illustrates an embodiment of a layered protocol stack. The layered protocol stack 200 may include any form of layered communication stack, such as a QPI stack, a PCIe stack, a next generation high performance computing interconnect (HPI) stack, or other layered stacks. In one embodiment, the protocol stack 200 may include a transaction layer 205, a link layer 210, and a physical layer 220. Figure 1 Interfaces such as interfaces 117, 118, 121, 122, 126, and 131 in may be represented as a communication protocol stack 200. A representation as a communication protocol stack may also be referred to as a module or interface that implements / includes the protocol stack.
[0039] Packets can be used to transfer information between components. Packets can be formed in the transaction layer 205 and the data link layer 210 to convey information from the transmitting component to the receiving component. As the transmitted packets flow through the other layers, they are expanded with additional information used to process the packets at those layers. The reverse process occurs on the receiving side and the packet is transformed from its physical layer 220 representation to the data link layer 210 representation and finally (for transaction layer packets) into a form that can be processed by the transaction layer 205 of the receiving device.
[0040] In one embodiment, the transaction layer 205 may provide an interface between the processing core of the device and the interconnect architecture (such as the data link layer 210 and the physical layer 220). In this regard, the primary responsibilities of the transaction layer 205 may include the assembly and disassembly of packets (i.e., transaction layer packets or TLPs). The translation layer 205 may also manage credit-based flow control for the TLPs. In some implementations, split transactions, i.e., transactions with requests and responses divided in time, may be utilized, thereby allowing the link to carry other traffic while the target device collects data for the response, as well as other examples.
[0041] Credit-based flow control can be used to implement virtual channels and networks using interconnect fabrics. In one example, a device can advertise an initial amount of credit for each receive buffer in the transaction layer 205. External devices at the opposite end of the link, such as Figure 1 The controller hub 115 in the embodiment of the present invention can count the number of credits consumed by each TLP. If the transaction does not exceed the credit limit, the transaction can be transmitted. When a response is received, the credit amount is restored. An example of the advantage of this credit scheme is that if the credit limit is not counted, the waiting time for the credit to be returned does not affect performance, as well as other potential advantages.
[0042] In one embodiment, the four transaction address spaces may include a configuration address space, a memory address space, an input / output address space, and a message address space. Memory space transactions include one or more of read requests and write requests to transfer data to or from a memory mapped location. In one embodiment, memory space transactions can use two different address formats, for example, a short address format (such as a 32-bit address) or a long address format (such as a 64-bit address). Configuration space transactions can be used to access the configuration space of various devices connected to the interconnect. Transactions for configuration space may include read requests and write requests. Message space transactions (or simply, messages) may also be defined to support in-band communication between interconnect agents. Therefore, in an example embodiment, the transaction layer 205 may assemble a packet header / payload 206.
[0043] Quick Reference Figure 3 , illustrates an example embodiment of a transaction layer packet descriptor. In one embodiment, a transaction descriptor 300 may be a mechanism for carrying transaction information. In this regard, the transaction descriptor 300 supports identification of transactions in the system. Other potential uses include tracking modifications to default transaction ordering and association of transactions with channels. For example, the transaction descriptor 300 may include a global identifier field 302, an attribute field 304, and a channel identifier field 306. In the illustrated example, the global identifier field 302 is depicted as including a local transaction identifier field 308 and a source identifier field 310. In one embodiment, the global transaction identifier 302 is unique to all outstanding requests.
[0044] According to one implementation, the local transaction identifier field 308 is a field generated by the requesting agent and may be unique to all outstanding requests requiring completion for the requesting agent. Additionally, in this example, the source identifier 310 uniquely identifies the requesting agent within the interconnect hierarchy. Thus, along with the source ID 310, the local transaction identifier 308 field provides a global identification of a transaction within the hierarchy domain.
[0045] Attribute field 304 specifies the characteristics and relationships of the transaction. In this regard, attribute field 304 is potentially used to provide additional information that allows modification of the default processing of the transaction. In one embodiment, attribute field 304 includes priority field 312, reserved field 314, sort field 316, and no-snoop field 318. Here, priority subfield 312 can be modified by the initiator to assign priorities to transactions. Reserved attribute field 314 is reserved for future use or vendor-defined use. Possible use models using priority or security attributes can be implemented using reserved attribute fields.
[0046] In this example, the sort attribute field 316 is used to provide optional information that conveys the type of sorting that can modify the default sorting rules. According to an example implementation, a sort attribute of "0" indicates that the default sorting rules are applied, wherein a sort attribute of "1" indicates a relaxed sorting, wherein writes can pass writes in the same direction, and read completions can pass writes in the same direction. The listening attribute field 318 is used to determine whether the transaction is listened. As shown, the channel ID field 306 identifies the channel with which the transaction is associated.
[0047] Back to Figure 2 As discussed above, the link layer 210 (also referred to as the data link layer 210) may act as an intermediate level between the transaction layer 205 and the physical layer 220. In one embodiment, the responsibility of the data link layer 210 is to provide a reliable mechanism for exchanging transaction layer packets (TLPs) between two components on a link. One side of the data link layer 210 accepts the TLPs assembled by the transaction layer 205, applies a packet sequence identifier 211 (i.e., an identification number or packet number), calculates and applies an error detection code (i.e., CRC 212), and presents the modified TLP to the physical layer 220 for transmission across the physical to an external device.
[0048] In one example, the physical layer 220 includes a logical sub-block 221 and an electrical sub-block 222 to physically transmit packets to an external device. Here, the logical sub-block 221 is responsible for the "digital" functions of the physical layer 221. In this regard, the logical sub-block may include a transmission section to prepare outgoing information for transmission by the physical sub-block 222, and a receiver section to identify and prepare received information before delivering it to the link layer 210.
[0049] Physical block 222 includes a transmitter and a receiver. The transmitter is provided by logic sub-block 221 using symbols, and the transmitter serializes the symbols and transmits them to an external device. The receiver is provided with serialized symbols from the external device and converts the received signal into a bit stream. The bit stream is deserialized and provided to logic sub-block 221. In an example embodiment, 8b / 10b transmission code is used, in which 10-bit symbols are transmitted / received. Here, special symbols are used to construct packets using frames 223. In addition, in one example, the receiver also provides a symbol clock recovered from the incoming serial stream.
[0050] As described above, although the transaction layer 205, link layer 210, and physical layer 220 are discussed with reference to a specific embodiment of a protocol stack (such as a PCIe protocol stack), the layered protocol stack is not limited thereto. In fact, any layered protocol may be included / implemented and employ the features discussed herein. As an example, a port / interface represented as a layered protocol may include: (1) a first layer, i.e., a transaction layer, for assembling packets; a second layer, i.e., a link layer, for sorting packets; and a third layer, i.e., a physical layer, for transmitting packets. As a specific example, a high-performance interconnect layered protocol as described herein is utilized.
[0051] See next Figure 4 , illustrates an example embodiment of a serial point-to-point configuration. A serial point-to-point link may include any transmission path for transmitting serial data. In the illustrated embodiment, the link may include two low-voltage differentially driven signal pairs: a transmission pair 406 / 411 and a reception pair 412 / 407. Thus, device 405 includes transmission logic 406 for transmitting data to device 410, and reception logic 407 for receiving data from device 410. In other words, in some implementations of the link, two transmission paths, namely paths 416 and 417, and two reception paths, namely paths 418 and 419, are included.
[0052] A transmission path refers to any path for transmitting data, such as a transmission line, copper wire, optical line, wireless communication channel, infrared communication link, or other communication path. A connection between two devices, such as the connection between device 405 and device 410, is referred to as a link, such as link 415. A link may support one channel - each channel represents a set of differential signal pairs (one pair for transmission and one pair for reception). To scale bandwidth, a link may aggregate multiple channels, represented as xN, where N is any supported link width, such as 1, 2, 4, 8, 12, 16, 20, 24, 32, 64, or wider.
[0053] A differential pair may refer to two transmission paths, such as lines 416 and 417, to transmit differential signals. As an example, when line 416 is toggled from a low voltage level to a high voltage level (i.e., a rising edge), line 417 is driven from a high logic level to a low logic level (i.e., a falling edge). Differential signals potentially exhibit better electrical characteristics, such as better signal integrity (i.e., cross-coupling), voltage overshoot / undershoot, ringing, and other example advantages. This allows for better timing windows, which can enable faster transmission frequencies.
[0054] In one embodiment, a new high performance interconnect (HPI) is provided. The HPI may include a next generation cache-coherent, link-based interconnect. As an example, the HPI may be used in a high performance computing platform, such as a workstation or server, which is included in a system in which PCIe or another interconnect protocol is typically used to connect processors, accelerators, I / O devices, etc. However, the HPI is not limited to this. Alternatively, the HPI may be used in any of the systems or platforms described herein. In addition, the various concepts developed may be applied to other interconnects and platforms, such as PCIe, MIPI, Ethernet, USB, QPI, etc.
[0055] To support multiple devices, in one example implementation, HPI may include instruction set architecture (ISA) agnostic (i.e., HPI can be implemented in multiple different devices). In another scenario, HPI may also be used to connect high-performance I / O devices, not just processors or accelerators. For example, a high-performance PCIe device may be coupled to HPI via a suitable conversion bridge (i.e., HPI to PCIe). In addition, HPI links may be utilized by many HPI-based devices (such as processors) in various ways (e.g., star, ring, mesh, etc.). Figure 5 The figure illustrates an example implementation of multiple potential multi-socket configurations. As depicted, the dual-socket configuration 505 may include two HPI links; however, in other implementations, one HPI link may be utilized. For larger topologies, any configuration may be utilized as long as an identifier (ID) is assignable and there is some form of virtual path, as well as other additional or alternative features. As shown, in one example, a four-socket configuration 510 has an HPI link from each processor to another processor. But in the eight-socket implementation shown in configuration 515, not every slot is directly connected to each other via an HPI link. However, if a virtual path or channel exists between processors, the configuration is supported. The range of supported processors includes 2-32 processors in the native domain. In addition to other examples, a higher number of processors may be achieved by using other interconnections between multiple domains or node controllers.
[0056] The HPI architecture includes the definition of a layered protocol architecture, including in some examples protocol layers (conformant, non-conformant, and optionally other memory-based protocols), routing layers, link layers, and physical layers. In addition, HPI may also include enhancements related to power managers (such as power control units (PCUs)), design for test and debugging (DFT), error handling, registers, security, and other examples. Figure 5 An example HPI layered protocol stack is shown. In some implementations, Figure 5At least some of the layers illustrated in the diagram may be optional. Each layer handles its own level of information granularity or information share (protocol layer 605a,b with packet 630, link layer 610a,b with flits 635, and physical layer 605a,b with physical flits (phit) 640). Note that a packet may, in some embodiments, include a partial flits, a single flits, or multiple flits based on the implementation.
[0057] As a first example, the width of the physical flit 640 comprises a 1-to-1 mapping of link width to bits (e.g., a 20-bit link width comprises a 20-bit physical flit, etc.). Flits may have larger sizes, such as 184, 192, or 200 bits. Note that if the physical flit 640 is 20 bits wide and the flit 635 is 184 bits in size, a fraction of the physical flit 640 is used to transmit one flit 635 (e.g., 9.2 physical flit s at 20 bits are used to transmit a 184-bit flit 635 or 9.6 physical flit s at 20 bits are used to transmit a 192-bit flit, among other examples). Note that the width of the underlying link may vary at the physical layer. For example, the number of lanes in each direction may include 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, etc. In one embodiment, the link layer 610a,b can embed multiple different transactions in a single microchip, and one or more headers (e.g., 1, 2, 3, 4) can be embedded in the microchip. In one example, HPI divides the header into corresponding slots to enable multiple messages in the microchip to different nodes.
[0058] In one embodiment, the physical layer 605a, b may be responsible for quickly transmitting information on a physical medium (electrical or optical, etc.). A physical link may be point-to-point between two link layer entities (such as layers 605a and 605b). The link layer 610a, b may extract the physical layer 605a, b from the upper layer and provide the ability to reliably transmit data (and requests) and manage flow control between two directly connected entities. The link layer may also be responsible for virtualizing the physical channel into multiple virtual channels and message classes. The protocol layer 620a, b relies on the link layer 610a, b to map the protocol messages to the appropriate message classes and virtual channels before processing them to the physical layer 605a, b for transmission across the physical link. The link layer 610a, b may support multiple messages, such as requests, listens, responses, write-backs, non-consistent data, and other examples.
[0059] The physical layer 605a, b (or PHY) of the HPI may be implemented above the electrical layer (ie, the electrical conductors connecting two components) and below the link layer 610a, b, such as Figure 6As shown in the figure. The physical layer and corresponding logic may reside on each agent and connect the link layers on two separate agents (A and B) (for example, on devices on either side of the link). The local electrical layer and the remote electrical layer are connected by a physical medium (for example, wires, conductors, optics, etc.). In one embodiment, the physical layer 605a, b has two main phases, initialization and operation. During initialization, the connection is opaque to the link layer, and signaling may involve a combination of timing states and handshake events. During operation, the connection is transparent to the link layer and signaling is at a certain speed, where all channels operate together as a single link. During the operation phase, the physical layer transmits microchips from agent A to agent B and from agent B to agent A. The connection is also called a link, and some physical aspects are extracted from the link layer, including media, width, and speed, while exchanging microchips and control / states of the current configuration (for example, width) with the link layer. The initialization phase includes secondary phases, such as polling and configuration. The operation phase also includes secondary phases (for example, link power management state).
[0060] In one embodiment, the link layer 610a, b may be implemented to provide reliable data transmission between two protocol or routing entities. The link layer may extract the physical layer 605a, b from the protocol layer 620a, b, and may be responsible for flow control between two protocol agents (A, B), and provide virtual channel services to the protocol layer (message class) and the routing layer (virtual network). The interface between the protocol layer 620a, b and the link layer 610a, b may generally be at the packet level. In one embodiment, the minimum transmission unit at the link layer is called a flit, which is a specified number of bits, such as 192 bits or some other amount. The link layer 610a, b relies on the physical layer 605a, b to construct the 605a, b transmission unit (physical flit) of the physical layer into the 610a, b transmission unit (flit) of the link layer. In addition, the link layer 610a, b may be logically divided into two parts, namely a transmitter and a receiver. A transmitter / receiver pair on one entity may be connected to a receiver / transmitter pair on another entity. Flow control is typically performed on both a flit and packet basis. Error detection and correction is also potentially performed on a flit-level basis.
[0061] In one embodiment, the routing layer 615a, b can provide a flexible and distributed method to route HPI transactions from a source to a destination. The scheme is flexible because the routing algorithms for multiple topologies can be specified by a programmable routing table at each router (in one embodiment, the programming is performed by firmware, software, or a combination thereof). The routing functionality can be distributed; routing can be completed by a series of routing steps, each of which is defined by a lookup of a table at any of the source, intermediate, or destination routers. The lookup at the source can be used to inject the HPI packet into the HPI structure. The lookup at the intermediate router can be used to route the HPI packet from the input port to the output port. The lookup at the destination port can be used to target the destination HPI protocol agent. Note that in some implementations, the routing layer may be thinner because the routing table and therefore the routing algorithm are not specifically defined by the specification. This allows flexibility and a variety of usage models, including a flexible platform architecture topology to be defined by the system implementation. The routing layer 615a,b relies on the link layer 610a,b to provide the use of up to three (or more) virtual networks (VNs) - in one example, two deadlock-free VNs, VN0 and VN1, with several message classes defined in each virtual network. A shared adaptive virtual network (VNA) may be defined in the link layer, but the adaptive network may not be directly exposed in the routing concept because each message class and virtual network may have dedicated resources and guaranteed forward progress, as well as other features and examples.
[0062] In one embodiment, HPI may include a consistency protocol layer 620a, b to support agents that cache data lines from memory. Agents that wish to cache memory data may use a consistency protocol to read data lines to load into their cache. Agents that wish to modify data lines in their cache may use a consistency protocol to obtain ownership of the line before modifying the data. After modifying the line, the agent may follow the following protocol requirements: keep it in its cache until the line is written back to memory or the line is included in a response to an external request. Finally, the agent may satisfy an external request to invalidate the line in its cache. The protocol ensures data consistency by specifying rules that all caching agents may follow. It also provides a means for agents without cache to read and write memory data consistently.
[0063] Two conditions may be enforced to support transactions utilizing the HPI consistency protocol. First, the protocol may maintain data consistency (e.g., on a per-address basis) among data in the proxy cache and between that data and data in memory. Informally, data consistency may refer to each valid data line in the proxy cache representing the latest value of the data, and the data transmitted in the consistency protocol packet may represent the latest value of the data at the time the data was sent. When no valid copy of the data exists in the cache or in transit, the protocol may ensure that the latest value of the data resides in memory. Second, the protocol may provide well-defined commit points for requests. Commit points for reads may indicate when data is available; and for writes, they may indicate when the data being written is globally visible and when it will be loaded by subsequent reads. The protocol may support these commit points for cacheable and uncacheable (UC) requests in a consistent memory space.
[0064] In some implementations, HPI may utilize embedded clocks. A clock signal may be embedded in the data being transmitted using the interconnect. With a clock signal embedded in the data, different and dedicated clock channels may be omitted. This may be useful, for example, because it may allow more device pins to be dedicated to data transmission, particularly in systems where space for pins is at a premium.
[0065] A link may be established between two agents on either side of the interconnect. The agent that sends data may be a local agent and the agent that receives data may be a remote agent. A state machine may be used by both agents to manage various aspects of the link. In one embodiment, a physical layer data path may transfer microchips from the link layer to the electrical front end. In one implementation, the control path includes a state machine (also referred to as a link training state machine or the like). The actions of the state machine and the exit from the state may depend on internal signals, timers, external signals, or other information. In fact, some states, such as a small number of initialization states, may have a timer to provide a timeout value to exit the state. Note that in some embodiments, detection refers to detecting events on both branches of a channel; but not necessarily at the same time. However, in other embodiments, detection refers to detecting an event by a reference agent. As an example, debounce refers to the continuous assertion of a signal. In one embodiment, HPI supports operations in events of non-functional channels. Here, a channel may fall at a specific state.
[0066] The states defined in the state machine may include reset states, initialization states, and operational states, as well as other classifications and subclassifications. In one example, some initialization states may have a secondary timer that is used to exit the state upon timeout (essentially abort due to failure to make progress in the state). Aborts may include updates to registers such as state registers. Some states may also have (one or more) primary timers that are used to time the primary functions in the state. Other states may be defined so that an internal or external signal (such as a handshake protocol) drives the transition from the state to another state, as well as other examples.
[0067] The state machine may also support debugging through individual steps, freezing of initialization aborts, and use of testers. Here, state exits may be delayed / held until debug software is ready. In some instances, exits may be delayed / held until auxiliary timeouts. In one embodiment, actions and exits may be based on the exchange of training sequences. In one embodiment, the link state machine will run in the local agent clock domain and transition from one state to the next to coincide with transmitter training sequence boundaries. A state register may be used to reflect the current state.
[0068] Figure 7 The figure shows a representation of at least a portion of a state machine used by an agent in one example implementation of HPI. It should be appreciated that the state machine included in Figure 7 The states in the state table include a non-exhaustive list of possible states. For example, some transitions are omitted to simplify the illustration. Also, some states may be combined, divided, or omitted, while other states may be added. Such states may include:
[0069] Event reset status : Entered on a warm reset or cold reset event. Restores default values. Initializes counters (e.g., synchronizes counters). May exit to another state, such as another reset state.
[0070] Timed reset status : A timed state for in-band reset. Can drive a predefined electrical ordered set (EOS) so a remote receiver can detect the EOS and also go into a timed reset. Receiver has a channel to hold the electrical settings. Can exit to an agent to calibrate the reset state.
[0071] Calibration reset status : Calibration without signaling on the channel (e.g., receiver calibration state) or with the driver turned off. Can be a predetermined amount of time in a timer-based state. Can set the speed of operation. Can act as a wait state when the port is not enabled. Can include a minimum dwell time. Receiver adjustments or stagger off can be done based on design. Can exit to the receiver detection state after timeout and / or completion of calibration.
[0072] Receiver detection status : Detects the presence of a receiver on channel(s). May look for receiver termination (e.g., receiver pull-down insertion). May exit to the Calibration Reset state when a specified value is set or when another specified value is not set. May exit to the Transmitter Calibration state if a receiver is detected or a timeout is reached.
[0073] Transmitter calibration status : Used for transmitter calibration. May be a timed state assigned for transmitter calibration. May include signaling on the channel. May drive EOS continuously, such as an Electrical Idle Exit Ordered Set (EIEIOS). May exit to the Compliant state when calibration is complete or a timer expires. May exit to the Transmitter Detect state if a counter has expired or an Auxiliary Timeout has occurred.
[0074] Transmitter detection status : Define valid signaling. It can be a handshake state, where the agent completes the action based on the remote agent signaling and exits to the next state. The receiver can define valid signaling from the transmitter. In one embodiment, the receiver looks for wake-up detection, and if debounced on one or more channels, it looks on other channels. The transmitter drives the detection signal. In response to debounce completed for all channels and / or timeout or if debounce on all channels is not completed and there is a timeout, it can exit to the polling state. Here, one or more monitoring channels can remain awake to debounce the wake-up signal. And if debounced, potentially debounce other channels. This can enable power saving in low power states.
[0075] Polling Status: The receiver adapts to lock bits (e.g., by initializing a drift buffer), lock bytes (e.g., by identifying symbol boundaries), and lock training sequence boundaries (e.g., by locking to an Electrically Ordered Set (EOS) and / or a training sequence header). The channel can then be de-skewed. A handshake can also be completed to exit polling to one of several potential states. For example, an exit to any of a link width state, a compliant state, a loopback flag state, or a reset state can be caused (by a proxy). The handshake can include the completion of various defined receiver actions or conditions and the sending of an acknowledgement message (ACK) to trigger an exit to a state corresponding to a set of completed actions and the ACK. For polling de-skew, the channel-to-channel skew at the remote transmitter can be covered with a first length for the highest speed and a second length for the slow speed. De-skew can be performed in a slow mode as well as an operating mode. The receiver can have a specific maximum value for de-skewing the channel-to-channel skew, such as a skew interval of 8, 16, or 32. In some implementations, the receiver action can also include latency repair. In one embodiment, receiver actions may be completed upon successful deskew of valid channel mapping.In one example, a successful handshake may be achieved after the receiver has completed its actions upon receiving multiple consecutive training sequence headers with an acknowledgement and multiple training sequences with the acknowledgement are transmitted.
[0076] Submissive state: Entered from the polling state. The agent can act as a compliant master or slave for verification purposes. A device (acting as a master) can send the compliance mode to another device (slave), and the slave can loop back the mode after retiming it to its local clock (for example, without undoing any polarity reversal or channel inversion). The compliance mode can be used to characterize the operation of the analog front end on some subset of the channels when loopback does not work or is unsatisfactory. For example, entry into loopback can be pre-conditioned on successful byte lock, TS lock, de-skew, latency tests, and other tests that rely on the correct operation of several digital logics. Compliance can be entered in the case of completed bit lock, and compliance can be used for jitter or noise investigation, debugging, exploring links, and other adjustments. In other words, if the results of polling do not allow direct entry into loopback, compliance can be used as an alternative exit to polling. The compliance state can utilize the transmitter from the master to drive the super sequence. The receiver looks for wakeup on the monitor channel, debounces the wakeup, discards bad channels, adapts, and bit locks, etc. The slave transmitter can drive the compliance mode until its receiver action is completed. The loopback is then retimed and undeskewed. Similar monitoring and debounce, etc., are performed by the slave receiver. Exit can be to a reset state (such as a timer reset) or to a loopback mode state to start testing, among other examples. In the case of an exit to a loopback mode state, the master device (e.g., by a software controller) can be sent to the loopback mode state to try a more specialized mode.
[0077] Proxy loopback flag status : The loopback flag is a proxy state, but unlike the other proxy states, the actions and exits can be different for the master and slave. A loopback slave may undo any polarity reversal and / or lane reversal, but may not descramble or re-scramble the bits that were looped back. The acknowledge exchange is not applicable to the slave because it is looping back. Since the slave may deskew before looping back onto a symbol boundary, the master may not be forced to re-byte lock or re-deskew, but the master may re-lock to the training sequence to avoid locking to some aliases. The means for doing this may include re-seeding of the LFSR, comparing TS and / or EIEOS, or some combination of these. The end of the SDS marks the end of the loopback setup and the beginning of pattern generation, checking, and counting.
[0078] Proxy loopback mode status (or blocked link status): Entered from the Loopback Flag state or the Compliance state. In the Loopback Mode state, in lieu of the Control pattern, the master transmitter may send additional dedicated patterns to supplement those loopbacks in the Compliance or Loopback Mode states. The receiver may receive the dedicated pattern in the Loopback mode and check for errors in the received pattern. For transmitter adaptation, both agents may be masters. For a predetermined period of time, the transmitter may send a pattern and the remote receiver may compare this pattern and determine a metric or value of the received pattern recorded in a storage element such as a register. The comparison method and metric may depend on the design (e.g., BER with jitter injection). At the end of this period of time, both agents may exit to reset in order to return to the channel to check the metric and set up the next iteration of transmitter adaptation.
[0079] Link Width Status : The agent communicates the final channel mapping to the remote transmitter. The receiver receives the information and decodes it. The receiver may record the configured channel mapping in the structure after a checkpoint of the previous channel mapping value in the second structure. The receiver may also respond with an acknowledgement ("ACK"). An in-band reset may be initiated. As an example, a first state initiates an in-band reset. In one embodiment, an exit to a next state, such as a microchip configuration state, is performed in response to the ACK. In addition, before entering a low-power state, a reset signal may also be generated if the frequency of occurrence of the wake-up detection signal drops below a specified value (e.g., 1 per unit interval (UI) number, such as 4K UI). The receiver may maintain the current and previous channel mappings. Based on training sequences with different values, the transmitter may use different channel groups. In some embodiments, the channel mapping may not modify some status registers.
[0080] Microchip lock configuration status : When both the transmitter and receiver have exited to the blocked link state or other link state, the state is entered by the transmitter but is considered to be exited (i.e., the auxiliary timeout assumption case). In one embodiment, the transmitter exit to the link state includes a training sequence (TS) boundary and the start of a data sequence (SDS) after receiving an alignment signal with a track. Here, the receiver exit can be based on receiving an SDS from a remote transmitter. This state can be a bridge from an agent to a link state. The receiver recognizes the SDS. If the SDS is received after the descrambler is initialized, the receiver can exit to the blocked link state (BLS) (or control window). If a timeout occurs, the exit can be to a reset state. The transmitter drives the channel using a configuration signal. Based on a condition or timeout, the transmitter exit can be to a reset, BLS, or other state.
[0081] Transmission link status: Link state. Flicks are sent to remote agent. Can be entered from blocking link state and returned to blocking link state on events such as timeout. Transmitter transmits flits. Receiver receives flits. Can also exit to low power link state. In some implementations, the transport link state (TLS) can be referred to as L0 state.
[0082] Block link status : Link state. The transmitter and receiver operate in a unified manner. Can be a timed state during which the link layer microchips are held up while physical layer information is transmitted to the remote agent. Can exit to a low power link state (or other link state based on design). In one embodiment, the blocked link state (BLS) occurs periodically. The period is called the BLS interval and can be timed, and can vary between slow speed and operating speed. Note that the link layer can be periodically blocked from sending microchips, such as during a transmit link state or a partial width transmit link state. Physical layer control sequences of a certain length can be sent. In some implementations, the blocked link state (BLS) may be referred to as the L0 control (or L0c) state.
[0083] Partial Width Transmission Link Status : Link state. Power can be saved by entering the partial width state. In one embodiment, asymmetric partial width refers to each direction of the bidirectional link having a different width, which may be supported in some designs. Fig. 9 An example of an initiator, such as a transmitter, is shown in the example of FIG. 1 , which sends a partial width indication to enter a partial width transmission link state. Here, a partial width indication is sent while transmitting on the link using a first width to transform the link to transmit at a new second width. A mismatch may cause a reset. Note that the speed may not be changed but the width may be changed. Therefore, the microchip is potentially sent at different widths. It may be similar to transmitting the link state in a logical manner; but due to the presence of a smaller width, it may take a longer time to transmit the microchip. It may exit to other link states, such as a low power link state based on specific receive and transmit messages or an exit from a partial width transmission link state or a link blocking state based on other events. In one embodiment, as shown in the timing diagram, the transmitter port may turn off idle channels in a staggered manner to provide better signal integrity (i.e., noise mitigation). Here, non-retryable microchips, such as empty microchips, may be utilized during the period when the link width is changing. The corresponding receiver may discard these empty microchips and turn off idle channels in a staggered manner, and record the current and previous channel mappings in one or more structures. Note that the state and associated state registers may remain unchanged. In some implementations, the partial width transmission link state can be referred to as a partial L0 (or L0p) state.
[0084] Exit partial width link state: Exit the partial width state. In some implementations, the blocked link state may or may not be used. In one embodiment, the transmitter initiates the exit by sending a partial width exit pattern on an idle channel to train and deskew it. As an example, the exit mode begins with EIEOS, which is detected and debounced to signal that the channel is ready to begin entering the full transmission link state, and may end with SDS or a fast training sequence (FTS) on an idle channel. Any failure during the exit sequence (receiver action, such as deskew not completed before a timeout) stops the microchip transmission to the link layer and asserts a reset, which is handled by resetting the link when the next blocked link state occurs. SDS may also initialize the scrambler / descrambler on the channel to appropriate values.
[0085] Low power link state : is a lower power state. In one embodiment, it is lower power than the partial width link state because signaling in this embodiment is stopped in all channels and in both directions. The transmitter can use the blocking link state to request the low power link state. Here, the receiver can decode the request and respond with an ACK or NAK; otherwise a reset can be triggered. In some implementations, the low power link state can be called the L1 state.
[0086] In some implementations, state transitions may be facilitated to allow states to be bypassed, for example, when state actions for a state (such as certain calibrations and configurations) have been completed. Previous state results and configurations of a link may be stored and reused in subsequent initialization and configuration of the link. Instead of repeating such configurations and state actions, the corresponding state may be bypassed. However, conventional systems that implement state bypasses often implement complex designs and expensive verification escapes. In contrast to using conventional bypasses, in one example, HPI may utilize short timers in certain states, such as when state actions do not need to be repeated. This may potentially allow more consistent and synchronized state machine transitions, as well as other potential advantages.
[0087] In one example, a software-based controller (e.g., through an external control point for the physical layer) can enable a short timer for one or more specific states. For example, for a state where an action has been executed and stored, the state may be short-timed to facilitate a quick exit from the state to the next state. However, if the previous state action fails or cannot be applied within the short timer duration, the state exit may be executed. In addition, the controller may disable the short timer, for example, when the state action should be re-executed. A long or default timer may be set for each corresponding state. If the configuration action in the state cannot be completed within the long timer, the state exit may occur. The long timer may be set to a reasonable duration to allow completion of the state action. On the contrary, the short timer may be much shorter, making it impossible to execute the state action without referring back to the previously executed state action in some cases, and other examples.
[0088] In some instances, during the initialization (or reinitialization) of the link, when the agent advances toward the operational link state through the state machine, one or more faults or state exits that cause the state to be reset (e.g., reset to a reset state or other state) may occur. In fact, the initialization of the link can cycle through one or more states without completing the initialization and entering the link state. In one example, a count can be maintained in the number of non-productive cycles in the state transition within the initialization of the link. For example, whenever the initialization returns to the reset state without reaching the link state, the counter can be incremented. Once the link successfully enters the link state, the counter for the link can be reset. Such a counter can be maintained by agents on both sides of the link. In addition, a threshold can be set, for example, by a software-based controller utilizing one or more external control points. When the count of non-productive cycles meets (or exceeds) a defined threshold, the initialization of the link can be suspended (e.g., set and maintained at or before the reset state). In some implementations, in order to restart the initialization and release the initialization from the paused state, a software-based controller can trigger the restart or reinitialization of the link. In some instances, software-based tools can analyze the nature of the suspended initialization and perform diagnostics, set register values, and perform other operations to prevent further cycles of initialization. Indeed, in some implementations, the controller can set a higher counter threshold or even overrule the counter in association with restarting a suspended link initialization, among other examples.
[0089] In some implementations of HPI, a supersequence may be defined, each supersequence corresponding to a corresponding state or entering / entering / exiting a corresponding state / exiting a corresponding state from a corresponding state. A supersequence may include a repetitive sequence of data sets and symbols. In some instances, the sequence may be repeated until the completion of a state or state transition or the transmission of a corresponding event, as well as other examples. In some instances, the repetitive sequence of a supersequence may be repeated according to a defined frequency (such as the number of defined unit intervals (UI)). The unit interval (UI) may correspond to a time interval for transmitting a single bit on a channel of a link or system. In some implementations, a repetitive sequence may begin with an electrical ordered set (EOS). Therefore, an instance of EOS may be expected to repeat according to a predefined frequency. Such an ordered set may be implemented as a defined 16-byte code, which may be represented in hexadecimal format, as well as other examples. In one example, the EOS of a supersequence may be EIEIOS. In one example, EIEOS may be similar to a low-frequency clock signal (e.g., a predefined number of repeated FF00 or FFF000 hexadecimal symbols, etc.). A predefined set of data may follow the EOS, such as a predefined number of training sequences or other data. Such a supersequence may be used in state transitions, including link state transitions and initialization, among other examples.
[0090] In some implementations of the interconnect (such as in QPI), termination of a serial data link may be facilitated and implemented, such as when the link is reset or initialized. This approach may introduce complexity and time into the initialization of the link. In some implementations of HPI, the termination of the link may be maintained, including during the reset or reinitialization of the link. In addition, HPI may allow hot plugging of devices. When another device is introduced (by hot plugging or otherwise), the voltage characteristics of the channel to which the new remote agent is added will change. The local agent may sense these changes in the channel voltage to detect the presence of the remote agent and prompt the initialization of the link. State machine states and timers may be defined in the state machine to coordinate the detection, configuration, and initialization of the link without termination.
[0091] In one implementation, HPI can support reinitialization of in-band reset without changing the termination value by screening the channel for incoming signaling by the receiving agent. Signaling can be used to identify good channels. As an example, the channel can be screened for any one of a set of predefined signals to be sent by the transmitter device to facilitate the discovery and configuration of the link. In one example, a supersequence corresponding to one or more initialization or reinitialization tasks can be defined. The predefined sequence can include an electrical idle exit ordered set (EIEOS) followed by additional sequence data. In some instances, when each device on either side of the channel becomes active, the device can begin sending a supersequence corresponding to a specific initialization state, etc. In one embodiment, two types of pin resets can be supported; power-on (or "cold") resets and warm resets. A reset initiated by software or a reset initiated on one agent (in the physical layer or another layer) can be transmitted in-band to another agent. However, due to the use of an embedded clock, an in-band reset can be handled by communication with another agent using an ordered set such as a specific electrical ordered set or EIOS.
[0092] An ordered set may be sent during initialization, and a PHY control sequence (or "block link state") may be sent after initialization. The block link state may block the link layer from sending flits. As another example, link layer traffic may be blocked to send a few NULL flits that may be discarded at the receiver.
[0093] As described above, in one embodiment, initialization may be initially completed at a slow speed, followed by a fast initialization. Initialization at a slow speed uses default values for registers and timers. The software then uses the slow link to set up registers, timers, and electrical parameters, and clears the calibration signal (semaphore) to pave the way for fast initialization. As an example, initialization may include states or tasks such as reset, detect, poll, comply, and configure, as well as potential other states or tasks.
[0094] In one example, a link layer blocking control sequence (i.e., blocking link state (BLS) or L0c state) may include a timed state during which link layer flits are stalled while PHY information is transmitted to a remote agent. Here, the transmitter and receiver may start a blocking control sequence timer. And upon expiration of the timer, the transmitter and receiver may exit the blocking state and may take other actions, such as exiting to reset, exiting to a different link state (or other state), including a state that allows flits to be sent across the link.
[0095] In one embodiment, link training may be provided and include, for example, sending one or more of a scrambled training sequence, an ordered set, and a control sequence in relation to a defined supersequence. The training sequence symbols may include one or more of: a header, a reserved portion, a target latency, a logarithm, a physical channel mapping code reference channel or channel group, and an initialization state. In one embodiment, a header with an ACK or NAK may be sent, among other examples. As an example, the training sequence may be sent as part of a supersequence and may be scrambled.
[0096] In one embodiment, ordered sets and control sequences are not scrambled or interleaved and are transmitted equally, simultaneously and completely on all channels. Valid reception of ordered sets may include checking at least a portion of the ordered set (or the entire ordered set for a partial ordered set). Ordered sets may include electrical ordered sets (EOS), such as electrical idle ordered sets (EIOS) or EIEOS. Super sequences may include the beginning of a data sequence (SDS) or a fast training sequence (FTS). Such sets and control super sequences may be predefined and may have any pattern or hexadecimal representation and any length. For example, ordered sets and super sequences may be 8 bytes, 16 bytes, or 32 bytes in length, etc. As an example, FTS may be additionally used for fast bit locking during the exit of a partial width transmission link state. Note that FTS definitions may be per channel and may utilize a rotating version of FTS.
[0097] In one embodiment, the supersequence may include the insertion of an EOS (such as EIEOS) in the training sequence stream. In one implementation, when signaling begins, the channels are powered up in an interleaved form. However, this may cause the initial supersequence to appear truncated at the receiver on some channels. However, the supersequence may be repeated at short intervals, such as approximately one thousand unit intervals (or ~1 KUI). The training supersequence may additionally be used for one or more of: de-skew, configuration, and for transmitting initialization targets, channel mapping, etc. EIEOS may be used for one or more of: channel transitions from inactive to active states, screening good channels, identifying symbol and TS boundaries, and other examples.
[0098] In one embodiment, the clock may be embedded in the data, so there is no separate clock channel. The microchips sent through the channel may be scrambled to facilitate clock recovery. As an example, the receiver clock recovery unit may deliver the sampling clock to the receiver (i.e., the receiver recovers the clock from the data and uses it to sample the incoming data). The receiver continuously adapts to the incoming bit stream in some implementations. By embedding the clock, pinouts can be potentially reduced. However, embedding the clock in the in-band data can change the way in-band resets are treated. In one embodiment, the blocking link state (BLS) can be utilized after initialization. And, the electrical ordered set super sequence can be used during initialization to facilitate resets (e.g., as described above) and other considerations. The embedded clock can be common between devices on the link, and the common operating clock can be set during calibration and configuration of the link. For example, the HPI link can reference a common clock with a drift buffer. This implementation can achieve lower latency and other potential advantages compared to the elastic buffer used in the non-common reference clock. In addition, the reference clock distribution segment can be matched to within specified limits.
[0099] In some implementations, HPI may support flits having widths that are not multiples of the nominal lane width in some cases (e.g., using a flits width of 192 bits and 20 lanes as a purely illustrative example). In fact, in implementations that allow partial-width transmission states, the number of lanes over which flits are transmitted may fluctuate (even during the life of the link). For example, in some instances, the flits width may be a multiple of the number of active lanes at one instant, but may not be a multiple of the number of active lanes at another instant (e.g., as the link changes state and lane widths). In instances where the number of lanes is not a multiple of the current lane width (e.g., the example of a flits width of 192 bits on 20 lanes), in some embodiments, consecutive flits may be configured to be transmitted to overlap on the lanes, thereby conserving bandwidth (e.g., transmitting five consecutive 192-bit flits overlapping on the 20 lanes).
[0100] Figure 8 The figure shows a representation of the transmission of consecutive microsheets overlapping on multiple channels. For example, Figure 8 A representation of five overlapping 192-bit flits sent over a 20-lane link (lanes represented by columns 0-19) is shown. Figure 8Each cell of represents a corresponding "half byte" or grouping of four bits (e.g., bits 4n+3:4n) included in the microchip sent over the 4UI span. For example, a 192-bit microchip may be divided into 48 four-bit half bytes. In one example, half byte 0 includes bits 0-3, half byte 1 includes bits 4-7, and so on. The bits in a half byte may be sent so as to overlap or be interleaved (e.g., "swizzled") so that higher priority fields of the microchip are presented earlier, error detection properties (e.g., CRC) are maintained, and other considerations. In practice, the swizzling scheme may also provide that some half bytes (and their corresponding bits) are sent out of order (e.g., as Figure 8 and 9 In some implementations, the mixing scheme may depend on the architecture of the link layer and the format of the microslices used in the link layer.
[0101] Bits (or nibbles) of microchips with lengths that are not multiples of active channels may be mixed, such as according to Figure 8 . For example, during the first 4UI, nibbles 1, 3, 5, 7, 9, 12, 14, 17, 19, 22, 24, 27, 29, 32, 34, 37, 39, 42, 44, and 47 may be sent. Nibbles 0, 2, 4, 6, 8, 11, 13, 16, 18, 21, 23, 26, 28, 31, 33, 36, 38, 41, 43, and 46 may be sent during the next 4UI. In UI 8-11, only 8 nibbles are retained for the first microchip. These last nibbles of the first microchip (i.e., 10, 15, 20, 25, 30, 40, 45) may be sent simultaneously with the first nibbles of the second microchip (i.e., nibbles 2, 4, 7, 9, 12, 16, 20, 25, 30, 35, 40, 45) so that the first and second microchips overlap or intermingle. Using this technique, in this example, five complete flits may be sent in 48 UI, with each flit being sent over a fractional 9.6 UI period.
[0102] In some instances, mixing can produce periodic "clean" microplate boundaries. Figure 8 In the example of , the first 5 microchip boundaries (top row of the first microchip) can also be referred to as clean microchip boundaries because all lanes transmit the starting nibble from the same microchip. The proxy link layer logic can be configured to recognize the swizzling of the lanes and can reconstruct the microchips from the swizzled bits. In addition, the physical layer logic can include functionality for recognizing when and how to swizzle the microchip data stream based on the number of lanes in use at the time. In fact, in the transition from one link width state to another, the proxy can configure itself to recognize how the swizzling of the data stream will be employed.
[0103] As described above, a link may transition between lane widths, in some instances operating at an original or full width and later transitioning to (and from) a partial width utilizing fewer lanes. In some instances, the defined width of a microchip may be divisible by the number of lanes. For example, Fig. 9 The example diagram of FIG. 1 shows an example in which the 192-bit microchip of the previous example is transmitted over an 8-lane link. Fig. 9 As shown, the 4-bit nibbles of a 192-bit flit can be evenly distributed and transmitted over 8 lanes (i.e., since 192 is a multiple of 8). In fact, when operating at 8 lane partial widths, a single flit can be sent over 24 UIs. Fig. 9 In the example of , each microchip boundary can be clean. Although clean microchip boundaries can simplify state transitions, determinism, and other characteristics, allowing for shuffling and occasional jagged microchip boundaries can allow for minimization of wasted bandwidth on the link.
[0104] In addition, although Fig. 9 The example shows channels 0-7 as channels that remain active in a partial width state, but any set of 8 channels can potentially be used. Also note that the above examples are for illustrative purposes only. Microchips can potentially be defined to have any width. Links can also potentially have any link width. In addition, the mixing scheme of the system can be flexibly constructed based on the format and fields of microchips, the preferred channel widths in the system (and other considerations and examples).
[0105] The HPI PHY logic layer can operate independent of the underlying transmission medium without latency creating latency repair errors or timeouts at the link layer and other considerations.
[0106] Link Layer
[0107] The link layer can abstract the physical layer from the protocol layer, handle flow control between two protocol agents, and provide virtual channel services to the protocol layer (message class) and the routing layer (virtual network). In some implementations, the link layer can handle a fixed share of information called a microchip. In one example, a microchip can be defined as a 192-bit length. However, any range of bits such as 81-256 (or more) can be used in different variants. Large microchip sizes such as 192 bits can include format, cyclic redundancy check (CRC), error correction code (ECC), and other features. For example, a larger microchip length can also allow the CRC field to be extended (e.g., to 16 bits) to handle larger microchip payloads. The number of physical microchips or unit intervals (UIs) used to transmit a single microchip (e.g., the time used to transmit a single bit or physical microchip, etc.) can vary with the link width. For example, in addition to other potential examples, a 20-channel or bit link width can potentially transmit a single 192-bit microchip in 9.6 UIs, while an 8-channel link width transmits the same microchip in 24 UIs. Link layer credit and protocol grouping can also be performed based on microchips.
[0108] Fig.10 The figure shows a representation 1000 of a generalized microslice for an 8-lane link width. Each column of the representation 1000 can symbolize a link lane, and each row can symbolize a corresponding UI. In some implementations, a single microslice can be subdivided into two or more slots. Each slot can contain a different message or link layer header to allow multiple different (and in some cases independent) messages corresponding to potentially different transactions to be sent in a single microslice. In addition, among other examples, multiple messages included in a slot of a single microslice can also go to different destination nodes. For example, Fig.10 The example diagram of FIG. 1 shows a microsheet format having three slots. The shaded portion may represent the portion of the microsheet included in the corresponding slot.
[0109] exist Fig.10In the example of , three slots are provided, slots 0, 1 and 2. Slot 0 may be provided with 72 bits of microchip space, of which 22 bits are dedicated to message header fields and 50 bits are used for message payload space. Slot 1 may be provided with 70 bits of microchip space, of which 20 bits are dedicated to message header fields and 50 bits are used for message payload space. The difference between the message header field space may be optimized to provide that certain message types will be designated as included in slot 0 (for example, where more message header encodings are used). A third slot (slot 2) may be provided that occupies much less space than slots 0 and 1, in which case 18 bits of microchip space are used. Slot 2 may be optimized to handle messages that do not employ larger message payloads, such as acknowledgments, credit returns, etc. In addition, a floating payload field may be provided that allows an additional 11 bits to be applied alternatively to supplement the payload field of slot 0 or slot 1.
[0110] continue Fig.10 For example, other fields may be global to the flits, (i.e., apply across flits rather than to a specific slot). For example, header bits may be provided along with a 4-bit flits control field that may be used to specify information such as a virtual network for the flits, identify how the flits are to be encoded, and other examples. Additionally, error control functionality may be provided, such as through a 16-bit cyclic CRC field, as well as other potential examples.
[0111] The microchip format can be defined to optimize the throughput of messages on the link layer. Some traditional protocols have used smaller microchips without slots. For example, in QPI, 80-bit microchips are used. Although the throughput of larger (e.g., 192-bit microchips) may be lower, the message or packet throughput can be increased by optimizing the use of microchip data. For example, in some implementations of QPI, the entire 80-bit microchip space is used regardless of the message size or type. By subdividing larger microchips into slots of predetermined lengths and fields, the 192 microchip length can be optimized even when one or more of the available slots are not used at times, thereby achieving higher efficiency. In fact, it can be assumed that the link layer traffic includes many different types of messages and traffic, including messages and packets with varying header lengths and fields. The corresponding lengths and organization of the slots defined in the microchips can be defined to correspond to the statistical or expected frequencies of various messages and the needs of these messages. For example, two larger slots can be defined for each small slot to accommodate the expected statistical frequency of message delivery using these larger message types and header lengths, as well as other examples. In addition, flexibility can be provided, such as by floating payload fields, to further accommodate changing traffic volumes, such as Fig.10 In some instances, the microchip format may be fixed (including bits dedicated to specific slots in a microchip).
[0112] exist Fig.10 In the example of , an "Hdr" field may generally be provided for a microchip, and this field indicates a header indication of the microchip. In some instances, the Hdr field may indicate whether the microchip is a header microchip or a data microchip. In a data microchip, the microchip may still remain slotted, but the use of certain fields may be omitted or replaced with payload data. In some cases, the data field may include an opcode and payload data. In the case of a header microchip, various header fields may be provided. Fig.10 In the example of , an "Oc" field may be provided for each slot, the Oc field representing the operation code. Similarly, one or more slots may have a corresponding "msg" field, which represents the message type of the corresponding packet to be included in the slot (if the slot is designed to handle such a packet type, etc.). The "DNID" field may represent the destination node ID, the "TID" field may represent the transaction or tracker ID, the "RHTID" field may represent the requester node ID or the home tracker ID, and other potential fields. In addition, one or more slots may be provided with a payload field. In addition, in addition to other examples, a CRC field may be included in the microchip to provide a CRC value for the microchip.
[0113] In some implementations, the link width may vary during the life of the link. For example, the physical layer may transition between link width states, such as from or to full or original channel width and different or partial channel width. For example, in some implementations, a link may be initialized to transmit data on 20 channels. Thereafter, the link may transition to a partial width transmission state where only 8 channels are actively used, as well as many other potential examples. Such channel width transitions may be utilized, for example, in conjunction with power management tasks managed by one or more power control units (PCUs), as well as other examples.
[0114] As mentioned above, link width can affect the flit throughput. Fig.11 is a representation of an example 192-bit flits sent over an 8-lane link, yielding a flits throughput of 24 UI. Fig.11 As shown in the example of , in some instances, the bits of a flits may be sent out of order, such as to send more time-sensitive fields (e.g., flits type fields (e.g., data or header flits), opcodes, etc.) earlier in the transmission, to preserve or facilitate specific error detection or other functionality embodied in the flits, and other examples. For example, in Fig.11In the example of FIG. 1 , bits 191, 167, 143, 119, 95, 71, 47, and 23 are sent in parallel on lanes L7 to L0 during the first UI (i.e., UI0) of the transfer, while bits 168, 144, 120, 96, 72, 48, 24, and 0 are sent during the 24th (or last) UI (i.e., UI23) of the flits transfer. It should be appreciated that other ordering schemes, flits lengths, lane widths, etc. may be used in other implementations and examples.
[0115] In some examples, the length of a microsheet may be a multiple of the number of active channels. In such examples, the microsheet may be transmitted evenly across all active channels, and the microsheets may be delivered substantially simultaneously with clean (i.e., non-overlapping) boundaries. For example, Figure 8 As shown in the representation of , the bits of a flit can be thought of as being transmitted in contiguous groups of 4 bits, or "nibbles." In this example, a 192-bit flit would be transmitted over an 8-lane link. Since 192 is a multiple of 8, the entire flit can be cleanly transmitted over the 8-lane link in 24 UIs. In other instances, the flit width may not be a multiple of the number of active lanes. For example, Fig. 9 Another representation of an example 192 bits transmitted over 20 lanes is shown. Since 192 is not evenly divisible by 20, the transmission of a complete microchip will require a non-integer number of intervals (e.g., 9.6 UIs). In this case, instead of wasting the "extra" lanes that are not used during the 10th UI transmitted, the second overlapping microchip can be transmitted with the last bits of the previous microchip. In some implementations, this overlapping or shuffling of microchips may result in jagged microchip boundaries and out-of-order transmission of microchip bits. The mode for transmission may be configured to allow more time-sensitive fields in the microchip to be transmitted earlier in the microchip, retention of error detection and correction, and other considerations. Logic may be provided in one or both of the physical layer and the link layer to transmit microchip bits according to such a mode, and dynamically change between modes based on the current link width. Other logic may be provided to reorder and reconstruct microchips from such a shuffled or ordered bit stream, as well as other examples.
[0116] In some implementations, a flit may be characterized as a header flit (e.g., carrying packet header data) or a data flit (e.g., carrying packet payload data). Fig.10 , a flit format may be defined that includes three (3) distinct slots (e.g., 0, 1, and 2), allowing up to three headers to be transmitted in a single flit (e.g., one header in each slot). Fig.10In the example of , three slots are provided, slots 0, 1 and 2. Slot 0 may be provided with 72 bits of microchip space, of which 22 bits are dedicated to message header fields and 50 bits are used for message payload space. Slot 1 may be provided with 70 bits of microchip space, of which 20 bits are dedicated to message header fields and 50 bits are used for message payload space. The difference between the message header field space may be optimized to provide that certain message types will be designated as included in slot 0 (for example, where more message header encodings are used). A third slot (slot 2) may be provided that occupies much less space than slots 0 and 1, in which case 18 bits of microchip space are used. Slot 2 may be optimized to handle messages that do not employ larger message payloads, such as acknowledgments, credit returns, etc. In addition, a floating payload field may be provided that allows an additional 11 bits to be applied alternatively to supplement the payload field of slot 0 or slot 1.
[0117] In some implementations, by allowing the field to float between two slots, additional bits can be provided as needed for certain messages while still staying within the predefined microchip length (e.g., 192 bits) and maximizing bandwidth utilization. Fig.19 In the example of , two instances 1905, 1910 of an example 192-bit microchip are shown on an 8-lane data link. In one instance, a microchip (e.g., 1905) may include three slots, slots 0, 1, and 2. Each of slots 0 and 1 may include a 50-bit payload field. A floating field may be provided to alternatively extend the payload field of either slot 0 or slot 1 by the field length of the floating field (e.g., 11 bits). The use of floating fields may also extend the efficiency gains provided by the defined multi-slot microchip format. The sizing of the slots within a microchip and the types of messages that can be placed in each slot can potentially provide increased bandwidth (even at reduced microchip ratios).
[0118] exist Fig.10 In the specific example of , messages that can use slots 1 and 2 can be optimized to reduce the number of bits set aside to encode the opcodes for those slots. When a header with more bits than slot 0 can provide enters the link layer, a slot splitting algorithm can be provided to allow it to take over the slot 1 payload bits to achieve the additional space. Special control (e.g., LLCTRL) microchips can also be provided that consume all three slots for the bit values they need. In the case where the link is partially busy, there can also be a slot splitting algorithm to allow individual slots to be used while other slots are not carrying information.
[0119] In one embodiment, the link layer may additionally define special control microchips that can be used, for example, for debug messages and other purposes. In one example, when the Enable Debug Control field is set, an LLCTRL-DEBUG microchip may be sent by the link layer. When this bit is not set, the LLCTRL-DEBUG microchip may not be transmitted on the link. Debug packets can be important for exposing the internal state of (otherwise inaccessible) devices connected through the HPI. The content of the debug packet can also be implementation specific. The content can include things like branch information (source IP and destination IP), timestamps, indications of internal event triggers, etc. The exposed data can be, for example, by monitoring devices such as logic analyzers for post-processing and fault analysis. Fig.12 An example flit encoding of a debug message type is shown in FIG.
[0120] In some implementations, the buffer memory interface and corresponding protocol can be implemented using the principles and protocols of a general purpose I / O interconnect (GPIO) architecture such as HPI. For example, the physical layer and link layer definitions listed above can also be implemented in the buffer memory protocol. In fact, the logic of the physical layer and link layer used to support the GPIO protocol can be reused at the interface supporting the buffer memory protocol. The buffer memory protocol can also share message classes, such as request, response, and write-back message classes, as well as other examples. Although the opcode values within the buffer memory protocol messages can be interpreted differently than in HPI (or another GPIO protocol), the same general packet format can be used in both the buffer memory protocol and the GPIO interconnect on which it is built. In fact, additional packet and micro-chip fields can be unique to both the buffer memory protocol and the GPIO interconnect protocol.
[0121] In some implementations, the buffer memory protocol utilizing the physical layer and link layer of HPI can be a transaction interface and protocol. This can allow data to be returned out of order to a series of received requests (e.g., read requests). The buffer memory interface can be used to interconnect the buffer chip to the CPU in some implementations. In one implementation, while some protocols send DRAM commands such as activate, read, write, and refresh across the buffer memory bus, in some implementations of the buffer memory protocol discussed herein, a read or write command can simply be sent with an address. The buffer chip can then decode the specific rank and group and decompose the request into DRAM commands. Speculative reads and demand reads can also be supported.
[0122] Go to Fig.13, a simplified block diagram 1300 is shown illustrating an example topology of a computing system including CPU devices 1305, 1310 interconnected by HPI-compliant links. Each CPU 1305, 1310 may be similarly connected to one or more corresponding buffer devices 1315a-1 using a corresponding MemLink. As described above, in some implementations, the MemLink interconnect may be based on the GPIO protocol because the physical and link layers of the MemLink protocol are based on the same physical and link layer definitions of the GPIO protocol (e.g., HPI). Although Fig.13 Not shown, but the CPUs 1305, 1310 may further connect to one or more downstream devices using the GPIO protocol.
[0123] like Fig.13 As further shown in the example of , the buffer device 1315a-1 can be connected to a memory device such as a dual in-line memory module (DIMM) device. The memory corresponding to each buffer device can be considered to be local to the CPU (e.g., 1305, 1301) to which the buffer device is connected. However, other devices (including other CPUs) can access the memory through other slots using links that comply with the GPIO protocol. In some implementations, the port running the buffer memory protocol can only support commands for communicating with the memory and only support the buffer memory protocol (i.e., not the GPIO protocol and the buffer memory protocol). In addition, in some implementations, the GPIO interconnect protocol can support routing and indicate information such as request node identifiers and destination node identifiers (e.g., in its packets). On the other hand, the buffer memory protocol can be a point-to-point interface that does not utilize routing. Therefore, in packets sent using the buffer memory interface, some fields used in the GPIO protocol may be omitted. Alternatively, a field for carrying address decoding information host to the buffer can be specified, as well as other examples.
[0124] In other implementations, the buffer devices 1315a-1 can support a two-level memory topology in which a certain amount of fast memory (e.g., DRAM) is used as a cache for larger, slower memory (e.g., non-volatile memory). In one such implementation, one or more of the buffer devices 1315a-1 can use DDR as near, fast memory and transactional DDR DIMMs as larger "far" memory, among other examples. The transactional DIMMs can use a protocol (e.g., DDR-Transaction (DDR-T)) to communicate to volatile memory single inline memory modules (SIMMs) using transactional commands.
[0125] Steering Fig.14, a simplified block diagram 1400 is presented showing a more detailed representation of a buffer chip 1315 connected to a near memory module and a far memory module (e.g., 1405, 1410). As shown in the above example, the buffer chip 1315 can be communicatively coupled to a CPU device or other processing device 1305 via a buffer memory protocol interconnect link 1415 using a defined buffer memory protocol. The buffer chip 1315 may include logic to perform additional functions. For example, an implementation of the buffer chip 1315 may include a memory controller 1420, a scheduler 1425, flow control logic 1430, error detection logic 1435, and directory state management logic 1436. The logic may be implemented using hardware circuits, firmware, and / or software.
[0126] In some implementations, the memory controller 1420 may include logic to convert requests into a memory-specific protocol (e.g., DDR4) of its memory device. The memory controller 1420 may extract the details of these memory-specific protocols from the host controller logic 1440 of the CPU 1305 (or another host) that communicates with the buffer chip 1315 using the link 1415. The scheduler 1425 may include logic to reorder requests and arbitrate responses in order to achieve higher performance. The buffer chip 1315 may additionally provide features such as flow control and error detection. In some implementations, the flow control logic 1430 may be implemented in the scheduler 1425 or otherwise interoperate with the scheduler 1425 to ensure higher efficiency and higher performance of the buffer chip 1315. The error detection logic 1435 may include logic to support an error correction code (ECC) detection scheme and logic to initiate correction or handle detected errors. The directory state management logic 1436 may maintain the directory state of each row in the memory. In response to some transactions, the directory state of the corresponding row may change. The directory state management logic 1436 may update a record identifying the current directory state of each row in the memory. In some cases, the directory state management logic 1436 may receive data from a home agent (e.g., at the host 1305) to indicate changes to the directory state of the row of the memory. In other instances, the directory state management logic 1436 may include logic to automatically update the directory state based on the nature of the corresponding request change (i.e., without the host indicating the change). This may save bandwidth, among other example advantages, because bandwidth used to indicate at least a portion of the directory state changes may be preserved, where the directory state management logic 1436 allows a portion of these directory state changes to be identified directly at the memory controller (e.g., 1420 at 1315).
[0127] In some implementations, the CPU 1305 may include host controller logic 1440 that can translate CPU (or host) addresses into memory addresses, including arrangements across channels and slots and other functionality. The host 1305 may include multiple processing cores 1445a-d that can allow multiple processes to be executed in parallel on the host 1305. In addition, the host may use one or more caches to cache previously accessed lines from memory (e.g., 1405, 1410) so that the host 1305 can re-access these lines without re-requesting data from the buffer chip 1315. This can help address the higher latency introduced by the buffer chip 1315 in the buffer memory architecture. The cache may include a local cache and an intermediate cache, as well as a last-level cache (LLC) 1450a-d. The cache (including the LLC) may be shared between multiple cores 1445a-d. In fact, various cache architectures may be used with a host that employs one or more processing cores.
[0128] As described above, the link layer of the buffer memory protocol can be a link layer based on the GPIO protocol. For example, it can be used in the buffer memory protocol (e.g., in Fig.10 The 192-bit, 3-slot microchip of the HPI protocol (described and shown in FIG. 1 ) can be implemented as a packet in the buffer memory protocol. The packet can be implemented as one or more microchips. Figures 15A-15C An implementation of microchips in a buffer memory protocol may be represented. In some implementations, read data may be sent back from a buffer device to a host via a combination of headered and headerless memory data (MemData) packets. An entire cache line may be transferred in three headerless microchips (rather than five microchips, as with headered microchips). Thus, headerless packets may provide higher throughput over what may be the most constrained portion of an interface. To accommodate three microchips, headerless packets remove some fields and make assumptions about their values. Most of the assumptions made may be true, but for all read returns, if any assumption is not true, a headered packet containing all fields will be used. In some implementations, a headered packet may begin a read return sequence to allow early identification of a tracker identifier (e.g., HTID, RTID, etc.) for use in mapping read returns to read requests in a transactional buffer memory protocol.
[0129] In some implementations, the headerless packet may contain the information listed in Table 1:
[0130] Table 1: Headerless Packet Fields
[0131] Fields Qty describe data 512 64-byte cache line data Table of contents 2 Two directory bits, which store data that is usually part of the ECC bits HTID 11 Request transaction ID. Due to the earlier HTID, the packet includes the HTID for the read data packet. CRC 48 16-bit CRC per microchip HIB 3 Header indication bit. One per fragment. This bit is always 0 for each fragment of a headerless packet.
[0132] also, Figures 15A to 15CThe figure shows three flits (e.g., flits 0 and 1) that can be used to transfer an entire cache line in a read return. Fig.15A ), Microchip 1 ( Fig. 15B ) and Microsheet 2 ( Fig. 15C Table 2 provides a summary of the examples of three micro-slices used in headerless packets:
[0133] Table 2: Headerless Packet
[0134] Microchip content Data Flake 0 Data from the first 32 bytes of the cache line Data microchip 1 The remainder of the first 32 bytes plus the first part of the second 32 bytes Data Flick 2 The remainder of the second 32 bytes
[0135] The HIB bit can be set to indicate whether the packet is with a header or without a header. The HIB can be set to indicate a packet without a header only for the data micro-slice that follows the header micro-slice. This allows certain other packets to be interleaved in the middle of the packet. When the interleaved packets are completed, the data portion of the original headered packet can be sent in the micro-slice with HIB = 0. This technique can also be used to delay data packets by inserting empty micro-slices. In some implementations, when a header would normally be required, the start of a headerless packet is signaled by the HIB bit being 0. All other packets (including empty micro-slices) may contain a header with the HIB bit set.
[0136] As described above, a headered packet may be a total of five microchips in length and may include one header microchip and four data microchips. To reduce idle latency, a headered microchip may send data microchip 1 twice. The first time is sent with only data from the first 32 bytes and is referred to as microchip la. This allows the microchip to be sent before the second 32 bytes of data are available from the memory device. The result is that a critical 32-byte block of data arrives at the host earlier in idle conditions. Microchip 1 may then be sent a second time, this time complete. It will contain a repetition of the first 32-byte data portion plus the second 32-byte portion. Table 3 describes the five microchips of a headered packet:
[0137] Table 3: Packet with header
[0138] Microchip content Header Packet Header Data Flake 0 Data from the first 32 bytes of the cache line Data Flake 1a Only the remainder of the first 32 bytes. The second 32-byte portion is reserved. The remaining flitter payload bits (RTID[6:3]) are the same as Data Flit 1 Data microchip 1 Complete Flit 1. The first 32-byte portion is retransmitted. The second 32-byte portion is also included. Data Flick 2 The remainder of the second 32 bytes.
[0139] In some implementations, a MemData packet with a header may not be cancelled and will be followed by a data flits. The header is not sent until it is known that data may also be sent. A complete cache line may not be accumulated in the buffer before sending the header.
[0140] In conventional systems, home agents on various hosts may be assigned the task of explicitly updating the directory state of the corresponding row stored in memory after each read. Specifically, updating the directory state may involve writing back the complete cache line plus the directory state. Such an implementation may result in a large amount of bandwidth for updating the directory state, particularly in a buffer memory architecture where the home agent and the memory controller are not tightly coupled. A buffer memory link may be used to interconnect the home agent and the memory controller. Thus, the buffer memory link may be used by the memory controller to receive both requests from the host and to receive writebacks, the sole purpose of which in some cases may be to simply specify a change in the directory state of a given row based on a read. Furthermore, in the case where the directory state is indicated within the cache line itself, the directory state change may involve transmitting the entire copy of the cache line from the home agent to the memory controller via the buffer memory link, resulting in a high bandwidth cost for such an update.
[0141] These are other issues that can be addressed using an improved architecture that utilizes new command types to allow the memory controller to explicitly or implicitly direct the memory controller to update the directory state when possible without guidance from the home agent or other logic outside the memory controller. Therefore, significant write bandwidth on the interface between the home agent and the memory controller can be saved. This interface can be a buffered memory link interface. Other architectures can also utilize such a model, such as implementing the internal grid of the memory controller on the same system on chip (SoC) as the home agent, as well as other examples.
[0142] Figures 16A-16C 1600a-c are diagrams illustrating example read transactions on a buffer memory interconnect involving a host 1305 and a buffer chip 1315. Although this example specifically utilizes a read transaction, it should be appreciated that Figures 16A to 16C The principles shown in the example of can also be applied to other types of requests (including memory invalidation requests). Back FIG. 16A to FIG. 16C , a memory read 1610 may be sent from the host 1305 to the buffer chip 1315 via the buffer memory interconnect. The memory read or read request 1610 may originate from the local host 1305 or from another remote host connected to the local host 1305 (e.g., via an HPI link, among other examples). A memory controller of the buffer chip 1315 may receive the read request 1610 and determine a corresponding memory read operation 1615 to perform in response to the request to retrieve a corresponding row from the memory 1605.
[0143] The read request 1610 may be a request according to a buffered memory protocol. The buffered memory protocol read request 1610 may correspond to a read request 1650 of a general purpose input / output (GPIO) interconnect protocol. The GPIO protocol may also support several different request types, including read and memory invalidation requests. In one example, the GPIO protocol request types may include:
[0144] • RdCur: Requests a non-cacheable "snapshot" copy of a specific cache line
[0145] • RdCode: Requests a shared (or "S") copy of the cache line
[0146] • RdData: Requests an exclusive (or "E") or S copy of the cache line
[0147] • RdDataMig: Requests a modified (or "M"), E, or S copy of a cache line
[0148] • RdInvOwn: Request the M or E copy of a cache line
[0149] • RdInv: Requests the E copy of the cache line from the home agent; any modified copies are committed to memory before receiving the data
[0150] • InvItoE: Request ownership of a cache line in the E state without requiring a data response
[0151] • InvXtoI: Invalidate all caches, including the requesting proxy's cache
[0152] • InvItoM: Request ownership of a cache line in the M empty state, no data required in response, with the intention of writing back the modified line shortly afterwards
[0153] • NonSnpRd: Requests a non-cacheable “snapshot” copy of a cache line from memory
[0154] As shown in the above examples, a set of coherence states may be maintained for the GPIO protocol, including shared ("S"), exclusive ("E"), modified ("M"), and invalid ("I") states, among other potential examples.
[0155] The cache memory protocol itself may define a number of different request types. The cache memory protocol may also maintain a directory state for each cache line, including Any ("A") (e.g., any of the M, E, S, or I consistency states), Shared ("S") (e.g., any of the S or I consistency states), Exclusive ("E"), Invalid ("I") (e.g., guaranteed to be in the I consistency state), or Unknown, among other examples. In one example, the read request types of the cache memory protocol may include:
[0156] •MemRd: Normal memory read. Directory information is retained and not modified
[0157] • MemRdXtoI: Memory read, directory result invalid. If the directory indicates anything other than I, it is written back as I
[0158] • MemInvXtoI: Memory Invalidate. Directory results are invalid. No read data is returned. The memory controller reads the cache line and, if it has not already set the directory to I, overwrites it by setting the directory to I
[0159] • MemRdXtoS: Memory read, directory result is shared. If the directory indicates anything other than S, it is written back as S
[0160] • MemRdData: Memory read, rewritten (if necessary) using the following directory: I to A; S no change; A no change; if a change from A is required, the host must specifically change it
[0161] • MemRdXtoA: Memory read, directory result is any. If the directory indicates anything other than A, it is written back as A
[0162] • MemInvXtoA: Memory invalidate, directory result is any. No read data is returned. The memory controller reads the cache line and, if it has not already done so, overwrites it by setting the directory to A.
[0163] A buffer memory protocol read request type may correspond to a GPIO protocol read request type. A buffer or memory controller may sometimes cover only a subset of cache agents, and therefore, the home logic may be responsible for issuing the correct memory command depending on whether the requested cache agent is covered by the directory. In one implementation, cache agents may be grouped between local and remote cache agents (such as in the examples of Tables 4 and 5), but other alternative groupings may be used in other implementations. In the examples of Tables 4 and 5, a "local" requester is an agent that is not covered by the directory, and a "remote" cache agent is covered by the directory. Tables 4 and 5 list the association between GPIO protocol read request types and buffer memory (MEM) protocol request types. Table 4 lists read requests from a "local" host or cache agent. Table 5 lists read requests originating from a "remote" host or cache agent.
[0164] Table 4: Directory state transitions for read requests from local cache proxy
[0165]
[0166] Table 5: Directory state transitions for read requests from remote cache proxies
[0167]
[0168] like Figures 16A-16C As shown, in some implementations, cache line 1630 may include a field or bit (e.g., 1635) encoded to identify the directory status of cache line 1630. In response to a read request, a copy of cache line 1630 may be returned to the requesting host (e.g., 1305) including the encoded directory status bit 1635. Fig. 16B In some cases, a read transaction may result in a change in the directory state of the corresponding cache line. In fact, Tables 4 and 5 list the state transitions that may occur in the directory state at the time of the read request based on the type of read request and the requested cache line. In addition, in some implementations, the directory state transition may also depend on whether the read request originates from a local cache agent or a remote cache agent. For example, an implementation may use the directory state to primarily track remote agents (e.g., whether the remote agent has a shared copy, an exclusive copy, etc.). In some implementations, the home agent can only know the agent covered by the directory (e.g., a remote agent) or the agent not covered by the directory (e.g., a local agent), which results in these determinations being made only for one or the other of the remote agent or the local agent. The directory bit may reflect the directory state in the remote slot and may be used to determine whether remote snooping should be performed. In some implementations, the local agent may have a copy of the cache line, and a read request from the local agent may typically result in the directory state being transitioned to an invalid ("I") or "clean" state.
[0169] As an example, in Fig. 16B In the illustration of , read request 1610 may cause copy 1630 to be retrieved (at 1620) by the buffer chip 1315 memory controller and forwarded (at 1640) to the host 1305 (via the buffer memory link). In this example, the directory state of cache line 1630 may be "A" when request 1610 is received. The directory state may be encoded in directory state bits 1635. Upon receiving cache line 1630, the host 1305 may change the directory state. In some cases, the change may be based on a result of a snoop performed by the host's home agent. For example, when the request is an "RdCur" read request, the host 1305 may snoop a remote cache agent in association with request 1610. Fig. 16BIn the example of , based on the results of the snoop, the host determines that the directory state should be updated to "S" (as reflected by bit 1635a). To allow this directory update to be transmitted to the buffer memory, the host can write back 1650 cache line 1630 with updated directory state bit 1635a. The memory controller of the buffer chip can then write 1655 the updated cache line with updated directory state bit 1635 to the memory.
[0170] In some cases, the memory controller of the buffer chip 1315 can infer a change in directory state without the change being communicated back to the buffer chip (e.g., via a writeback (e.g., 1650)). For example, as shown in Tables 4 and 5, some combinations of read request type and current directory state will always result in the same directory change (or no change). In Tables 4 and 5, the column "New Directory State" shows the directory state that results from a particular type of read request when the cache line has a particular directory state (and depending on whether the request originates from a local agent or a remote agent). As an example, as shown in Table 5, a change in directory state to an "A" state can be reliably predicted when the read request comes from a remote agent and is an "RdData" request while the directory state is in the "I" state, as well as other examples including those shown in Tables 4 and 5.
[0171] In Tables 4 and 5, when the "new directory state" is indicated as "-", it can be predicted that no change to the directory state will result based on the read request. When the "new directory state" is indicated as "*", changes are possible, but unpredictable. For example, any directory state change may depend on the result of a snoop performed by the requesting host. As an example, in Table 5, when the read request comes from a remote agent and is an "RdData" request while the directory state is in the "A" state, a snoop may be performed by the home agent, resulting in potentially unpredictable changes to the directory state of the cache line. Therefore, in the case indicated by "*", the directory state can sometimes be updated only by the memory controller through a writeback or other explicit communication from the home agent. In the case where the new directory state is predictable, the memory controller can directly update the directory state without the intervention of the local agent.
[0172] For example, in Fig. 16C, an example of receiving a read request 1610 when the cache line is in the "A" state (as recorded in the directory state bits 1635) is shown. In this example, based on the type of read request 1610 and the cache line being in the "A" state (and based on the source of the request), the memory controller can determine that the home agent will update the directory state to "I", as shown. The memory controller can identify the type of read request and the source of the request based on the fields of the read request, and deduce what the directory state will be changed to based on this information. The memory controller can preemptively change the directory state bits of the cache line copy in the memory by writing (at 1645) a new copy of the cache line with the new directory state bits (1635b) to the memory (at 1665) to reflect the changes that the home agent will make to the directory state (at 1660). Therefore, the memory controller of the buffer chip 1315 can be equipped with logic for determining whether a directory state change can be predicted based on a specific read request type from a specific cache agent when the cache line is in a specific directory state. The memory controller may likewise determine whether it should wait and rely on communications from the home agent to identify unpredictable changes to the directory state. Similarly, the home agent may include logic for determining when updated directory state bits should be written back (or otherwise communicated) to facilitate updates to the directory state of the cache lines recorded in memory, or whether the home agent may forgo any such communications based on the understanding that the separate memory controller will perform these updates directly. Both the memory controller and the home agent may include or access a data structure having machine-readable data that embodies combined information describing directory state changes, such as the information described in example Tables 4 and 5 and other examples.
[0173] Fig.17 1700 is a simplified representation of a cache line 1630 according to some examples. In this particular example, the cache line can include payload data 1705 as well as error correction code (ECC) data (e.g., in ECC field 1710). In this example, directory status bits 1635 can be included in or pushed out of ECC bits 1710 as stored in DRAM. For example, in one example, the directory bits can be two directory bits stored within a 64-byte cache line. Directory status bits 1635 can be set by a host device on writes and by a memory controller on some read transactions (e.g., as Fig. 16C ) directly, or based on other read transactions (for example, Fig. 16B In one example, the two-bit directory status bit implementation can be implemented according to the example of Table 6 below:
[0174] Table 6: Directory status codes
[0175] Dir[1] Dir[0] state describe Remote Slot Cache Local slot cache 0 0 I Invalid state. The cache line is not cached in the remote slot cache. It may be I, S, or F in the local slot cache. Not cached Any status 0 1 - Not used 1 1 S Shared. The cache line is cached in the S or F state, or is not cached at all I, S, or F I, S, or F 1 0 A Any. The cache line can be in any cache state (M, E, F, S, or I) in the remote slot. It can also be in any state in the local slot. Any status (M, E, F, S, or I) Any status (M, E, F, S, or I)
[0176] Other implementations may support additional or alternative directory states. In addition, other implementations may use additional directory state bits. In addition, although Tables 4 and 5 list various read request types and predicted directory state changes resulting from these read requests, other implementations may support alternative sets of read request types and predicted impacts on directory states based on read request types, used directory states, and the source of the read request, as well as other alternatives and examples.
[0177] HPI and the associated buffer memory access protocol may be incorporated into any kind of computing device and system, including mainframes, server systems, personal computers, mobile computers (such as tablets, smartphones, personal digital systems, etc.), smart appliances, gaming or entertainment consoles and set-top boxes, among other examples. Fig.18 , which shows a block diagram of a second system 1800 according to an embodiment of the present invention. Fig.18 As shown, multiprocessor system 1800 is a point-to-point interconnect system and includes a first processor 1870 and a second processor 1880 coupled via a point-to-point interconnect 1850. Each of processors 1870 and 1880 may be a certain version of a processor. In one embodiment, 1852 and 1854 are part of a serial point-to-point coherent interconnect structure (such as a high-performance architecture). As a result, the present invention can be implemented within a QPI architecture.
[0178] Although only two processors 1870, 1880 are shown, it should be understood that the scope of the present invention is not limited in this regard. In other embodiments, one or more additional processors may be present in a given processor. Processors 1870 and 1880 are shown as including integrated memory controller units 1872 and 1882, respectively. Processor 1870 also includes point-to-point (PP) interfaces 1876 and 1878 as part of its bus controller unit; similarly, second processor 1880 includes PP interfaces 1886 and 1888. Processors 1870, 1880 may exchange information via point-to-point (PP) interface 1850 using PP interface circuits 1878, 1888. As shown in FIG. Fig.18 As shown, IMCs 1872 and 1882 couple the processors to respective memories, namely, memory 1832 and memory 1834, which may be portions of main memory locally attached to the respective processors.
[0179] Processors 1870, 1880 each exchange information with chipset 1890 via separate PP interfaces 1852, 1854 using point-to-point interface circuits 1876, 1894, 1886, 1898. Chipset 1890 also exchanges information with high-performance graphics circuit 1838 via interface circuit 1892 along high-performance graphics interconnect 1839.
[0180] A shared cache (not shown) may be included in either processor or external to both processors; but connected to the processors through the PP interconnect so that if the processors are placed in a low power mode, local cache information of either or both processors may be stored in the shared cache.
[0181] Chipset 1890 may be coupled to first bus 1816 via interface 1896. In one embodiment, first bus 1816 may be a peripheral component interconnect (PCI) bus or a bus such as a PCI Express bus or another third generation I / O interconnect bus, but the scope of the invention is not limited in this regard.
[0182] like Fig.18 As shown, various I / O devices 1814 are coupled to first bus 1816, along with a bus bridge 1818 which couples first bus 1816 to a second bus 1820. In one embodiment, second bus 1820 comprises a low pin count (LPC) bus. Various devices are coupled to second bus 1820 including, for example, a keyboard and / or mouse 1822, communication devices 1827, and a storage unit 1828 such as a disk drive or other mass storage device, which typically includes instructions / code and data 1830 in one embodiment. Additionally, audio I / O 1824 is shown coupled to second bus 1820. Note that other architectures are possible in which the components included and the interconnect architecture vary. For example, instead of Fig.18 Instead of a point-to-point architecture, the system can implement a multi-point branch bus or other such architecture.
[0183] Reference now Fig.19 , shows a block diagram of an embodiment of a multi-core processor. Fig.19 As shown in the embodiment of , the processor 1900 includes multiple domains. Specifically, the core domain 1930 includes multiple cores 1930A-1930N, the graphics domain 1960 includes one or more graphics engines with a media engine 1965, and the system agent domain 1910.
[0184] In various embodiments, system agent domain 1910 handles power control events and power management so that the units (e.g., cores and / or graphics engines) of domains 1930 and 1960 are independently controllable to dynamically operate in an appropriate power mode / level (e.g., active, turbo, sleep, hibernate, deep sleep, or other state similar to an advanced configuration power interface) based on the activity (or inactivity) taking place in a given unit. Each of domains 1930 and 1960 can operate at different voltages and / or powers, and further, each unit within a domain can potentially operate at independent frequencies and voltages. It is noted that while only three domains are shown, it should be understood that the scope of the present invention is not limited in this regard, and additional domains may be present in other embodiments.
[0185] As shown, each core 1930 includes a low-level cache in addition to various execution units and additional processing elements. Here, the various cores are coupled to each other and to a shared cache memory formed by multiple units or slices 1940A-1940N of the last level cache (LLC); these LLCs often include storage and cache controller functionality and are shared between the cores and potentially also between the graphics engines.
[0186] As seen, ring interconnect 1950 couples the cores together and provides interconnection between core domain 1930, graphics domain 1960, and system agent circuitry 1910 via a plurality of ring stops 1952A-1952N (each at a coupling between a core and an LLC slice). Fig.19 As seen in FIG. 1 , interconnect 1950 is used to carry various information, including address information, data information, response information, and snoop / invalid information. Although a ring interconnect is shown, any known on-die interconnect or structure may be utilized. As illustrative examples, some of the structures discussed above (e.g., another on-die interconnect, Intel on-chip system structure (IOSF), advanced microcontroller bus architecture (AMBA) interconnect, multi-dimensional mesh structure, or other known interconnect architectures) may be utilized in a similar manner.
[0187] As further depicted, the system agent domain 1910 includes a display engine 1912, which will provide control of the associated display and an interface to the associated display. The system agent domain 1910 may include other units, such as: an integrated memory controller 1920, which provides an interface to the system memory (e.g., DRAM implemented with multiple DIMMs); consistency logic 1922 to perform memory consistency operations. There may be multiple interfaces to enable interconnection between the processor and other circuits. For example, in one embodiment, at least one direct media interface (DMI) 1916 interface and one or more PCIe™ interfaces 1914 are provided. The display engine and these interfaces are typically coupled to the memory via a PCIe™ bridge 1918. In addition, in order to provide communication between other agents (such as additional processors or other circuits), one or more other interfaces may be provided.
[0188] While the present invention has been described with respect to a limited number of embodiments, those skilled in the art will recognize numerous modifications and variations therefrom. The appended claims are intended to cover all such modifications and variations that fall within the true spirit and scope of the present invention.
[0189] A design can go through various stages from creation to simulation to manufacturing. Data representing a design can represent the design in a variety of ways. First, as useful in simulation, the hardware can be represented using a hardware description language (HDL) or another functional description language. In addition, a circuit-level model with logic and / or transistor gates can be generated at a certain stage in the design process. In addition, most designs reach a data level representing the physical placement of various devices in the hardware model at a certain stage. In the case of using conventional semiconductor manufacturing technology, the data representing the hardware model can be data specifying the presence or absence of various features on different mask layers of a mask used to produce an integrated circuit. In some implementations, such data can be stored in a database file format such as a Graphic Data System II (GDS II), an Open Artwork System Interchange Standard (OASIS), or a similar format.
[0190] In some implementations, software-based hardware models and HDL and other functional description language objects may include register transfer language (RTL) files, as well as other examples. Such objects may be machine-parseable, so that a design tool may accept an HDL object (or model), parse the HDL object for the properties of the described hardware, and determine the physical circuit and / or on-chip layout from the object. The output of the design tool may be used to manufacture a physical device. For example, a design tool may determine the configuration of various hardware and / or firmware elements from an HDL object, such as bus width, registers (including size and type), memory blocks, physical link paths, structural topology, and other properties of the system to be implemented to achieve the modeling in the HDL object. Design tools may include tools for determining the topology and structural configuration of systems on chip (SoCs) and other hardware devices. In some instances, HDL objects may be used as the basis for developing models and design files that can be used by manufacturing equipment to manufacture the described hardware. In fact, the HDL object itself may be provided as input to the manufacturing system software to facilitate the described hardware.
[0191] In any representation of the design, data may be stored in any form of machine-readable medium. Memory or magnetic or optical storage, such as a disk, may be a machine-readable medium to store information transmitted via light or electrical waves that are modulated or otherwise generated to transmit such information. When an electrical carrier wave indicating or carrying a code or design is transmitted, a new copy is made to the extent that duplication, buffering, or retransmission of the electrical signal is performed. Thus, a communications provider or network provider may at least temporarily store an article of manufacture such as information encoded into a carrier wave on a tangible, machine-readable medium, thereby implementing the techniques of embodiments of the present disclosure.
[0192] As used herein, a module refers to any combination of hardware, software, and / or firmware. For example, a module includes hardware, such as a microcontroller, associated with a non-transitory medium to store code adapted to be executed by the microcontroller, and therefore, in one embodiment, a reference to a module refers to hardware that is particularly configured to recognize and / or execute code to be retained on a non-transitory medium. In addition, in another embodiment, the use of a module refers to a non-transitory medium including code, which is particularly adapted to be executed by a microcontroller to perform a predetermined operation. And as can be inferred, in yet another embodiment, the term module (in this example) may refer to a combination of a microcontroller and a non-transitory medium. Typically, module boundaries that are described as separate typically change and potentially overlap. For example, a first and a second module may share hardware, software, firmware, or a combination thereof, while potentially retaining some independent hardware, software, or firmware. In one embodiment, the use of the term logic includes hardware, such as transistors, registers, or other hardware, such as programmable logic devices.
[0193] In one embodiment, the use of the phrase "configured to" refers to arranging, putting together, manufacturing, offering for sale, importing and / or designing a device, hardware, logic or element to perform a specified or determined task. In this example, if it is designed, coupled and / or interconnected to perform the specified task, then no device or element that is operating is still "configured to" perform the specified task. As an illustrative example only, a logic gate can provide a 0 or 1 during operation. But a logic gate is "configured to" provide an enable signal to the clock, not including every potential logic gate that can provide a 1 or 0. Instead, a logic gate is a logic gate coupled in a way that a 1 or 0 output is used to enable the clock during operation. Note again that the use of the term "configured to" does not require operation, but instead focuses on the potential state of the device, hardware and / or element, where the potential state of the device, hardware and / or element is designed to perform a specific task when the device, hardware and / or element is operating.
[0194] Additionally, in one embodiment, the use of the phrases "to," "can," and / or "operable to" refers to some devices, logic, hardware, and / or elements designed in such a manner that enables the device, logic, hardware, and / or element to be used in a specified manner. As noted above, in one embodiment, the use of to, can, or operable to, refers to a potential state of a device, logic, hardware, and / or element, wherein the device, logic, hardware, and / or element is not operating but is designed in such a manner that enables the device to be used in a specified manner.
[0195] As used herein, value includes any known representation of a number, state, logic state, or binary logic state. Typically, the use of logic levels, logic values, or logical values is also referred to as 1 and 0, which represent only binary logic states. For example, 1 refers to a high logic level and 0 refers to a low logic level. In one embodiment, a storage unit, such as a transistor or a flash memory unit, may be able to hold a single logic value or multiple logic values. However, other representations of values in a computer system are used. For example, the decimal number 10 may also be represented as a binary value of 1010 and the letter A in hexadecimal. Therefore, a value includes any representation of information that can be held in a computer system.
[0196] In addition, states may be represented by values or portions of values. For example, a first value, such as a logical 1, may represent a default or initial state, while a second value, such as a logical 0, may represent a non-default state. In addition, in one embodiment, the terms reset and set, refer to default and updated values or states, respectively. For example, a default value potentially includes a high logical value, i.e., reset, while an updated value potentially includes a low logical value, i.e., set. Note that any combination of values may be used to represent any number of states.
[0197] Embodiments of the methods, hardware, software, firmware, or code set forth above may be implemented via instructions or code stored on a machine-accessible, machine-readable, computer-accessible, or computer-readable medium that is executable by a processing element. Non-transitory machine-accessible / readable media include any mechanism that provides (i.e., stores and / or transmits) information in a form that is readable by a machine, such as a computer or electronic system. For example, non-transitory machine-accessible media include random access memory (RAM), such as static RAM (SRAM) or dynamic RAM (DRAM); ROM; magnetic or optical storage media; flash memory devices; electrical storage devices; optical storage devices; acoustic storage devices; other forms of storage devices for holding information received from a transient (propagating) signal (e.g., a carrier wave, an infrared signal, a digital signal); etc., which would be distinguished from a non-transitory medium from which information may be received.
[0198] Instructions for programming logic to perform embodiments of the present invention may be stored in a system's memory, such as a DRAM, cache, flash memory, or other storage device. In addition, instructions may be distributed via a network or by means of other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), but not limited to, a floppy disk, an optical disk, a compact disk read-only memory (CD-ROM), and a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic card or an optical card, a flash memory, or a tangible, machine-readable storage device for transmitting information via an electrical, optical, acoustic, or other form of propagation signal (e.g., a carrier wave, an infrared signal, a digital signal, etc.) through the Internet. Therefore, computer-readable media include any type of tangible machine-readable media that is suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).
[0199] The following examples pertain to embodiments according to the present specification. One or more embodiments may provide an apparatus, system, machine-readable storage, machine-readable medium, hardware-based and / or software-based logic, and method for: receiving a request over a link, the request requesting a particular row in a memory; identifying a directory state record in a memory, the directory state record identifying a directory state of the particular row; identifying a type of the request; determining, based on the directory state of the particular row and the type of the request, that the directory state of the particular row is to be changed from the particular state to a new state; and in response to receiving the request, changing the directory state record to reflect the new state. A copy of the particular row may be sent in response to the request.
[0200] In one example, the method is performed at least in part by a memory controller.
[0201] In one example, the request is to be received from a host device that is separate from the memory controller.
[0202] In one example, the request will be received from the host device over a buffered memory access link.
[0203] In one example, the memory controller is provided on a buffer chip that is separate from the host device.
[0204] In one example, the request originates from a specific caching agent, and the change will be determined based on the type of the specific caching agent.
[0205] In one example, the type of the particular caching agent is one of a local caching agent or a remote caching agent.
[0206] In one example, the request includes one of a read request and a memory invalidate request.
[0207] In one example, a directory status record includes two or more bits, and the bits will be included in the row and reflect the particular status.
[0208] In one example, the memory controller writes the new state directly to a specific row in memory.
[0209] In one example, the home agent updates the directory state to a new state, and the memory controller updates the directory state to the new state independently of the home agent.
[0210] In one example, the bits are included in the error correction code bits of a particular row.
[0211] In one example, the memory controller determines that a directory state change associated with another request is unpredictable.
[0212] In one example, the memory controller is further configured to receive a communication from the home agent indicating a directory status change to the row corresponding to the other request, and based on the communication, perform a write to the row corresponding to the other request to update the directory status of the row corresponding to the other request.
[0213] In one example, the communication includes a writeback of a line corresponding to the other request, wherein the written back line indicates a directory status of the line corresponding to the other request.
[0214] In one example, the directory status of the row corresponding to the other request includes a directory status record of the row corresponding to the other request.
[0215] In one example, the directory state is one of a set of directory states, and the set of directory states includes a shared state, an any state, and an invalid state.
[0216] One or more embodiments may provide apparatus, systems, machine-readable storage, machine-readable media, methods, and hardware-based and / or software-based logic (e.g., implementing cache proxy logic) for: determining a buffer memory request corresponding to a specific row of memory and having a specific request type among multiple request types; sending the request to a buffer chip via a buffer memory link; receiving a copy of the specific row of memory in response to the request; determining that the specific row is in a specific directory state based on a directory status record included in the specific row; determining a change to the directory state; and determining whether to transmit the change to the buffer chip based on the specific directory state and the specific request type of the request.
[0217] In one example, if the change can be predicted based on the specific directory state and the specific request type being requested, the change is not communicated to the buffer chip.
[0218] In one example, if the change could not be predicted by the buffer chip based on the particular directory state and the particular request type of the request, the change is communicated to the buffer chip.
[0219] One or more embodiments may provide a system including: a buffer chip; a memory accessed by the buffer chip; and a processor block coupled to the buffer chip by a memory access link. The buffer chip includes a memory controller for: receiving a request related to a specific line in the memory from the processor block by the memory access link; identifying a directory status record in the memory, the directory status record identifying the directory status of the specific line; identifying the type of the request according to the request; determining that the directory status of the specific line is to be changed from a specific state to a new state based on the directory status of the specific line and the type of the request; and in response to receiving the request, changing the directory status record to reflect the new state.
[0220] In one example, the processor block interfaces with one or more other devices via a general purpose input / output (GPIO) interconnect link, the memory access link is different from the GPIO interconnect link, and the physical layer of the memory access link is based on the physical layer of the GPIO interconnect link.
[0221] References throughout this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the various appearances of the phrases "in one embodiment" or "in an embodiment" throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0222] In the foregoing description, detailed descriptions have been given with reference to specific exemplary embodiments. However, it will be apparent that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention as set forth in the appended claims. Therefore, the description and drawings should be viewed in an illustrative sense rather than a restrictive sense. Furthermore, the foregoing use of embodiments and other exemplary language do not necessarily refer to the same embodiment or the same example, but may refer to different and distinct embodiments as well as potentially referring to the same embodiment.
Claims
1. A device comprising: Buffer equipment; a processor block for performing image operations and coupled to the buffer device via a memory access link utilizing a memory protocol that shares buffers of different message classes; a memory controller coupled to a memory module device including a far memory module viewed as local by the processor block and a near memory module used as a cache for the far memory module; An on-chip interconnect fabric, the on-chip interconnect fabric is used to carry data information, the on-chip interconnect fabric is configured as an I / O interface and / or I / O interconnect coupled to an I / O device; coherence logic to perform memory coherence operations; and A graphics domain is associated with the processor block.
2. The apparatus of claim 1, wherein the buffer device is to store memory protocol packets transmitted from the memory module device to the processor block.
3. The apparatus of claim 2, wherein credit-based flow control is used to implement a virtual channel for the memory protocol packets.
4. The device according to any one of claims 1 to 3, wherein: The buffered memory protocol used by the memory access link maintains a consistency state, and the consistency state includes an exclusive state, a modified state, a shared state, and an invalid state.
5. The apparatus of claim 1, further comprising a plurality of domains, the plurality of domains including a system agent domain for handling power control and power management so that independent domains are individually controllable.
6. The device according to claim 1, wherein: The processor block is to be coupled to another processor block via a High Performance Interconnect (HPI) compatible link.
7. The device according to claim 1, wherein: The consistency logic includes a directory that maintains consistency states, including an exclusive state, a modified state, a shared state, and an invalid state.
