A versatile communications interface that supports memory sharing between data processing systems.

Abstract

In a data processing environment, a communications interface of a second host data processing system receives a host command in a first command set from a first host data processing system. The host command specifies a memory access to a memory coupled to the second host data processing system. The communications interface translates the host command into a command in a second command set that is different and emulates the coupling of an additional functional unit to the communications interface. The communications interface presents a second command to a host bus protocol interface of the second host data processing system. Based on receiving the second command, the host bus protocol interface initiates a host bus protocol memory access request on a system fabric of the second host data processing system that specifies the memory access.

Description

[Technical Field]

[0001] The present invention relates generally to data processing, and more particularly to an improved communication interface and communication protocol for communication between components of a data processing system and the data processing system. [Background technology]

[0002] A traditional multiprocessor (MP) data processing system comprises multiple processing units (each of which may include one or more processor cores and their various cache memories), input / output (I / O) devices, and data storage, which may include both system memory (volatile and / or nonvolatile) and nonvolatile mass storage. As processor technology continues to mature and code sizes and working data sets continue to grow, system memory increasingly becomes a dominant factor in the overall cost of enterprise-scale systems. As a result, data processing system designs that support dynamic sharing of system memory among different data processing systems, or the sharing of large, centralized "memory tanks" between data processing systems, are becoming increasingly favored, in that the per-system cost of system memory can be reduced.

[0003] Some data processing systems support sharing of system memory through dedicated hardware, which tends to be expensive (reducing or negating the benefits of memory sharing) and tends not to scale well as enterprises grow in size. The present application recognizes that it would be useful and desirable to extend the communication interfaces of existing data processing systems to support communication between systems and / or communication between systems and a centralized memory tank without increasing cost and complexity. Summary of the Invention

[0004] In at least one embodiment, a communication interface of a second host data processing system receives a host command in a first command set from a first host data processing system. The host command specifies a memory access to a memory coupled to the second host data processing system. The communication interface translates the host command into a command in a different second command set that emulates the coupling of an attached function unit to the communication interface. The communication interface presents a second command to a host bus protocol interface of the second host data processing system. Based on receipt of the second command, the host bus protocol interface initiates a host bus protocol memory access request on the system fabric of the second host data processing system that specifies the memory access. As a result of this process, the host data processing system can employ an existing communication interface suitable for supporting the attachment of attached function units for memory sharing between hosts.

[0005] In at least one embodiment, a page table entry at an address specified in the second command is fixed in a page frame table of the second host data processing system, and with the page table entry fixed in the page frame table, asymmetry between the first and second command sets is permitted.

[0006] In at least one embodiment, the communication interface is a first communication interface, the second host data processing system includes a second communication interface, and the host command is the first host command. In such an embodiment, the second communication interface issues a second host command specifying a memory access based on receiving the host bus protocol memory access request over the system fabric. In this manner, the memory access request is conveyed over the system fabric of the second host data processing system to an additional memory or a third host data processing system coupled to the second communication interface.

[0007] In at least one embodiment, the communications interface includes a first mode of operation that supports attachment of an additional function unit to a second host data processing system and a second mode of operation that supports coupling of a first host data processing system to a second host data processing system for memory sharing between the hosts. In such an embodiment, the communications interface is configured in the second mode of operation to support memory sharing between the hosts.

[0008] In at least one embodiment, a communications controller for a host data processing system including a system fabric includes a controller circuit configured to receive a host command in a first command set from another data processing system, the host command specifying a memory access to a memory coupled to the host data processing system. The host command is translated by the controller circuit into a command in a different second command set that emulates the coupling of an additional functional unit to the communications controller. The controller circuit presents a second command to a host bus protocol interface, which, based on receipt of the second command, initiates a host bus protocol memory access request specifying the memory access over the system fabric of the host data processing system.

[0009] In at least one embodiment, a design structure for designing, manufacturing, or testing an integrated circuit is tangibly embodied in a machine-readable storage device. The design structure includes a communication controller for a host data processing system including a system fabric. The communication controller includes a controller circuit configured to receive host commands in a first command set from another data processing system, the host commands specifying memory accesses to memory coupled to the host data processing system. The host commands are translated by the controller circuit into commands in a second command set that emulate the coupling of an additional functional unit to the communication controller. The controller circuit presents the second commands to a host bus protocol interface, which, based on receipt of the second commands, initiates, on the system fabric of the host data processing system, a host bus protocol memory access request specifying the memory access.

[0010] According to one aspect, a method of communication in a data processing environment is provided, the method including receiving, from a first host data processing system, a host command in a first command set at a communication interface of a second host data processing system, the host command specifying a memory access to a memory coupled to the second host data processing system; translating the host command into a command in a different second command set that emulates the coupling of an additional functional unit to the communication interface; presenting the second command to a host bus protocol interface of the second host data processing system; and initiating, by the host bus protocol interface, a host bus protocol memory access request specifying the memory access on a system fabric of the second host data processing system based on receipt of the second command.

[0011] According to another aspect, there is provided a communications controller for a host data processing system including a system fabric, the communications controller comprising controller circuitry configured to: receive a host command in a first command set from another data processing system, the host command specifying a memory access to a memory coupled to the host data processing system; translate the host command into a command in a different second command set that emulates coupling of an additional functional unit to the communications controller; present the second command to a host bus protocol interface; and initiate, by the host bus protocol interface, a host bus protocol memory access request specifying the memory access on the system fabric of the host data processing system based on receipt of the second command.

[0012] In one embodiment, there is provided a communications controller of the preceding paragraph, wherein the host bus protocol memory access command is a read command, and the controller circuitry is further configured to receive data specified by the host bus protocol memory access request from a system memory of the host data processing system, and issue a read response including the data to the other data processing system.

[0013] According to another aspect, there is provided a design structure tangibly embodied in a machine-readable storage device for designing, manufacturing, or testing an integrated circuit, the design structure including a communications controller for a host data processing system including a system fabric, the communications controller comprising controller circuitry configured to: receive a host command in a first command set from another data processing system, the host command specifying a memory access in a memory coupled to the host data processing system; translate the host command into a command in a different second command set that emulates the coupling of an additional functional unit to the communications controller; present the second command to a host bus protocol interface; and initiate, by the host bus protocol interface, a host bus protocol memory access request specifying the memory access, on the system fabric of the host data processing system based on receipt of the second command.

[0014] According to another embodiment, there is provided a system comprising the communications controller of paragraph 13, wherein the communications controller is a first communications controller, and a second communications controller of another host data processing system is coupled to the first communications controller of the host data processing system, the second communications controller including controller circuitry configured to receive read responses and convert the read responses to a different second set of responses that emulate an additional memory coupled to the second communications controller.

[0015] According to another embodiment, there is provided a system comprising the communications controller of paragraph 12, wherein the host bus protocol memory access command is a write command, and wherein the controller circuitry is further configured to issue a write response to the other data processing system indicating success of the write command, regardless of success or failure of the write command.

[0016] According to another embodiment, there is provided a system comprising the communications controller of paragraph 12, wherein a host data processing system is coupled to the communications controller, the host data processing system includes a system memory storing a page frame table, the second command specifies an address, and the host data processing system is configured to fix a page table entry for the address within the page frame table in the system memory.

[0017] According to another embodiment, a system is provided in which the host command is a first host command and the system includes the communications controller of paragraph 12, wherein the communications controller is the first communications controller, a host data processing system is coupled to the communications controller, the host data processing system includes a second communications controller, and the second communications controller issues a second host command specifying a memory access based on receiving the host bus protocol memory access request on the system fabric.

[0018] Preferred embodiments of the present invention will now be described, by way of example only, with reference to the following drawings in which: [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a high-level block diagram of an exemplary host data processing system, according to one embodiment. [Figure 2] FIG. 2 is a more detailed block diagram of an exemplary processing unit of a host data processing system, according to one embodiment. [Figure 3] 1 illustrates an exemplary protocol stack for a communication interface between a host data processing system and an attached functional unit or memory (AFUM), according to one embodiment. [Figure 4]FIG. 1 illustrates an exemplary protocol stack for a communication interface configured to support attachment of a memory device to a host data processing system, according to one embodiment. [Figure 5] FIG. 1 illustrates an exemplary protocol stack for a communication interface configured to support the attachment of an attached functional unit (AFU) to a host data processing system, according to one embodiment. [Figure 6] FIG. 1 illustrates a time-space diagram of a read command for an AFU, according to one embodiment. [Figure 7] 1A-1C are time-space diagrams of various responses to a read command of an AFU by a host data processing system, according to one embodiment. [Figure 8] 1A-1C are time-space diagrams of various responses to a read command of an AFU by a host data processing system, according to one embodiment. [Figure 9] 1A-1C are time-space diagrams of various responses to a read command of an AFU by a host data processing system, according to one embodiment. [Figure 10] 1A-1C are time-space diagrams of various responses to a read command of an AFU by a host data processing system, according to one embodiment. [Figure 11] FIG. 1 illustrates a time-space diagram of an AFU write command, according to one embodiment. [Figure 12] 4A-4C illustrate time-space diagrams of various responses to an AFU write command by a host data processing system, according to one embodiment. [Figure 13] 4A-4C illustrate time-space diagrams of various responses to an AFU write command by a host data processing system, according to one embodiment. [Figure 14] 4A-4C illustrate time-space diagrams of various responses to an AFU write command by a host data processing system, according to one embodiment. [Figure 15]4A-4C illustrate time-space diagrams of various responses to an AFU write command by a host data processing system, according to one embodiment. [Figure 16] 1 is a high-level logical flowchart of an exemplary method by which a host transaction layer of a host data processing system responds to an AFU's commands, according to one embodiment. [Figure 17] 1 is a high-level logical flowchart of an exemplary method by which an AFU issues an AFUM command to a host data processing system and processes a host response, according to one embodiment. [Figure 18] 1 is a time-space diagram of a read command issued by a host data processing system to an attached memory, according to one embodiment. [Figure 19] 4 is a time-space diagram of various responses of an attached memory to a host data processing system's read command, according to one embodiment. [Figure 20] 4 is a time-space diagram of various responses of an attached memory to a host data processing system's read command, according to one embodiment. [Figure 21] 1 is a time-space diagram of a write command issued by a host data processing system to an attached memory, according to one embodiment. [Figure 22] 1 is a time-space diagram of various responses of an attached memory to a host data processing system write command, according to one embodiment. [Figure 23] 1 is a time-space diagram of various responses of an attached memory to a host data processing system write command, according to one embodiment. [Figure 24] 1 is a high-level logical flowchart of an exemplary method by which a transaction layer providing an AFUM responds to commands of a host data processing system, according to one embodiment. [Figure 25] 1 is a high-level logical flowchart of an exemplary method by which a host transaction layer of a host data processing system issues commands to an AFUM and processes AFUM responses, according to one embodiment. [Figure 26] FIG. 1 illustrates an exemplary protocol stack for a communications interface configured to support coupling of a host data processing system to another host data processing system to enable memory sharing, according to one embodiment. [Figure 27] FIG. 27 is a more detailed block diagram of the command, control and credit, and response translation (CCR XLATE) layer in the protocol stack of FIG. 26. [Figure 28] 4 is a time-space diagram of a read command issued by a host data processing system to an attached host data processing system, according to one embodiment. [Figure 29] 10 is a time-space diagram of various responses by an attached host processing system to a read command of another host data processing system, according to one embodiment. [Figure 30] 10 is a time-space diagram of various responses by an attached host processing system to a read command of another host data processing system, according to one embodiment. [Figure 31] 4 is a time-space diagram of a write command issued by a host data processing system to an attached host data processing system, according to one embodiment. [Figure 32] 10A-10C are time-space diagrams of various responses by an attached host processing system to a write command of another host data processing system, according to one embodiment. [Figure 33] 10A-10C are time-space diagrams of various responses by an attached host processing system to a write command of another host data processing system, according to one embodiment. [Figure 34] 4 depicts a high-level logical flowchart of an exemplary process for initializing data processing systems to share memory, according to one embodiment. [Figure 35]1 is a time-space diagram of a write command issued by an AFU to a host data processing system and the associated response of the host data processing system, according to one embodiment implementing a fast write response mode. [Figure 36] 1 is a high-level logical flowchart of an exemplary method by which a host transaction layer of a host data processing system responds to an AFUM write command, according to one embodiment. [Figure 37] 1 is a time-space diagram of a write command issued by one host data processing system to another host data processing system, and an associated write response, according to one embodiment implementing a fast write response mode. [Figure 38] FIG. 1 is a high-level block diagram of an exemplary topology of a data processing environment in which multiple host data processing systems are coupled to enable sharing of memory of the host data processing systems and / or memory within a memory device, according to one embodiment. [Figure 39] 1 is a space-time diagram of a multi-hop read command issued by an initiating host data processing system to a receiving host data processing system via an intervening host data processing system, and associated responses, according to one embodiment. [Figure 40] 1 is a space-time diagram of a multi-hop write command issued by an initiating host data processing system to a receiving host data processing system via an intervening host data processing system, and associated responses, according to one embodiment. [Figure 41] 10 is a space-time diagram of a multi-hop write command issued by an initiating host data processing system to a receiving host data processing system via an intervening host data processing system, and associated responses, according to another embodiment. [Figure 42] FIG. 1 is a data flow diagram illustrating a design process, according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0020] Referring now to the figures, wherein like reference numerals refer to like corresponding parts throughout, and with particular reference to Figure 1, there is shown a high-level block diagram illustrating an exemplary data processing system 100, according to one embodiment. In various use cases and topologies, a data processing system such as data processing system 100, which includes hardware components and may further include software and / or firmware components, is sometimes referred to in the art as a "host" or a "host data processing system."

[0021] In the illustrated embodiment, data processing system 100 is a cache-coherent multiprocessor (MP) data processing system that includes multiple processing nodes 102 for processing data and instructions. The processing nodes 102 are coupled to a system interconnect 110 for communicating address, data, and control information. The system interconnect 110 may be implemented, for example, as a bus interconnect, a switch interconnect, or a hybrid interconnect.

[0022] In the illustrated embodiment, each processing node 102 is implemented as a multi-chip module (MCM) including one or more (e.g., four) processing units 104a-104d, each preferably implemented as an integrated circuit. The processing units 104 within each processing node 102 are coupled to each other and to a system interconnect 110 by a local interconnect 114, which, like the system interconnect 110, may be implemented, for example, using one or more buses and / or switches. The system interconnect 110 and the local interconnect 114 together form a system fabric. In at least some preferred embodiments, communication over the system fabric conforms to a so-called host bus protocol, which, among other things, defines a predefined set of legal requests, responses, and control information that may be conveyed between communicating participants (e.g., caches, memory controllers, etc.) over the system fabric.

[0023] As described in further detail below with reference to FIG. 2, in some embodiments, one or more of the processing units 104 (and possibly all of the processing units 104) each include a memory controller 106 coupled to a local interconnect 114 to provide an interface to a respective system memory 108. Data and instructions residing in the system memory 108 may typically be accessed, cached, and modified by processor cores within any processing unit 104 of any processing node 102 within the data processing system 100. In alternative embodiments, one or more memory controllers 106 (and system memories 108) may be directly coupled, or indirectly coupled (e.g., via a switch), to the system interconnect 110 instead of the local interconnect 114.

[0024] Those skilled in the art will appreciate that data processing system 100 of Figure 1 can include many additional, not-shown, components, such as interconnection bridges, non-volatile storage, ports for connection to a network, or additional devices. Such additional components are not necessary to an understanding of the described embodiments and are therefore not shown in Figure 1 or described further herein. However, it should also be understood that the extensions described herein are applicable to data processing systems of a variety of architectures and are in no way limited to the generalized data processing system architecture shown in Figure 1.

[0025] 2, a more detailed block diagram of an exemplary processing unit 104 and system memory 108 is shown, according to one embodiment. In the illustrated embodiment, each processing unit 104 is an integrated circuit that includes one or more processor cores 200 for processing instructions and data. In the illustrated example, processor core 200 includes one or more execution units 202 that execute instructions from multiple concurrently executing hardware threads.

[0026] Processor core 200 further includes a memory management unit 204 (MMU) responsible for translating effective addresses determined by execution of memory-referencing instructions in execution unit 202 into real addresses within the real address space referenced by all processing units 104 in data processing system 100. MMU 204 performs effective-to-real address translations by referencing one or more translation structures 206, such as a translation lookaside buffer (TLB), an effective-to-real address translation (ERAT) cache, or a segment lookaside buffer (SLB). The number and / or type of these address translation structures may vary between implementations and architectures. Address translation structures 206 reduce latency associated with address translations by buffering local copies of selected address translations, which may be retrieved from system memory 108, as further described below.

[0027] The operation of each processor core 200 is supported by multiple levels of memory hierarchy, including at the lowest level a composite system memory provided by various system memories 108 and made accessible via memory controllers 106. The actual address ranges for which individual memory controllers 106 are responsible may be defined by the hypervisor and / or operating system software, for example, through appropriate configuration of one or more base address registers (BARs) 216 within the memory controllers 106. As shown, one or more of the system memories 108 store one or more system data structures (SDSs) 224 that provide bookkeeping for memory sharing between hosts as described herein. For example, the SDSs 224 may define various address ranges assigned to different memories and communication links of the data processing system 100, as described below. Additionally, one or more system memories 108 store a page frame table 210 including a plurality of page table entries (PTEs) 212, each PTE 212 specifying an effective address-to-real address translation for each corresponding memory page residing in one of the system memories 108. The PTEs 212 further specify access protection (e.g., read-only, read / write (R / W), etc.) for different memory pages. PTEs 212 accessed from the page frame table 210 by the MMU 204 may be cached by the MMU 204 for subsequent access, for example, in the address translation structure 206. The SDS 224 and page frame table 210 may be established, maintained, and updated, for example, by an operating system and / or hypervisor software running within the data processing system 100.

[0028] The multi-level memory hierarchy of each processor core 200 further includes one or more levels of cache memory, including, in the example embodiment, a private level one (L1) store-through cache 208 within each processor core 200 and a respective level two (L2) store-in cache 230 per processor core 200. While the cache hierarchy shown includes only two levels of cache, those skilled in the art will appreciate that alternative embodiments may include additional levels (L3, L4, etc.) of on-chip or off-chip cache, private or shared cache, in-line or indexed cache, and that the additional levels may fully, partially, or not encompass the contents of the cache above.

[0029] In the illustrated embodiment, each processing unit 104 further includes an integrated or distributed fabric controller 214 responsible for controlling the flow of operations on the system fabric in accordance with the host bus protocol and for performing the coherence communications required to implement the desired cache coherence protocol. Processing unit 104 may further include an integrated I / O (input / output) controller 218 that supports the attachment of one or more I / O devices and / or I / O channels (not shown).

[0030] In the illustrated example, processing unit 104 also includes an attached function unit or attached memory (AFUM) controller 220 that supports, in at least one operational mode, the attachment of an attached device (referred to herein as an attached function unit or attached memory (AFUM) 222) to host data processing system 100. Thus, AFUM 222 may be a memory-like device that simply responds to host commands received from host data processing system 100, or alternatively, may be a device that can issue AFUM commands (including AFUM read commands and AFUM write commands) to host data processing system 100. The real address range for which AFUM controller 220 is responsible when providing attached memory may be defined by a hypervisor and / or operating system software, for example, through appropriate configuration of one or more base address registers (BARs) 224 within AFUM controller 220. In at least some embodiments, the AFUM controller 220 may include an address translation cache (ATC) 226 that provides low latency storage for address translations between the real address space referenced by communications on the system fabric and the effective address space utilized by the attached AFUM 222 (which is preferably the same as the effective address space referenced by the processor core 200).

[0031] As shown, the AFUM 222 is coupled to the AFUM controller 220 via an AFUM interface 225. In some cases, the AFUM interface 225 may be integrated into or packaged with the AFUM 222. In other cases, the AFUM interface 225 may be implemented in a device or package separate from the AFUM 222. It should also be understood that multiple different types of AFUMs 222 may be implemented simultaneously within the host data processing system 100. For example, one or more AFUMs 222 may be attached functional units (AFUs) (e.g., accelerator chips), while one or more other AFUMs 222 may be attached memory devices.

[0032] 2, processing unit 104 further includes a nest memory management unit (NMMU) 228 that provides address translation, upon request, to other communication participants, such as AFUM controller 220. It should be understood that in other embodiments, NMMU 228 may be communicatively coupled, for example, by being coupled to system interconnect 110 rather than local interconnect 114, to provide address translation to communication participants, including AFUM controller 220, in an alternative or additional manner.

[0033] Referring now to FIG. 3, an exemplary protocol stack 300 for communicating information between a host data processing system (e.g., data processing system 100) and AFUM 222 is shown, according to one embodiment. The communication protocol implemented by protocol stack 300 defines the rules, syntax, semantics, and timing of communication between host data processing system 100 and AFUM 222. As is typical, protocol stack 300 includes multiple individual layers, each of which executes a specific subset of the communication protocol and communicates the results of its processing to adjacent layers. Each of the layers of protocol stack 300 may be implemented by hardware, software, or a combination of both hardware and software.

[0034] In the illustrated example, on the host side, protocol stack 300 includes host bus protocol layer 302, host bus protocol interface layer 304, host transaction layer 306, host transaction frame / parse layer 308, host data layer 310, and host physical layer 312, all of which may be implemented, for example, in AFUM controller 200. Host bus protocol layer 302 is configured to receive and issue commands (requests), responses, and control information over the system fabric of host data processing system 100 using the host bus protocol implemented by that host data processing system. Host bus protocol interface layer 304 provides an interface that converts information received from host bus protocol layer 302 into individual transactions that are received and processed by host transaction layer 306. The host bus protocol interface layer 304 similarly translates transactions received from the host transaction layer 306 into commands, responses, and control information in the host bus protocol implemented by the host bus protocol layer 302. The host transaction framing / parsing layer 308 packs and unpacks sequences of one or more transactions into frames that are processed by the host data layer 310. The host data layer 310 oversees the delivery of frames to and reception of frames from the AFUM interface 225 via the host physical layer 312. For example, the host data layer 310 can perform functions such as checking frame integrity, providing error correction, and replaying frames containing errors on the host physical layer 312.

[0035] The layers of the protocol interface 300 implemented within the AFUM interface 225, which correspond to the analogous protocol layers on the host side, include an AFUM protocol layer 324, an AFUM protocol interface layer 322, an AFUM transaction layer 320, an AFUM transaction frame / parse layer 318, an AFUM data layer 316, and an AFUM physical layer 314. The AFUM protocol layer 324 is configured to communicate commands (requests), responses, and control information to the AFUM 222 using the protocol implemented by the AFUM 222. The AFUM protocol interface layer 322 provides an interface that translates between the information communicated by the AFUM protocol layer 324 and the individual transactions processed by the AFUM transaction layer 320. The AFUM transaction frame / parse layer 318 packs and unpacks sequences of one or more transactions into frames that are processed by the AFUM data layer 316. The AFUM data layer 316 monitors the delivery of frames to and receipt of frames from the host via the AFUM physical layer 314, for example, by checking frame integrity, providing error correction, and reconstructing frames that contain errors.

[0036] In at least some embodiments, the layers of protocol stack 300, collectively designated by reference numeral 330, may be implemented in a conventional manner. To avoid unnecessarily obscuring the innovative concepts disclosed herein, the following description and figures omit protocol layer 330, and merely assume its existence.

[0037] 4, an exemplary protocol stack 400 for a communications interface configured to support the attachment of an additional memory device 402 to a host data processing system 100 is shown, according to one embodiment. As indicated by like reference numerals, the protocol stack 400 includes a host bus protocol layer 302, a host bus protocol interface layer 304, and a host transaction layer 306, which may be implemented within the AFUM controller 220 of the host data processing system 100, as previously described. The AFUM interface 225 similarly includes an AFUM protocol layer 324, an AFUM protocol interface layer 322, and an AFUM transaction layer 320, as previously described. The AFUM protocol layer 324 of the AFUM interface 225 is communicatively coupled to an AFUM implemented as additional memory 402.

[0038] In this configuration, host bus protocol interface layer 304 receives host bus protocol requests, such as host bus protocol read requests and write requests, over the system fabric of host data processing system 100. Each host bus protocol read and write request specifies at least the type of request and the real address to be accessed, and host bus protocol write requests further specify the data to be written to the specified real address. For each such host bus protocol request, host bus protocol interface layer 304 determines, by reference to BAR 224, whether the real address of the request falls within the real address range associated with attached memory 402. If not, the host bus protocol request is simply ignored. However, if the real address of the host bus protocol request falls within the real address range associated with attached memory 402, host bus protocol interface layer 304 forwards the host bus protocol request to host transaction layer 306. The host transaction layer 306 then translates the host bus protocol request into an appropriate corresponding host command, which, in the case of a host command targeting the attached memory 402, is either a host read command specifying a real address (e.g., Host_Rd(Addr)) or a host write command specifying a real address and data to be written to the attached memory 402 (e.g., Host_Wr(Addr,Data)).

[0039] In response to a host read command, the AFUM interface 225 processes the host command via the protocol layers 320-324 and responds with one of two responses: an AFUM read response (e.g., AFUM_Rd_Resp(Data)) indicating success and providing the requested data, or an AFUM read response (e.g., AFUM_Rd_Failed) indicating failure of the read request. The AFUM interface 225 similarly processes host write commands and responds to the host write command by providing one of two responses: an AFUM write response (e.g., AFUM_Wr_Resp) indicating success of the host write command in updating the additional memory 402, or an AFUM write response (e.g., AFUM_Wr_Failed) indicating failure of the write command to update the additional memory 402. The AFUM controller 220 processes AFUM responses received from the AFUM interface 225 within the host transaction layer 306 and host bus protocol interface layer 304 and, if requested or allowed by the host bus protocol, issues a host bus protocol response generated based on the AFUM response on the system fabric of the host data processing system 100. However, it should be understood that some host bus protocols may not require or support a host bus protocol response for an AFUM write response.

[0040] 5, an exemplary protocol stack 500 for a communications interface configured to support attachment of an add-on functional unit (AFU) 502 to host data processing system 100 is shown, according to one embodiment. As indicated by like reference numerals, protocol stack 500 includes host bus protocol layer 302, host bus protocol interface layer 304, and host transaction layer 306, which may be implemented within AFUM controller 220 of host data processing system 100, as previously described. AFUM interface 225 similarly includes AFUM protocol layer 324, AFUM protocol interface layer 322, and AFUM transaction layer 320, as previously described. AFUM protocol layer 324 of AFUM interface 225 is communicatively coupled to an AFUM, such as an accelerator implemented as AFU 502.

[0041] AFU 502 may initiate and issue AFUM commands directed to host data processing system 100. These AFUM commands received by AFUM protocol layer 304 each specify at least a command type and an effective address to be accessed, and may further specify data to be written to the specified effective address. Each such AFUM command is processed (and possibly translated) via AFUM protocol interface layer 322 and AFUM transaction layer 320 to issue one of a set of AFUM commands, including at least an AFUM read command (e.g., AFUM_Rd(Addr)) and an AFUM write command (e.g., AFUM_Wr(Addr,Data)).

[0042] In response to the AFUM command, host transaction layer 306 of AFUM controller 220 determines whether ATC 226 holds an address translation entry that can translate the effective address specified by the AFUM command into a real address within the real address space of host data processing system 100. Based on the success of this effective address-to-real address translation and, if successful, the host data processing system 100's response to the host bus protocol request generated by protocol layers 302-306 from the AFUM command, protocol layers 302-306 provide an appropriate host response that is communicated back to AFU 502 via protocol layers 320-324 of AFUM interface 225.

[0043] For an AFUM read command, the host data processing system 100 can provide one of three host responses: (1) a host read response indicating successful completion of the read without error and providing the requested data (e.g., Host_Rd_Resp(Data,NErr)), (2) a host read response indicating completion of the read and providing data that contains an error (e.g., Host_Rd_Resp(Data,Err)), or (3) a host read response indicating an early failure to find an effective address-to-real address translation for the effective address specified by the read / write command (e.g., Host_Rd_Failed_XLATE). Host data processing system 100 similarly responds to an AFUM write command by providing one of three host responses: (1) a host write response indicating successful completion of the write without error (e.g., Host_Wr_Resp(NErr)), (2) a host write response indicating completion of the write with an error (e.g., Host_Wr_Resp(Err)), or (3) a host write response indicating initial failure to find an effective address-to-real address translation for the effective address specified by the AFUM write command (e.g., Host_Wr_Failed_XLATE). In addition to these six host responses, host data processing system 100 is further configured to provide two additional responses (e.g., Host_XLATE_Complete and Host_XLATE_Err) that indicate eventual success or eventual failure, respectively, in finding a translation for the effective address specified in the AFUM command.

[0044] It should be understood that protocol stacks 300, 400, 500 may support additional commands and responses other than the read and write commands and responses specifically detailed in Figures 4-5. For example, protocol stacks 300-500 may support communications that facilitate other memory access commands (e.g., atomic memory operations (AMOs)), request interrupts, and support flow control through implementation of credit and / or virtual channel maintenance.

[0045] 6-10, there is shown a time-space diagram of an AFU read command of an AFU 502 and various possible host responses to the AFU read command, according to one embodiment.

[0046] As shown in FIG. 6, AFU 502 issues an AFUM read command 600 (e.g., AFUM_Rd(Addr)) to host data processing system 100 via protocol stack 500 as described above with reference to FIG. 5. In the successful case shown in FIG. 7, AFUM controller 220 of host data processing system 100 successfully obtains an address translation for the effective address specified in AFUM read command 600, translates the effective address to a real address by reference to an address translation entry, and initiates a host bus protocol read request on the system fabric of host data processing system 100 specifying the real address obtained from the address translation. In response to the host bus protocol read request, the data requested by the host bus protocol read request may be returned to AFUM controller 220, for example, by a cache memory (e.g., one of L2 caches 230) or memory controller 106. In response to receiving the requested data, the AFUM controller 220 provides the AFU 502 with an error-free host read response 700 (e.g., Host_Rd_Resp(Data, NErr)) that provides the requested data.

[0047] 8 illustrates a scenario in which a read error occurs. In this case, the AFUM controller 220 of the host data processing system 100 successfully obtains an address translation for the effective address specified in the AFUM read command 600, translates the effective address to a real address by referencing an address translation entry, and initiates a host bus protocol read request on the system fabric of the host data processing system 100 specifying the real address obtained from the address translation. However, in this case, the host bus protocol read request fails in the host data processing system 100 due to, for example, a parity or ECC (error correcting code) error. In response to the read error indication, the AFUM controller 220 issues a host read response 800 (e.g., Host_Rd_Resp(Data,Err)) indicating the error. In some implementations, the error indication may be provided within the data field of the host read response.

[0048] 9 illustrates a case in which an AFUM read command 600 fails due to the AFUM controller 220 being unable to obtain an address translation entry for the effective address specified by the AFUM read command 600, but the required address translation entry is subsequently provided by operating system software. In this case, the AFUM controller 220 of the host data processing system 100 receives the AFUM read command 600 and attempts to obtain, but fails to obtain, the address translation entry for the effective address specified in the AFUM read command 600. In response to the failure to obtain the address translation entry for the effective address, the AFUM controller 220 issues a host read failure response 900 (e.g., Host_Rd_Failed_XLATE) to the AFU 502 indicating that the failure was due to an inability to obtain an address translation entry for the effective address. In addition, the AFUM controller 220 initiates an operating system interrupt requesting the required address translation. In response to the operating system indicating that the address translation in page frame table 210 is available, AFUM controller 220 issues a host translation complete response 902 (e.g., Host_XLATE_Complete) to AFUM 222 indicating that the address translation entry for the effective address of AFUM read request 600 is now available. In response to receiving host translation complete response 902, AFU 222 may optionally reissue the AFUM read command as an AFUM read command 904 (e.g., AFUM_Rd(Addr)), which may then succeed (assuming the address translation entry is still valid in page frame table 210 when AFUM read command 904 is issued), as shown in FIG.

[0049] 10 illustrates a case in which an AFUM read command 600 fails due to the AFUM controller 220 being unable to obtain an address translation entry for the effective address specified by the AFUM read command 600, and the required address translation is subsequently not provided by the operating system. In this case, the AFUM controller 220 of the host data processing system 100 receives the AFUM read command 600 and attempts to obtain, but fails to obtain, the address translation entry for the effective address specified in the AFUM read command 600. In response to the failure to obtain the address translation entry for the effective address, the AFUM controller 220 issues a host read failure response 1000 (e.g., Host_Rd_Failed_XLATE) to the AFU 502 indicating that the failure was due to an inability to obtain an address translation entry for the effective address. In addition, the AFUM controller 220 initiates an operating system interrupt requesting the required address translation. In response to the operating system indicating that address translation is not available, the AFUM controller 220 issues a host translation error response 1002 (e.g., Host_XLATE_Err) to the AFUM 222 indicating that an address translation entry was not found for the effective address of the AFUM read request 600.

[0050] 11-15, there are shown time-space diagrams of AFU 502 write commands and various responses by a host data processing system to AFU 502 write commands, according to one embodiment.

[0051] As shown in FIG. 11 , AFU 502 issues an AFUM write command 1100 (e.g., AFUM_Wr(Addr,Data)) to host data processing system 100 via protocol stack 500 as described above with reference to FIG. 5 . In the successful example shown in FIG. 12 , AFUM controller 220 of host data processing system 100 successfully obtains address translation and write permission for the effective address specified in AFUM write command 1100. After the required address translation and write permission are obtained, AFUM controller 220 translates the effective address to a real address by reference to an address translation entry and initiates a host bus protocol write request on the system fabric of host data processing system 100. The host bus protocol write request specifies the real address obtained from the address translation and includes or is accompanied by the data payload of AFUM write command 1100. In response to a successful host bus protocol write request updating system memory 108 (or another memory-mapped resource, such as additional memory 402), AFUM controller 220 provides a host write response 1200 (e.g., Host_Wr_Resp(NErr)) to AFU 502 indicating no error.

[0052] 13 illustrates a case in which a write failure occurs. In this case, the AFUM controller 220 of the host data processing system 100 responds to the AFUM write command 1100 by obtaining an address translation entry and write permission for the effective address specified in the AFUM write command 1100. After the required address translation and write permission are obtained, the AFUM controller 220 translates the effective address to a real address by reference to the address translation entry and initiates a host bus protocol write request on the system fabric of the host data processing system 100. Again, the host bus protocol write request specifies the real address obtained from the address translation and includes or accompanies the data payload of the AFUM write command 1100. However, in this case, the host bus protocol write request fails at the host data processing system 100 due to, for example, a parity or ECC error. In response to a message on the system fabric indicating a failure of the host bus protocol write request, the AFUM controller 220 issues a host write response 1300 (eg, Host_Wr_Resp(Err)) indicating a write error.

[0053] 14 illustrates a case in which an AFUM write command 1100 fails due to the AFUM controller 220 being unable to obtain an address translation entry and write permission for the effective address specified by the AFUM write command 1100, but the required address translation entry and write permission are subsequently provided by the operating system. In this case, the AFUM controller 220 of the host data processing system 100 receives the AFUM write command 1100 and attempts, but fails, to obtain an address translation entry and write permission for the effective address specified in the AFUM write command 1100. In response to the failure to obtain an address translation entry and write permission for the effective address, the AFUM controller 220 issues a host write failure response 1400 (e.g., Host_Wr_Failed_XLATE) to the AFU 502 indicating that the failure was due to an inability to obtain an address translation entry and write permission for the effective address. In addition, the AFUM controller 220 initiates an operating system interrupt requesting the required address translation and write permission. In response to the operating system indicating that the address translation and write permission in page frame table 210 are available, AFUM controller 220 issues host translation complete response 1402 (e.g., Host_XLATE_Complete) to AFUM 222 indicating that the address translation entry for the effective address of AFUM write command 1100 is now available. In response to receiving host translation complete response 1402, AFU 222 may optionally reissue the AFUM write command as AFUM write command 1404 (e.g., AFUM_Wr(Addr)), which may then succeed (assuming that the address translation entry and write permission are still valid in page frame table 210 when AFUM write command 1404 is issued), as shown in FIG.

[0054] 15 illustrates a case in which an AFUM write command 1100 fails due to the AFUM controller 220 being unable to obtain an address translation entry for the effective address specified by the AFUM write command 1100, and subsequently, the address translation entry and write permission are not provided by the operating system. In this case, the AFUM controller 220 of the host data processing system 100 receives the AFUM write command 1100 and attempts, but fails, to obtain an address translation entry and write permission for the effective address specified in the AFUM write command 1100. In response to the failure to obtain the address translation entry and write permission for the effective address, the AFUM controller 220 issues a host write failure response 1400 (e.g., Host_Wr_Failed_XLATE) to the AFU 502 indicating that the failure was due to an inability to obtain an address translation entry and write permission for the effective address. In addition, the AFUM controller 220 initiates an operating system interrupt requesting the required address translation and write permission. In response to the operating system indicating that the requested address translation entry and / or write permission is not available, the AFUM controller 220 issues a host translation error response 1502 (e.g., Host_XLATE_Err) to the AFUM 222 indicating that the address translation entry and / or write permission for the effective address of the AFUM write command 1100 was not found.

[0055] 16, a high-level logic flowchart of an exemplary method by which host transaction layer 306 of host data processing system 100 responds to commands from AFU 502 is shown, according to one embodiment. The process of FIG. 16 begins at block 1600 and proceeds to block 1602, which illustrates host transaction layer 306 receiving either an AFUM read command 600 (see, e.g., FIG. 6) or an AFUM write command 1100 (see, e.g., FIG. 11) from AFU 502. In response to receiving the AFUM command, host transaction layer 306 determines, at block 1604, whether to skip translation of the effective address specified in the AFUM command. The host transaction layer 306 may determine to skip translation of the effective address based on, for example, that the effective address of the AFUM command 600 or 1100 falls within a predetermined address range, an instruction provided in the AFUM command 600 or 1100, or the configuration of the AFUM controller 220, or a combination thereof. In response to a determination at block 1604 to skip translation of the effective address specified in the AFUM command, the process proceeds to block 1636, described below. If not, the process proceeds to block 1606.

[0056] Block 1606 illustrates host transaction layer 306 determining whether ATC 226 holds an address translation entry for translating the effective address specified in AFUM command 600 or 1100 to a real address. In response to host transaction layer 306 determining that ATC 226 holds an associated address translation entry, the process proceeds to block 1630 and subsequent blocks. On the other hand, if host transaction layer 306 determines that ATC 226 does not hold an associated address translation entry, the process proceeds from block 1606 to block 1610 and subsequent blocks.

[0057] Referring now to block 1610, the host transaction layer 306 initiates sending an NMMU translation request for the effective address of the AFUM command to the NMMU 228 via the host bus protocol interface layer 304 and the host bus protocol layer 302 (block 1610). The host transaction layer 306 then waits to receive a host bus protocol response to the NMMU translation request (block 1612). In response to receiving the host bus protocol response to the NMMU translation request indicating success and providing the requested address translation entry, the host transaction layer 306 installs the address translation entry in the ATC 226 (block 1614). The process then proceeds to block 1630, described below.

[0058] In response to a host bus protocol response to the NMMU translation request that does not indicate success (i.e., no address translation entry is returned by the NMMU 228), the process proceeds from block 1612 to block 1616, which depicts the host transaction layer 306 issuing a host read failed response 900, 1000 (e.g., Host_Rd_Failed_XLATE) or a host write failed response 1400, 1500 (e.g., Host_Wr_Failed_XLATE) to the AFU 502. In addition, the host transaction layer 306 initiates sending an interrupt request to the operating system or hypervisor running on the processor core 200 to the processing unit 102 via the host bus protocol interface layer 304 and the host bus protocol layer 302. This interrupt request requests address translation and any required access permissions for the effective address specified by the AFUM command (block 1618).

[0059] Block 1620 indicates that the host transaction layer 306 monitors to determine whether a successful response to the interrupt request has been received from the operating system or hypervisor (often this response is provided by a memory-mapped input / output (MMIO) operation). In response to receiving a success indication, the host transaction layer 306 issues a host translation completion response 902, 1402 (e.g., Host_XLATE_Complete) to the AFU 502 (block 1622). However, in response to receiving a failure indication, the host transaction layer 306 issues a host translation error response 1002, 1502 (e.g., Host_XLATE_Err) to the AFU 502 (block 1624). Following either block 1622 or block 1624, the process of FIG. 16 ends at block 1640.

[0060] Referring now to block 1630, the host transaction layer 306 determines whether the AFUM command received in block 1602 is an AFUM write command. If it is not an AFUM write command, the process proceeds to block 1636, described below. However, if the host transaction layer 306 determines that the AFUM command received in block 1602 is an AFUM write command, then in block 1632 the host transaction layer 306 determines whether the address translation entry accessed from the ATC 226 in block 1606 or obtained from the NMMU 228 in blocks 1612-1614 includes write permission for the real address referenced by the AFUM write command. If it does include write permission, the process proceeds to block 1636, described below. However, if, at block 1632, the host transaction layer 306 determines that the address translation entry does not provide write permission for the real address referenced by the AFUM write command, the host transaction layer 306 removes the address translation entry from the ATC 226 (block 1634). The process then proceeds to block 1616 as described.

[0061] Referring to block 1636, host transaction layer 306 initiates processing of the read or write operation specified by the AFUM command within host data processing system 100 by causing the issuance of an appropriate host bus protocol request on the system fabric via host bus protocol interface layer 304 and host bus protocol layer 302. If a decision is made in block 1604 to skip address translation for the effective address of the AFUM command, it should be understood that other techniques, not shown, including appropriate configuration of page frame table 210, are utilized to ensure that the effective address is within a permitted address range and has the required write permission. Depending on the success or failure of the host bus protocol request initiated in block 1636, host transaction layer 306 issues an appropriate host response to AFU 502 (block 1638). Specifically, in response to an AFUM read request, host transaction layer 306 provides the requested data and issues either a host read response 700 indicating no error (e.g., Host_Rd_Resp(Data, NErr)) or a host read response 800 indicating a read error (e.g., Host_Rd_Resp(Data, Err)). In response to an AFUM write command, host transaction layer 306 issues either a host write response 1200 indicating no error (e.g., Host_Wr_Resp(NErr)) or a host write response 1300 indicating a write error (e.g., Host_Wr_Resp(Err)). Following block 1638, the process ends at block 1640.

[0062] 17, a high-level logic flowchart of an exemplary method for AFU 502 to issue commands to host data processing system 100 and process host responses is shown, according to one embodiment. The process of FIG. 17 begins at block 1700 and proceeds to block 1702, which illustrates AFUM transaction layer 320 issuing an AFUM command, such as an AFUM read command 600 (e.g., AFUM_Rd(Addr)) or an AFUM write command 1100 (e.g., AFUM_Wr(Addr,Data)), to host transaction layer 306. AFUM transaction layer 320 then monitors responses to the AFUM command, as shown in blocks 1704-1706. In response to determining at block 1704 that the host response is a host read response 700 (e.g., Host_Rd_Resp(Data, NErr)) that provides the data requested by the AFUM read command 600 and indicates no error, or a host write response 1200 (e.g., Host_Wr_Resp(NErr)) that indicates no error, the process proceeds to block 1710. As shown in block 1710, if the AFUM command issued at block 1702 was an AFUM write command 1100 rather than an AFUM read command 600, then processing by AFUM transaction layer 320 simply terminates at block 1726. However, if the AFUM command issued at block 1702 was an AFUM read command 600 rather than an AFUM write command 1100, then AFUM transaction layer 320 returns the requested data to AFU 502 (block 1712). The process of FIG. 17 then terminates at block 1726.

[0063] Referring again to block 1704, if the received host response is neither a host read response 700 nor a host write response 1200, then at block 1706, the AFUM transaction layer 320 determines whether the host response indicates an initial translation failure (e.g., Host_Rd_Failed_XLATE 900, 1000 or Host_Wr_Failed_XLATE 1400, 1500). If not, the process returns to block 1704. However, if the AFUM transaction layer 320 determines that the host response indicates an initial translation failure, then at block 1720, the AFUM transaction layer 320 repeats to monitor for additional host translation responses indicating whether the address translation of the AFUM command's effective address was successfully loaded into the page frame table 210. In response to receiving a Host_XLATE_Complete response 902, 1402 indicating that the address translation for the AFUM command's effective address was successfully loaded into the page frame table 210, the process of FIG. 17 proceeds from block 1720 to block 1722 and then from block 1722 back to block 1702. Following this path, the process indicates that the AFU 502 can reissue the AFUM command, as indicated by reference numerals 904, 1404. However, if the AFUM transaction layer 320 instead receives a Host_XLATE_Err response 1002, 1502 indicating that the translation for the AFUM command's effective address was not loaded into the page frame table 210, the process proceeds from block 1722 to block 1724, which indicates that the AFUM transaction layer 320 begins error processing for the failed AFUM command. Following block 1724, the process of FIG. 17 ends at block 1726.

[0064] 18-20, there is shown a time-space diagram of a host read command issued by the host data processing system 100 to the additional memory 402 and various responses by the additional memory 402 to the host read command of the host data processing system 100, according to one embodiment.

[0065] As shown in FIG. 18 , the AFUM controller 220 of the host data processing system 100 issues a host read command 1800 (e.g., Host_Rd(Addr)) to the AFUM 222, configured or implemented as an attached memory 402, via the protocol stack 400 as described above with reference to FIG. 4 . In the successful example shown in FIG. 19 , the AFUM transaction layer 320 communicates a read command corresponding to the host read command 1800 to the attached memory 402 via the AFUM protocol interface layer 322 and the AFUM protocol layer 324, all of which may be implemented in the AFUM interface 225. In response to the read command, the attached memory 402 returns the data specified by the real address in the host read command 1800. The AFUM protocol interface layer 322 forwards the requested data to the host transaction layer 306 in an AFUM read response 1900 (e.g., AFUM_Rd_Resp(Data)).

[0066] 20 illustrates a case in which a read failure occurs. In this case, the additional memory 402 responds to the read command received from the AFUM interface 225 with a message indicating the failure of the read command. In response, the AFUM interface 225 issues an AFUM read response 2000 indicating the read failure (e.g., AFUM_Rd_Failed). In an alternative implementation, an indication of the read failure may be provided within the data field of the AFUM read response 1900.

[0067] 21-23, there is shown a time-space diagram of a write command issued by the host data processing system 100 to an AFUM 222 configured as additional memory 402, and various responses by the additional memory 402 to the write command, according to one embodiment.

[0068] As shown in Figure 21, the AFUM controller 220 of the host data processing system 100 issues a host write command 2100 (e.g., Host_Wr(Addr,Data)) to the AFUM 222, configured as an attached memory 402, via the protocol stack 400 as described above with reference to Figure 4. In the successful example shown in Figure 22, the AFUM transaction layer 320 communicates a write command corresponding to the host write command 1800 (including the actual address and write data) to the attached memory 402 via the AFUM protocol interface layer 322 and the AFUM protocol layer 324, all of which may be implemented in the AFUM interface 225. In response to the write command, the attached memory 402 returns an indication of the success of the write command to the AFUM protocol interface layer 322. In response, the AFUM protocol interface layer 322 issues an AFUM write response 2200 (eg, AFUM_Wr_Resp) to the host transaction layer 306 indicating success of the host write command 2100 .

[0069] 23 illustrates a case in which a write failure occurs. In this case, the attached memory 402 responds to the write command received from the AFUM interface 225 with a message indicating the failure of the write command. In response, the AFUM interface 225 issues an AFUM write response 2300 indicating the write failure (e.g., AFUM_Wr_Failed).

[0070] 24, a high-level logic flowchart of an exemplary method by which the AFUM transaction layer 320, which provides the AFUM 222, responds to commands from the host data processing system 100 is shown, according to one embodiment. The process of FIG. 24 begins at block 2400 and proceeds to block 2402, which illustrates the AFUM transaction layer 320 receiving either a host read command 1800 (e.g., see FIG. 18) or a host write command 2100 (e.g., see FIG. 21) from the AFUM controller 220. In response to receiving the host command, the AFUM transaction layer 320 begins processing the read or write operation specified by the host command in the attached memory 402 (block 2404). Depending on the success or failure of the read or write operation initiated in block 2404, the AFUM transaction layer 320 issues an appropriate AFUM response to the AFUM controller 220 (block 2406). Specifically, in response to a successful host command, the AFUM transaction layer 320 issues either an AFUM read response 1900 (e.g., AFUM_Rd_Resp(Data)) providing the requested data or an AFUM write response 2200 (e.g., AFUM_Wr_Resp) indicating a successful write, as shown in block 2410. In response to a failed host command, the AFUM transaction layer 320 issues either an AFUM read response 2000 (e.g., AFUM_Rd_Failed) indicating a failed read or an AFUM write response 2300 (e.g., AFUM_Wr_Failed) indicating a failed write, as shown in block 2408. Following either block 2408 or block 2410, the process of FIG. 24 ends at block 2412.

[0071] 25, a high-level logic flowchart of an exemplary method by which the host transaction layer 306 processes commands issued to the AFUM 222 configured as attached memory 402 is shown, according to one embodiment. The process of FIG. 25 begins at block 2500 and proceeds to block 2502, which illustrates the host transaction layer 306 issuing a host command, such as a host read command 1800 (e.g., Host_Rd(Addr)) or a host write command 2100 (e.g., Host_Wr(Addr,Data)), to the AFUM transaction layer 320. The host transaction layer 306 then monitors the AFUM response to the host command, as shown in blocks 2504-2506. In response to determining at block 2504 that the AFUM response is either an AFUM read response 1900 (e.g., AFUM_Rd_Resp(Data)) that provides the data requested by the host read command 900, or an AFUM write response 2200 (e.g., Host_Wr_Resp) that indicates success of the host write command 2100, the process proceeds to block 2510. As shown at block 2510, if the host command issued at block 2502 was a host write command 2100 rather than a host read command 1800, then processing by the host transaction layer 306 simply terminates at block 2516. However, if the host command issued at block 2502 was a host read command 1800 rather than a host write command 2100, then the host transaction layer 306 returns the requested data to the host data processing system 100 (block 2512). The process of FIG. 25 then terminates at block 2516.

[0072] Referring again to block 2504, if the received AFUM response is neither an AFUM read response 1900 nor an AFUM write response 2200, then at block 2506, the host transaction layer 306 determines whether the AFUM response indicates a read or write failure (e.g., AFUM_Rd_Failed 2000 or AFUM_Wr_Failed 2300). If not, the process returns to block 2504. However, if the host transaction layer 306 determines that the AFUM response indicates a read or write failure, then the host transaction layer 306 begins error handling for the failed host command (block 2514). Following block 2514, at block 2516, the process of FIG. 25 ends.

[0073] The foregoing description details a multi-function communication interface that may be utilized to support communication between host data processing system 100 and AFUM 222 configured as additional memory 402 or AFU 502. It should be understood that the communication interface supports simultaneous communication of both host commands initiated by host data processing system 100 and AFUM commands initiated by AFUM 222. Thus, the communication interface provides a convenient function for enabling communication between host processing system 100 and additional devices, thereby relieving the additional devices from the requirement to support the host bus protocol and / or coherence protocol of host data processing system 100 while still enabling data sharing between the additional devices and host data processing system 100. In accordance with one aspect of the invention disclosed herein, the described communication interface may be extended to allow the attachment of more than one host data processing system for communication, for example, to allow memory sharing between the host data processing systems. In the context of this application, each such "host" is understood to mean, at a minimum, a set of processing and memory resources that form a coherent memory region where the processing resources have read and write access to the memory resources without using a communication interface as disclosed herein. By definition, because the sharing of memory resources by hosts via a communication interface is non-coherent, higher-level software (e.g., operating system or hypervisor software) can optionally determine whether access to overlapping memory regions by different hosts should be restricted, for example, by memory page protection.

[0074] Referring now to FIG. 26, an exemplary protocol stack 2600 for a communications interface configured to support attachment of host data processing systems to support memory sharing is shown, according to one embodiment. In the illustrated embodiment, a first host data processing system 100 (identified as “Host A”) is communicatively coupled to a different second host data processing system 100 (identified as “Host B”). As described further below, for example, with reference to FIG. 38, the communicative coupling of Host A and Host B may be implemented by directly coupling the AFUM controller 220 in Host A to the AFUM controller 220 in Host B via a communications link. Alternatively or additionally, the AFUM controller 220 in Host A and the AFUM controller 220 in Host B may be communicatively coupled via one or more intervening devices, such as other host data processing systems, switches, communications links, etc.

[0075] As indicated by like reference numerals, the portions of the protocol stack 2600 implemented in each of Host A and Host B include the host bus protocol layer 302a or 302b, the host bus protocol interface layer 304a or 304b, and the host transaction layer 306a or 306b, as previously described. Again, these protocol layers (as well as the host transaction framing / parsing layers 308a-308b, the host data layers 310a-310b, and the host physical layers 312a-312b, which are not shown) may be implemented within the AFUM controller 220. Additionally, each of Host A and Host B includes a command, control, and credit, and response translation (CCR XLATE) layer 2602a or 2602b. From the perspective of host A, CCR XLATE layer 2602b translates inbound host commands initiated by host A into AFUM commands, thus emulating the initiating host (i.e., host A) being the AFU 502 or making the initiating host appear to the receiving host (i.e., host B) to be the AFU 502. Similarly, CCR XLATE layer 2602a translates the receiving host's (i.e., host B) inbound host responses issued in response to AFUM commands output by CCR XLATE layer 2602b into AFUM responses, thus emulating the receiving host (i.e., host B) being the attached memory 402 or making the receiving host appear to the initiating host (i.e., host A) to be the attached memory 402. For host commands initiated by host B, the functions performed by CCR XLATE layers 2602a, 2602b are reversed. Thus, for these host commands, the CCR XLATE layer 2602a translates the host command for Host B into an AFUM command (which emulates the attachment of an AFU 502 to Host A), and the CCR XLATE layer 2602b translates the host response for Host A into an AFUM response (which emulates the attachment of additional memory to Host B).Therefore, despite the asymmetry between the set of AFUM commands and responses and the set of host commands and responses, communication between hosts can be handled seamlessly by reusing the existing AFUM communication protocol.

[0076] As alluded to above, the set of commands and responses implemented by the host and AFUM 222 are asymmetric. For example, comparing Figures 9-10 and 14-15 with Figures 19-20 and 23-23, it can be observed that the set of host responses is richer than the set of AFUM responses, particularly in that the set of host responses includes host read and write failure responses 900, 1400, host translation completion responses 902, 1402, and host translation error responses 1002, 1502. The set of AFUM responses simply does not include these or the corresponding response messages. As a result, the set of AFUM responses reused in host-to-host communications cannot convey the occurrence of translation errors or page protection failures. Therefore, in a preferred embodiment, the possibility of address translation errors and page protection failures is eliminated by fixing the page table entries referenced by the initiating host's host commands in the target host's page frame table 210.

[0077] Referring now to FIG. 27, a more detailed block diagram of the CCR XLATE layer 2602 within the protocol stack 2600 of FIG. 26 is shown. In this embodiment, the CCR XLATE layer 2602, which may be implemented in hardware, software, or a combination of hardware and software, includes a layer input 2700 that receives host commands, host responses, and host control / credit messages originating from another host data processing system 100. As previously described, host commands are utilized to, among other things, initiate read and write accesses to shared memory, and host responses provide responses to AFUM commands. The host control / credit messages are utilized, for example, to regulate the flow control of commands and responses between hosts, allocate and release credits employed in credit-based allocation of communication links, and implement virtual channels. The layer input 2700 is coupled via a bypass path 2702 to a first set of inputs 2704 of selection logic, represented in FIG. 27 by a mode multiplexer 2706. The layer input 2700 is further coupled to the transformation logic 2710 .

[0078] In the illustrated embodiment, the translation logic 2710 includes command translation (Command XLATE) logic 2720, which translates host commands of an initiating host coupled to the AFUM controller 220 into AFUM commands, as described in detail below with reference to Figures 28-33. In a preferred embodiment, the translation logic 2710 is implemented using only combinatorial logic capable of operating at wire speed. Associated with the command translation logic 2720 is optional address translation (Addr XLATE) logic 2722, which can translate the actual address specified by the host command. In a particularly preferred embodiment, if the address translation logic 2722 is present, the address translation performed by the address translation logic 2722 is performed without reference to an address translation structure (e.g., an address translation cache or translation lookaside buffer), but instead is implemented directly utilizing combinatorial logic that may, for example, add or subtract an address offset. The translation logic 2710 further includes response translation logic 2724 that translates host responses of a receiving host coupled to the AFUM controller 220 into AFUM responses, as described in detail below with reference to Figures 28-33. Additionally, the translation logic 2710 includes control and credit translation logic 2726 that translates host control and credit messages into AFUM control and credit messages. The outputs of the command translation logic 2720, the response translation logic 2724, and the control and credit logic 2726 are received at a second set 2708 of inputs to the mode multiplexer 2706.

[0079] The mode multiplexer 2706 selects between the messages presented to the first set of inputs 2704 and the second set of inputs 2708 based on the setting of a configuration register 2712, which may be initialized by a memory-mapped I / O (MMIO) operation of the host data processing system 100 to configure the CCR XLATE layer 2602, for example. For example, in one example, when an AFUM controller 220 is utilized to communicatively couple the host data processing system 100 to the AFUM 222, the setting of the configuration register 2712 controls the mode multiplexer to select the messages presented to the first set of inputs 2704 for forwarding to the associated host transaction layer 306 via the layer output 2714. However, when the AFUM 220 is utilized to couple the host data processing system 100 for memory sharing, the setting of the configuration register 2712 controls the mode multiplexer to select messages presented to the second set of inputs 2708 for forwarding via the layer output 2714.

[0080] 28-30, there is shown a time-space diagram of a read command issued by a first host data processing system (e.g., Host A) to an additional host data processing system (e.g., Host B) via a communication interface provided by the coupled AFUM controller 220, and various responses of the additional host data processing system to the read command, according to one embodiment.

[0081] 28, the AFUM controller 220 of Host A (i.e., the "initiating host") issues a host read command 2800 (e.g., Host_Rd(Addr)) specifying a real address from which data is to be read to the AFUM controller 220 of Host B (i.e., the "receiving host" or "target host"), and the AFUM controllers 220 of both hosts are configured by setting configuration registers 2712 for communication between the hosts via protocol stack 2600. In response to receiving the host read command 2800, the CCR XLATE layer 2602b of Host B translates the host read command 2800 and, optionally, the real address specified by the host read command 2800. For example, the command translation logic 2720 translates the host read command 2800 into an AFUM read command 2802 (AFUM_Rd(TAddr)), emulating, or making it appear to the receiving Host B, that the AFUM read command 2802 was initiated by an AFU 502 attached directly to the AFUM controller 220 of Host B. Optionally, the address translation logic 2722 of Host B may translate the actual address specified by the host read command 2800 and obtain the translated address (TAddr). The CCR XLATE layer 2602b then passes this AFUM read command 2802 to the host transaction layer 306b of Host B, which processes the AFUM read command 2802 as described above with reference to FIG. 16 .

[0082] 29, in response to the host bus protocol read request initiated by the host transaction layer 306b in block 1636, Host B returns the data requested by the AFUM read command 2802 to the AFUM controller 220. Host B's host bus protocol interface layer 304b forwards the requested data to Host B's host transaction layer 306b, which issues a host read response 2900 (e.g., Host_Rd_Resp(Data, NErr)) providing the requested data and indicating no errors.

[0083] The host read response 2900 is received and processed by the CCR XLATE layer 2602a of host A. In response to receiving the host read response 2900, the response translation logic 2724 of the CCR XLATE layer 2602a translates the host read response 2900 into an AFUM read response 2902 (e.g., AFUM_Rd_Resp(Data)) to emulate, or appear from host A's perspective, that host A has attached additional memory 402. It should be noted that the no error indication (NErr) provided in the host read response 2900 causes the response translation logic 2724 to provide AFUM_Rd_Resp(Data) rather than AFUM_Rd_Failed. CCR XLATE layer 2602a forwards AFUM read response 2902 to host bus protocol interface 304a, which processes AFUM read response 2902 as described above with reference to FIG. 25. For example, host bus protocol interface 304a may communicate the requested data to a requesting master (e.g., L2 cache 230) within host A using the host bus protocol of host A. The requesting master may then cache or otherwise process the requested data.

[0084] 30 illustrates a case in which a read failure occurs at Host B. In this case, in response to the host bus protocol read request initiated by Host Transaction Layer 306b in block 1636, Host B responds to Host Bus Protocol Interface Layer 304b of Host B with a message indicating a failure of the host bus protocol read request. In response, Host B's Host Bus Protocol Interface Layer 304b issues a host read response 3000 (e.g., Host_Rd_Resp(Data,Err)) indicating the read failure. In some implementations, an indication of the read failure may be provided within the data field of Host Read Response 3000.

[0085] The host read response 3000 is received and processed by the CCR XLATE layer 2602a of host A. In response to receiving the host read response 3000, the response translation logic 2724 of the CCR XLATE layer 2602a translates the host read response 3000 into an AFUM read response 3002 indicating failure (e.g., AFUM_Rd_Failed). Again, it should be noted that the (Err) indication provided in the host read response 3000 causes the response translation logic 2724 to provide AFUM_Rd_Failed rather than AFUM_Rd_Resp(Data). The CCR XLATE layer 2602a forwards the AFUM read response 3002 to the host bus protocol interface 304a, which initiates error processing as described above with reference to block 2514 of FIG. 25 .

[0086] 31-33, there is shown a time-space diagram of a write command issued by a first host data processing system (e.g., Host A) to an additional host data processing system (e.g., Host B) via a communication interface provided by a coupled AFUM controller 220, and the additional host data processing system's various responses to the write command, according to one embodiment. As previously described, the AFUM controllers 220 of both hosts are configured by setting configuration registers 2712 for memory sharing between the hosts via protocol stack 2600.

[0087] 31 , the AFUM controller 220 of the initiating host A issues a host write command 3100 (e.g., Host_Wr(Addr, Data)) that specifies a real address and data to be written to the real address. The host write command 3100 is issued to the AFUM controller 220 of the receiving host B. In response to receiving the host write command 3100, the CCR XLATE layer 2602b of host B translates the host write command 3100 and, optionally, the real address specified by the host write command 3100. For example, the command translation logic 2720 translates the host write command 3100 into an AFUM write command 3102 (AFUM_Wr(TAddr, Data)) so that the AFUM write command 3102 emulates, or appears to the receiving host B, to have been initiated by an AFU 502 attached directly to host B. Optionally, the address translation logic 2722 of host B may translate the real address specified by the host write command 3100 and obtain a translated address (TAddr). The CCR XLATE layer 2602b then passes this AFUM write command 3102 to the host transaction layer 306b of host B, which processes the AFUM write command 3102 as described above with reference to FIG.

[0088] In the success case illustrated in Figure 32, for example, in response to a host bus protocol write request initiated by host transaction layer 306b as described with reference to block 1636 of Figure 16, Host B performs the requested write and, optionally, returns an indication of the success of the write request to AFUM controller 220. (Some host bus protocols provide an indication of the success of a write request to the requestor, while others do not. For host bus protocols that do not provide an indication of the success of a write request, AFUM controller 220 assumes success and provides an appropriate host response.) In response to completion of the write request, Host transaction layer 306b of Host B issues a host write response 3200 (e.g., Host_Wr_Resp(NErr)) indicating no error.

[0089] The host write response 3200 is received and processed by the CCR XLATE layer 2602a of host A. In response to receiving the host write response 3200, the response translation logic 2724 of the CCR XLATE layer 2602a translates the host write response 3200 into an AFUM write response 3202 (e.g., AFUM_Wr_Resp) to emulate, or appear from the perspective of host A, that the attached memory 402 has completed the requested write. It should be noted that the no error indication (NErr) provided in the host write response 3200 causes the response translation logic 2724 to provide AFUM_Wr_Resp rather than AFUM_Wr_Failed. The CCR XLATE layer 2602a forwards the AFUM write response 3202 to the host bus protocol interface 304a, which processes the AFUM write response 3202 as described above with reference to FIG. 25. For example, the host bus protocol interface 304a may communicate success of the requested write to the requesting master (e.g., L2 cache 230) within host A using the host bus protocol of host A. Alternatively, the host bus protocol interface 304a may allow or require the write request to complete without issuing any host bus protocol communication on the system fabric of host A, in which case the AFUM write response 3202 may be discarded by the host transaction layer 306a.

[0090] Figure 33 illustrates a case in which a write failure occurs at Host B. In this case, in response to the host bus protocol write request initiated by Host Transaction Layer 306b in block 1636 of Figure 16, Host B responds to Host Bus Protocol Interface Layer 304b of Host B with a message indicating a failure of the host bus protocol write request. In response, Host Bus Protocol Interface Layer 304b of Host B issues a host write response 3300 (e.g., Host_Wr_Resp(Err)) indicating a write failure.

[0091] The host write response 3300 is received and processed by the CCR XLATE layer 2602a of host A. In response to receiving the host write response 3300, the response translation logic 2724 of the CCR XLATE layer 2602a translates the host write response 3300 into an AFUM write response 3302 indicating failure (e.g., AFUM_Wr_Failed). Again, it should be noted that the (Err) indication provided in the host write response 3300 causes the response translation logic 2724 to provide AFUM_Wr_Failed rather than AFUM_Wr_Resp. The CCR XLATE layer 2602a forwards the AFUM write response 3302 to the host bus protocol interface 304a, which initiates error handling as described above with reference to block 2514 of FIG. 25 .

[0092] Referring now to FIG. 34, a high-level logical flowchart of an exemplary process for initializing data processing systems to share memory is shown, according to one embodiment. The process of FIG. 34 begins at block 3400 and proceeds to block 3402, which illustrates host data processing systems 100 (e.g., Host A and Host B) coupled for memory sharing adjusting their respective allocations of memory regions within real address spaces. For example, this adjustment may be performed by a hypervisor and / or operating system software running on one or both of the host data processing systems 100. In addition, each of the host data processing systems 100 initializes page table entries 212 in page frame table 210 to provide effective address-to-real address translation at each of the hosts (block 3404).

[0093] As further shown in block 3406, the host data processing systems 100 communicate with each other (e.g., via hypervisor software), and each host data processing system 100 acting as a receiving (or target) host fixes, within its system memory 108, any page table entries 212 that are useful for translating a real address that may be specified by a host command of the initiating host (block 3406). Each host data processing system 100 further initializes its respective AFUM controller 220 and the appropriate BAR registers 216, 224 (block 3408). As previously mentioned, initialization of the AFUM controller 220 includes setting the configuration register 2712 to indicate communication between hosts via the AFUM controller 220. Initialization of the BAR registers 216, 224 causes host bus protocol memory access requests (e.g., reads, writes, etc.) on the host's system fabric to be appropriately routed to the local system memory 108 within the host via the memory controller 106 or to the receiving host via the AFUM controller 220. Following block 3408, at block 3410, the process of FIG.

[0094] 35, a time-space diagram of a write command issued by AFU 502 to host data processing system 100 and the associated response of host data processing system 100 is shown, according to one embodiment employing a fast response write mode. As described above with reference to FIG. 11, AFU 502 issues AFUM write command 1100 (e.g., AFUM_Wr(Addr,Data)) to host data processing system 100 via protocol stack 500 of FIG. 5. In the previous embodiment described above with reference to FIGS. 12-13, the host bus protocol write request corresponding to AFUM write command 1100 is processed in the receiving host data processing system 100 before host transaction layer 306 provides host write response 1200 or 1300 to AFU 502. In contrast, the AFUM controller 220 configured to operate in the fast response write mode shown in FIG. 35 instead responds to the AFUM write command 3500 with a host write response 3502 (e.g., Host_Wr_Resp(NErr)) indicating no errors, possibly prior to processing of the host bus protocol write request in the receiving host data processing system 100, regardless of that processing. Issuing the host write response 3502 regardless of the success or failure of the host bus protocol write means that the conversion / authorization failure cases shown in FIGS. 14-15 are not allowed to occur. To prevent such failure cases, the associated PTE 212 is fixed in the receiving host.

[0095] Referring now to Figure 36, there is shown a high-level logic flow diagram of an exemplary manner in which the host transaction layer 306 of the AFUM controller 220 responds to a write command in accordance with the fast response write mode illustrated in Figure 35. The process of Figure 36 begins at block 3600 and proceeds to block 3602, which illustrates the host transaction layer 306 of the AFUM controller 220 configured to operate in fast response write mode receiving an AFUM write command 3500 (e.g., AFUM_Wr(Addr,Data)). Following block 3602, the process of Figure 36 branches and proceeds in parallel to blocks 3604 and 3606. Block 3604 indicates that the host transaction layer 306 of the AFUM controller 220 issues a host response 3502 (e.g., Host_Wr_Resp(NErr)) indicating no error prior to processing the host bus protocol write request corresponding to the AFUM write command 3500 in the receiving host data processing system 100, regardless of the processing of the host bus protocol write request. The process then proceeds from block 3604 to node 3619.

[0096] Referring to block 3606, the host transaction layer 306 determines whether to skip translating the effective address and performing write permission specified in the AFUM write command 3500. The host transaction layer 306 may determine to skip translating the effective address and performing write permission based on, for example, that the effective address of the AFUM command falls within a predetermined address range, instructions provided in the AFUM command, or the configuration of the AFUM controller 220, or a combination thereof. In response to a positive determination at block 3606, the process proceeds to block 3616, described below. Otherwise, the process proceeds to block 3608. It should be understood that if a determination is made at block 3606 to skip translating the address and performing write permission for the AFUM command, other techniques not shown, including appropriate configuration of the page frame table 210, are utilized to ensure that the effective address is within a permitted address range and has the required write permission.

[0097] Block 3608 depicts the host transaction layer 306 determining whether the ATC 226 holds an address translation entry for translating the effective address specified in the AFUM write command 3500 to a real address. In response to the host transaction layer 306 determining that the ATC 226 holds an associated address translation entry, the process proceeds to block 3614, described below. On the other hand, if the host transaction layer 306 determines that the ATC 226 does not hold an associated address translation entry, the host transaction layer 306 initiates sending an NMMU translation request for the effective address specified by the AFUM write command 3500 to the NMMU 228 via the host bus protocol interface layer 304 and the host bus protocol layer 302 (block 3610). With the associated page table entry 212 fixed in the system memory 108 as described above with reference to block 3406 of FIG. 34, the NMMU 228 successfully obtains the required translation entry and provides the requested address translation entry to the AFUM controller 220 for installation into the ATC 226 (block 3612).

[0098] At block 3614, host transaction layer 306 protects against write protection errors by determining whether the address translation entry for the effective address specified by AFUM write command 3500 indicates write permission for the effective address specified by AFUM write command 3500. If write permission is indicated, host transaction layer 306 begins processing the write operation specified by AFUM write command 3500 within host data processing system 100 by causing the issuance of an appropriate host bus protocol write request on the system fabric via host bus protocol interface layer 304 and host bus protocol layer 302 (block 3616). However, if at block 3614 host transaction layer 306 determines that the address translation entry does not provide write permission for the effective address specified by AFUM write command 3500, host transaction layer 306 begins error processing, as shown at block 3618. Following block 3616 or block 3618, the process of FIG. 36 proceeds to node 3619. After both edges of the process of FIG. 36 meet at block 3619, the process of FIG. 36 ends at block 3620.

[0099] 35-36 in comparison with the alternative write modes illustrated in FIGS. 11-13, it should be understood that a design trade-off may be made between the allocation of communication resources (e.g., command queues, communication credits, virtual channels, etc.) to the write command and the availability of information about whether the host bus protocol write request completed successfully at the receiving host data processing system 100. In particular, the issuance of an early host write response 3502, regardless of the success or failure of the write operation at the receiving host data processing system 100, allows the initiating participant (e.g., AFU 502) to more quickly release its resources allocated to the write command, thus freeing those resources for earlier allocation to other commands. For an AFU 502 attached directly to the AFUM controller 220, this timing difference between the different write modes may not be large enough to warrant loss of information about the success or failure of the write operation at the receiving host data processing system 100. However, in implementations in which the AFUM controller 220 is coupled for memory sharing between hosts (especially over multiple hops), use of the fast response write mode can advantageously free up communication resources in the initiating data processing system 100 (and any data processing systems 100 intervening between the initiating data processing system 100 and the receiving data processing system 100), reducing errors and / or performance issues resulting from exhaustion of such communication resources and simplifying design complexity.

[0100] FIG. 37 is a time-space diagram of a write command issued by one host data processing system to another host data processing system, and an associated write response, according to one embodiment employing the fast write response mode described above and protocol stack 2600 of FIG. 26.

[0101] 37, the AFUM controller 220 of the initiating host A issues a host write command 3700 (e.g., Host_Wr(Addr, Data)) that specifies a real address and data to be written to the specified real address. The host write command 3700 is issued to the AFUM controller 220 of the receiving host B. In response to receiving the host write command 3700, the CCR XLATE layer 2602b of host B translates the host write command 3700 and, optionally, the real address specified by the host write command 3700. For example, the command translation logic 2720 translates the host write command 3700 into an AFUM write command 3702 (AFUM_Wr(TAddr, Data)), emulating, or making it appear to the receiving host B, that the AFUM write command 3702 was initiated by an AFU 502 attached directly to the AFUM controller 220 of host B. Optionally, the address translation logic 2722 of host B may translate the real address specified by the host write command 3700 to obtain a translated address (TAddr). The CCR XLATE layer 2602 then passes this AFUM write command 3702 to the host transaction layer 306b of host B, which processes the AFUM write command 3702 as described above with reference to Figure 16. This processing includes initiating a host bus protocol write request on the system fabric of host B to write the data, for example, to system memory 108.

[0102] Instead of waiting for the completion of the host bus protocol write request to provide a host response, the host transaction layer 306b responds to the AFUM write command 3500 with a host write response 3704 (e.g., Host_Wr_Resp(NErr)) indicating no errors, possibly prior to the processing of the host bus protocol write request within the receiving host data processing system 100. The host write response 3704 is received and processed by the CCR XLATE layer 2602a of Host A. In response to receiving the host write response 3704, the response translation logic 2724 of the CCR XLATE layer 2602a translates the host write response 3704 into an AFUM write response 3706 (e.g., AFUM_Wr_Resp) to emulate, or make it appear from the perspective of Host A, that the attached memory 402 has completed the requested write. CCR XLATE layer 2602a forwards AFUM write response 3706 to host bus protocol interface 304a, which processes AFUM write response 3706 as described above with reference to FIG. 25. For example, host bus protocol interface 304a may communicate success of the requested write to the requesting master (e.g., L2 cache 230) within host A using host bus protocol of host A. Alternatively, if allowed or required by host bus protocol of host A, host bus protocol interface 304a may allow the write request to complete without issuing any host bus protocol communication on host A's system fabric, in which case AFUM write response 3706 may be discarded by host transaction layer 306a.

[0103] As alluded to above, memory sharing between hosts is not limited to the interconnection of two hosts but instead may be extended to any desired number of participating host processing systems 100 and may utilize a number of different connection topologies. For example, FIG. 38 illustrates a high-level block diagram of an exemplary topology of a data processing environment in which multiple host data processing systems 100 are communicatively coupled to support memory sharing. As described below, memory shared among coupled data processing systems 100 may include a variety of memory types, such as memory provided by the system memory 108 of the host data processing system 100, memory provided by additional memory 402, or memory provided by a memory appliance, or combinations thereof. It should be understood that FIG. 38 omits illustration of many components of the illustrated host data processing system 100 to avoid unnecessarily obscuring the details of the invention.

[0104] In the illustrated example, data processing environment 3800 includes four host data processing systems 100a, 100b, 100c, and 100d, each of which includes multiple processing nodes 102, as previously described with reference to FIGS. 1-2. In this example, each host data processing system 100 includes at least two processing nodes. Thus, host data processing system 100a includes at least processing nodes 102a1-102a2, host data processing system 100b includes at least processing nodes 102b1-102b2, host data processing system 100c includes at least processing nodes 102c1-102c2, and host data processing system 100d includes at least processing nodes 102d1-102d2. In this example, one or more of the processing nodes 102 of each host processing system 100 includes one or more AFUM controllers 220. For example, processing node 102a1 of host data processing system 100a includes AFUM controllers 220a1 and 220a2, processing node 102b1 of host data processing system 100b includes AFUM controllers 220b1-220b4, processing node 102c1 of host data processing system 100c includes AFUM controllers 220c1 and 220c2, and processing node 102d1 of host data processing system 100d includes AFUM controllers 220d1 and 220d2. To support memory sharing between hosts, AFUM controller 220a2 of host processing node 100a is coupled to AFUM controller 220d1 of host processing node 100d, AFUM controller 220a1 of host processing node 100a is coupled to AFUM controller 220b1 of host processing node 100b, and AFUM controller 220b2 of host processing node 100b is coupled to AFUM controller 220c2 of host processing node 100c. In this manner, each of host data processing systems 100a-100d is communicatively coupled to one another via one or more hops.While in at least some embodiments it is preferable to avoid the cost and complexity of other mechanisms for communicatively coupling host data processing systems 100, it should be understood that host data processing systems 100 may optionally be coupled by other additional means, such as switch 3804, which in the illustrated embodiment is coupled to AFUM controller 200d2 of host data processing system 100d and AFUM controller 220b3 of host data processing system 100b.

[0105] The data processing environment 3800 further includes a memory device 3802, which provides a large memory reservoir available for sharing by all of the host data processing systems 100 in the data processing environment 3800. In this example, the memory device 3802 includes a host bus 3810 that supports the attachment of multiple memory controllers 106, each supporting a respective device memory 3812. The memory device 3802 further includes an AFUM controller 220e1 coupled for communication between the switch 3804 and the host bus 3810, and an AFUM controller 220e2 coupled for communication between the host bus 3810 and additional memory 402e1. The memory device 3802 also includes an AFUM interface 225 to which additional memory 402e2 is attached.

[0106] To reduce cost and complexity, the memory device 3802 preferably omits a processing node 102 for general-purpose processing (although it may include, for example, a service processor). The memory device 3802 preferably includes a fixed page frame table 210 (not shown) stored within one or more of the device memories 3812, into which a PTE 212 may be written by the host data processing system 100, for example, by utilizing an AFUM write command. The memory device 3802 preferably further includes, or is communicatively coupled to, an NMMU 228 (not shown) for retrieving any address translation entries needed to translate addresses specified in received commands.

[0107] As previously discussed, the illustrated data processing system environment 3800 supports shared read and write access by requestors in any of the host data processing systems 100 to any of the various types of memory within the data processing system environment 3800 via the described communication interfaces. For example, after the host data processing systems 100a-100d, memory devices 3802, and the communication links coupling them are configured and enabled as described in FIG. 34, each host data processing system 100 can access its own system memory 108 or the system memory 108 of any of the other host data processing systems 100, as represented by system memory 108c of host data processing system 100c. Similarly, each host data processing system 100 can access its own additional memory 402 or the additional memory 402 of either another host data processing system 100 or memory device 3802, as represented by additional memory 402b1 of host data processing system 100b and additional memories 402e1-402e2 in memory device 3802. Furthermore, each host data processing system 100 can access device memory 3812 of memory device 3802.

[0108] When host-to-host access to shared memory in the data processing system environment 3800 occurs through one or more intervening hosts, the multi-hop command and response flow is the same as that previously described, except that the host bus protocol of the intervening hosts is utilized to facilitate communication of commands and responses through the intervening hosts (hops). In certain cases, the native host bus protocol of the host data processing system 100 may not provide a set of commands and responses sufficient to convey all commands and responses employed in the described host-to-host memory sharing. In such cases, the host bus protocol is preferably extended as necessary to support the described communication of host bus protocol commands and responses. Of course, alternative embodiments may implement host bus protocols that are more directly compatible with the described commands and responses. Support for multi-hop memory access as described may also require the implementation of virtual channels in the host bus protocol of the host data processing system's system fabric and on the communication link between the AFUM controller 220 to prevent deadlock. The implementation of such virtual channels is known to those skilled in the art.

[0109] Multi-hop Command and Response Flows For purposes of illustration, the multi-hop host read command and response corresponding to Figures 28-29 are shown in Figure 39, the multi-hop host write command and response corresponding to Figures 31-32 are shown in Figure 40, and the multi-hop host write command and response corresponding to Figure 37 are shown in Figure 41. Data flows illustrating error cases (e.g., cases corresponding to one hop errors shown in Figures 30 and 33) have been omitted for brevity, but one skilled in the art will readily understand the implementation of these data flows from the following description.

[0110] Referring now to Figure 39, there is shown a time-space diagram of a multi-hop read command issued by an initiating host data processing system to a receiving host data processing system via an intervening host data processing system, and associated responses, according to one embodiment.

[0111] In the example of FIG. 39, L2 cache 230 initiates a host bus protocol read request (not shown) on the system fabric of host data processing system 100a. The host bus protocol read request specifies a real address identifying data requested by an associated processor core 200. In this example, the requested data resides in system memory 108 of host data processing system 100c. In response to receiving the host bus protocol read request, AFUM controller 220a1 of host data processing system 100a determines, by reference to its BAR 224, that AFUM controller 220a1 is responsible for the real address of the host bus protocol read request and, in response, issues a host read command 3900 (e.g., Host_Rd(Addr)) to AFUM controller 220b1 of host data processing system 100b. In response to receiving the host read command 3900, the CCR XLATE layer 2602 of the AFUM controller 220b1 translates the host read command 3900, and optionally the real address specified by the host read command 3900, to obtain an AFUM read command 3902 (AFUM_Rd(TAddr1)). The CCR XLATE layer 2602 of the AFUM controller 220b1 then passes the AFUM read command 3902 to the host transaction layer 306 of the AFUM controller 220b1, which initiates a host bus protocol read request 3904 (e.g., HBP_Rd(TAddr1)) on the system fabric of the host data processing system 100b.

[0112] In response to receiving host bus protocol read request 3904 on the system fabric of host data processing system 100b, AFUM controller 220b2 of host data processing system 100b determines, by reference to its BAR 224, that AFUM controller 220b2 is responsible for the real address of host bus protocol read request 3904 and, in response, issues a host read command 3906 (e.g., Host_Rd(TAddr1)) to AFUM controller 220c2 of host data processing system 100c. In response to receiving host read command 3906, CCR XLATE layer 2602 of AFUM controller 220c2 translates host read command 3906, and optionally the translated real address specified by host read command 3906, to obtain AFUM read command 3908 (e.g., AFUM_Rd(TAddr2)). The CCR XLATE layer 2602 of the AFUM controller 220c2 then passes this AFUM read command 3908 to the host transaction layer 306 of the AFUM controller 220c2, which then initiates a host bus protocol read request on the system fabric of the host data processing system 100c. For example, the host bus protocol read request may request data stored in the system memory 108c. In this case, a match between the actual address specified by the host bus protocol read request and the address range specified by the BAR 216 of the memory controller 106 causes the memory controller 106 to access the requested data in the system memory 108c and deliver the requested data back to the AFUM controller 220c2.

[0113] In response to receiving the requested data, the host transaction layer 306 of the AFUM controller 220c2 issues a host read response 3910 (e.g., Host_Rd_Resp(Data,NErr)) that provides the requested data and indicates no errors. The host read response 3910 is received and processed by the CCR XLATE layer 2602 of the AFUM controller 220b2 of the host data processing system 100b. In response to receiving the host read response 3910, the CCR XLATE layer 2602 of the AFUM controller 220b2 converts the host read response 3910 into an AFUM read response 3912 (e.g., AFUM_Rd_Resp(Data)) and forwards the AFUM read response 3912 to the host bus protocol interface layer 304 of the AFUM controller 220b2. In response, the host bus protocol interface layer 304 then initiates a host bus protocol read response 3914 (e.g., HBP_Rd_Resp(Data, NErr)) on the system fabric of the host data processing system 100b. In response to receiving the host bus protocol read response 3914, the host transaction layer 306 of the AFUM controller 220b1 issues a host read response 3916 (e.g., Host_Rd_Resp(Data, NErr)) that provides the requested data and indicates no errors. The host read response 3916 is received and processed by the CCR XLATE layer 2602 of the AFUM controller 220a1 of the host data processing system 100a. In response to receiving the host read response 3916, the CCR XLATE layer 2602 converts the host read response 3916 into an AFUM read response 3918 (e.g., AFUM_Rd_Resp(Data)) and forwards the AFUM read response 3918 to the host bus protocol interface layer 304 of the host data processing system 100a.In response, the host bus protocol interface layer 304 then initiates a host bus protocol read response (e.g., HBP_Rd_Resp(Data, NErr)), not shown, on the system fabric of the host data processing system 100a, and delivers the requested data to the original requestor (e.g., L2 cache 230).

[0114] Referring now to Figure 40, there is shown a time-space diagram of a multi-hop write command issued by an initiating host data processing system to a receiving host data processing system via an intervening host data processing system, and associated responses, according to one embodiment.

[0115] In the example of Figure 40, L2 cache 230 initiates a host bus protocol write request (not shown) on the system fabric of host data processing system 100a. The host bus protocol write request specifies a real address and data to be written to the real address. In response to receiving the host bus protocol write request, AFUM controller 220a1 determines, by reference to its BAR 224, that AFUM controller 220a1 is responsible for the real address of the host bus protocol write request and, in response, issues a host write command 4000 (e.g., Host_Wr(Addr,Data)) to AFUM controller 220b1 of host data processing system 100b. In response to receiving the host write command 4000, the CCR XLATE layer 2602 of the AFUM controller 220b1 translates the host write command 4000, and optionally the real address specified by the host write command 4000, to obtain an AFUM write command 4002 (AFUM_Wr(TAddr1,Data)). The CCR XLATE layer 2602 of the AFUM controller 220b1 then passes the AFUM write command 4002 to the host transaction layer 306 of the AFUM controller 220b1, which initiates a host bus protocol write request 4004 (e.g., HBP_Wr(TAddr1,Data)) on the system fabric of the host data processing system 100b.

[0116] In response to receiving a host bus protocol write request 4004 on the system fabric of host data processing system 100b, AFUM controller 220b2 determines, by reference to its BAR 224, that it is responsible for the real address of host bus protocol write request 4004 and, in response, issues a host write command 4006 (e.g., Host_Wr(TAddr1,Data)) to AFUM controller 220c2 of host data processing system 100c. In response to receiving host write command 4006, CCR XLATE layer 2602 of AFUM controller 220c2 translates host write command 4006, and optionally the translated real address specified by host write command 4006, to obtain an AFUM write command 4008 (e.g., AFUM_Wr(TAddr2,Data)). The CCR XLATE layer 2602 of the AFUM controller 220c2 then passes this AFUM write command 4008 to the host transaction layer 306 of the AFUM controller 220c2, which then initiates a host bus protocol write request on the system fabric of the host data processing system 100c. For example, the host bus protocol write request may request that a data payload be stored in the system memory 108c. In this case, a match between the actual address specified by the host bus protocol write request and the address range specified by the BAR 216 of the memory controller 106 causes the memory controller 106 to store the data payload of the host bus protocol write request in the system memory 108c.

[0117] In response to successful completion of the host bus protocol write request in host data processing system 100c (which may or may not be acknowledged by a response on the system fabric of host data processing system 100c), host transaction layer 306 of AFUM controller 2202c in host data processing system 100c issues a host write response 4010 (e.g., Host_Wr_Resp(NErr)) indicating no error. Host write response 4010 is received and processed by CCR XLATE layer 2602 of AFUM controller 220b2 of host data processing system 100b. In response to receiving the host write response 4010, the CCR XLATE layer 2602 of the AFUM controller 220b2 translates the host write response 4010 into an AFUM write response 4012 (e.g., AFUM_Wr_Resp) and forwards the AFUM write response 4012 to the host bus protocol interface layer 304 of the AFUM controller 220b2. In response, the host bus protocol interface layer 304 then initiates a host bus protocol write response 4014 (e.g., HBP_Wr_Resp(NErr)) on the system fabric of the host data processing system 100b. As previously mentioned, some host bus protocols do not natively include a host bus protocol write response that indicates no error. In embodiments employing end-to-end communication of write responses such as that shown in Figure 40, the message set of such a host bus protocol may be extended to allow communication of a write response indicating no errors on the system fabric, such as host bus protocol write response 4014. Of course, a different host bus protocol that is more directly compatible with the described message protocol may alternatively be employed.

[0118] In response to receiving the host bus protocol write response 4014, the host transaction layer 306 of the AFUM controller 220b1 issues a host write response 4016 (e.g., Host_Wr_Resp(NErr)) indicating no error. The host write response 4016 is received and processed by the CCR XLATE layer 2602 of the AFUM controller 220a1 of the host data processing system 100a. In response to receiving the host write response 4016, the CCR XLATE layer 2602 of the AFUM controller 220a1 converts the host write response 4016 into an AFUM write response 4018 (e.g., AFUM_Wr_Resp) and forwards the AFUM write response 4018 to the host bus protocol interface layer 304 of the AFUM controller 220a1. In response, the host bus protocol interface layer 304 may then initiate a host bus protocol write response on the system fabric of the host data processing system 100a, if requested or permitted by the host bus protocol of the host data processing system 100a.

[0119] Referring to FIG. 41, a time-space diagram of a multi-hop write command and associated response issued by an initiating host data processing system to a receiving host data processing system via an intervening host data processing system is shown in accordance with another embodiment implementing a fast write response mode.

[0120] In the example of Figure 41, L2 cache 230 initiates a host bus protocol write request (not shown) on the system fabric of host data processing system 100a. The host bus protocol write request specifies a real address and the data to be written to the real address. In response to receiving the host bus protocol write request, AFUM controller 220a1 determines, by reference to its BAR 224, that AFUM controller 220a1 is responsible for the real address of the host bus protocol write request and, in response, issues a host write command 4100 (e.g., Host_Wr(Addr,Data)) to AFUM controller 220b1 of host data processing system 100b. In response to receiving the host write command 4100, the CCR XLATE layer 2602 of the AFUM controller 220b1 translates the host write command 4100, and optionally the real address specified by the host write command 4100, to obtain the AFUM write command 4102 (AFUM_Wr(TAddr1, Data)). The CCR XLATE layer 2602 of the AFUM controller 220b1 then passes the AFUM write command 4102 to the host transaction layer 306 of the AFUM controller 220b1.

[0121] Instead of waiting for the write operation to complete, the host transaction layer 306 of the AFUM controller 220b1 responds to the AFUM write command 4102 with a host write response 4116 (e.g., Host_Wr_Resp(NErr)) indicating no errors in the processing of the write operation. The host write response 4116 is received and processed by the CCR XLATE layer 2602 of the AFUM controller 220a1, which converts the host write response 4116 into an AFUM write response 4118 (e.g., AFUM_Wr_Resp). The CCR XLATE layer 2602 of the AFUM controller 220a1 forwards the AFUM write response 4118 to the associated host bus protocol interface layer 304, which may then initiate a host bus protocol write response on the system fabric of the host data processing system 100a if requested or allowed by the host bus protocol of the host data processing system 100a.

[0122] In addition to issuing a host write response 4116, the host transaction layer 306 of the AFUM controller 220b1 responds to the AFUM write command 4102 by issuing a host bus protocol write request 4104 (e.g., HBP_Wr(TAddr1,Data)) on the system fabric of the host data processing system 100b. In response to receiving the host bus protocol write request 4104 on the system fabric of the host data processing system 100b, the AFUM controller 220b2 determines, by reference to its BAR 224, that it is responsible for the actual address of the host bus protocol write request 4104 and, in response, issues a host write command 4106 (e.g., Host_Wr(TAddr1,Data)) to the AFUM controller 220c2 of the host data processing system 100c. In response to receiving the host write command 4106, the CCR XLATE layer 2602 of the AFUM controller 220c2 translates the host write command 4106, and optionally the translated real address specified by the host write command 4106, to obtain an AFUM write command 4108 (e.g., AFUM_Wr(TAddr2,Data)). The CCR XLATE layer 2602 of the AFUM controller 220c2 then passes this AFUM write command 4108 to the host transaction layer 306 of the AFUM controller 220c2, which then initiates a host bus protocol write request on the system fabric of the host data processing system 100c and simultaneously issues a host write response 4110 (e.g., Host_Wr_Resp(NErr)) indicating no errors regarding the processing of the host bus protocol write request.

[0123] The host write response 4110 is received and processed by the CCR XLATE layer 2602 of the AFUM controller 220b2 of the host data processing system 100b. In response to receiving the host write response 4110, the CCR XLATE layer 2602 translates the host write response 4110 into an AFUM write response 4112 (e.g., AFUM_Wr_Resp) and forwards the AFUM write response 4112 to the host bus protocol interface layer 304 of the AFUM controller 220b2. As shown, the host bus protocol interface layer 304 of the AFUM controller 220b2 does not forward the AFUM write response 4112 further, thus avoiding forwarding the AFUM write response 4112 to the host data processing system 100a.

[0124] Given the foregoing description of memory sharing using multi-hop communication, the data processing environment 3800 may be configured to implement any of several different alternative modes for handling multi-hop memory sharing. In a first mode, multi-hop reads and writes may be handled using end-to-end synchronous communication, as shown in FIGS. 39-40. In a second mode, multi-hop reads may be handled using end-to-end synchronous communication, as shown in FIG. 39, and multi-hop writes may be handled using asynchronous communication in a fast write response mode, as shown in FIG. 41. In a third mode, multi-hop writes may be handled using asynchronous communication, as shown in FIG. 41, and multi-hop reads are not permitted over the communication interface provided by the AFUM controller 220. Instead, in this third mode, reads from remote memory may be performed using conventional remote direct memory access (RDMA) write operations. For example, an initiating host wishing to read from remote memory in a receiving host may write a read target address from which data is to be read to a specified memory location in the receiving host. In response to the receiving host detecting a write to the specified memory location, the receiving host utilizes the read target address to access the requested data and writes the data to the initiating host's memory.

[0125] Referring now to FIG. 42, a block diagram of an exemplary design flow 4200 for use, for example, in the logical design, simulation, testing, layout, and manufacturing of semiconductor ICs is shown. The design flow 4200 includes a process, machine, or mechanism, or a combination thereof, for processing a design structure or device to generate a logically equivalent or otherwise functionally equivalent representation of the design structure and / or device described herein. The design structure processed and / or generated by the design flow 4200 may be encoded on a machine-readable transmission medium or machine-readable storage medium to include data and / or instructions that, when executed on a data processing system or otherwise processed, generate a logically, structurally, mechanically, or otherwise functionally equivalent representation of a hardware component, circuit, device, or system. The machine includes, but is not limited to, any machine used in an IC design process, such as designing, manufacturing, or simulating a circuit, component, device, or system. For example, a machine may include a lithography machine, a machine or apparatus, or both, for generating a mask (e.g., an electron beam writer), a computer or apparatus for simulating a design structure, any apparatus used in a manufacturing or testing process, or any machine for programming a functionally equivalent representation of a design structure into any medium (e.g., a machine for programming a programmable gate array).

[0126] Design flow 4200 may vary depending on the type of representation being designed. For example, a design flow 4200 for building an application specific IC (ASIC) may differ from a design flow 4200 for designing a standard component, or from a design flow 4200 for instantiating a design into a programmable array (e.g., a programmable gate array (PGA) or field programmable gate array (FPGA) offered by Altera® or Xilinx®).

[0127] 42 illustrates multiple such design structures, including an input design structure 4220, which is preferably processed by the design process 4200. The design structure 4220 may be a logic simulation design structure generated and processed by the design process 4200 to generate a logically equivalent functional representation of a hardware device. The design structure 4220 may also or alternatively include data and / or program instructions that, when processed by the design process 4200, generate a functional representation of the physical structure of the hardware device. The design structure 4220, such as the design structure implemented by the central developer / designer, whether representing functional features and / or structural design features, may be generated using electronic computer-aided design (ECAD). When encoded in a machine-readable data transmission, gate array, or storage medium, design structure 4220 may be accessed and processed by one or more hardware and / or software modules within design process 4200 to simulate or otherwise functionally represent an electronic component, circuit, electronic or logic module, apparatus, device, or system as described herein. Thus, design structure 4220 may include files or other data structures, including human- and / or machine-readable source code, compiled structures, and computer-executable code structures, that, when processed by a design or simulation data processing system, functionally simulate or otherwise represent a circuit or other level of hardware logic design. Such data structures may include hardware-description language (HDL) design entities or other data structures that conform to or are compatible with low-level HDL design languages ​​such as Verilog and VHDL, and / or high-level design languages ​​such as C or C++.

[0128] Design process 4200 preferably employs and incorporates hardware and / or software modules for synthesizing, transforming, or otherwise processing functional equivalents of designs / simulations of components, circuits, devices, or logic structures shown herein to generate netlist 4280, which may include design structures such as design structure 4220. Netlist 4280 may include, for example, compiled or otherwise processed data structures representing lists of wires, individual components, logic gates, control circuits, I / O devices, models, etc., that describe connections to other elements and circuits within an integrated circuit design. Netlist 4280 may be synthesized using an iterative process in which netlist 4280 is resynthesized one or more times depending on the device's design specifications and design parameters. As with the other types of design structures described herein, netlist 4280 may be recorded on a machine-readable storage medium or programmed into a programmable gate array. The medium may be a non-volatile storage medium such as a magnetic or optical disk drive, a programmable gate array, compact flash, or other flash memory. Additionally or alternatively, the medium may be system memory or cache memory, or buffer space.

[0129] Design process 4200 may include hardware and software modules for processing various input data structure types, including netlist 4280. Such data structure types may include, for example, sets of commonly used elements, circuits, and devices present in library elements 4230, including models, layouts, and symbolic representations for specific manufacturing technologies (e.g., various technology nodes, 32 nm, 45 nm, 90 nm, etc.). Data structure types may further include design specifications 4240, characterization data 4250, verification data 4260, design rules 4270, and test data files 4285, which may include input test patterns, output test results, and other test information. Design process 4200 may also include standard mechanical design processes, such as stress analysis, thermal analysis, mechanical event simulation, and process simulation for operations such as casting, molding, and die-forming. Those skilled in the art of mechanical design will appreciate a range of mechanical design tools and applications that may be used in design process 4200 without departing from the scope and spirit of the present invention. The design process 4200 may include modules for performing standard circuit design processes, such as timing analysis, verification, design rule checking, place and route operations, etc.

[0130] Design process 4200 employs and incorporates logical and physical design tools, such as HDL compilers and simulation model building tools, to process design structure 4220 along with some or all of the illustrated supporting data structures, along with any additional mechanical design or data (if applicable), to generate second design structure 4290. Design structure 4290 resides on a storage medium or programmable gate array in a data format used for the exchange of mechanical device or structure data (e.g., information stored in IGES, DXF, Parasolid XT, JT, DRG, or any other suitable format for storing or rendering such mechanical design structures). Like design structure 4220, design structure 4290 resides on a transmission medium or data storage medium and preferably includes one or more files, data structures, or other computer-encoded data or instructions that, when processed by an ECAD system, generate a logically equivalent or otherwise functionally equivalent form of one or more of the embodiments of the present invention. In one embodiment, design structure 4290 may include a compiled, executable HDL simulation model that functionally simulates the devices shown herein.

[0131] Design structure 4290 may employ data formats used for the exchange of integrated circuit layout data and / or symbolic data formats (e.g., information stored in GDSII (GDS2), GL1, OASIS, map files, or any other suitable format for storing such design data structures). Design structure 4290 may include information such as symbolic data, map files, test data files, design content files, manufacturing data, layout parameters, wires, metals for each level, vias, shapes, data for routing by manufacturing line, and any other data required by a manufacturer or other designer / developer to manufacture the aforementioned devices or structures shown herein. Design structure 4290 may then proceed to stage 4295, where, for example, design structure 4290 may proceed to tape-out and be released for manufacturing, released to a mask company, sent to another design company, returned to a customer, etc.

[0132] As described, in at least one embodiment, a communications interface of a second host data processing system receives a host command in a first command set from a first host data processing system. The host command specifies a memory access to a memory coupled to the second host data processing system. The communications interface translates the host command into a command in a different second command set that emulates the coupling of an additional functional unit to the communications interface. The communications interface presents a second command to a host bus protocol interface of the second host data processing system. Based on receipt of the second command, the host bus protocol interface initiates a host bus protocol memory access request specifying the memory access on a system fabric of the second host data processing system.

[0133] While various embodiments have been shown and described in detail, it will be understood by those skilled in the art that various changes in form and detail may be made herein without departing from the spirit and scope of the appended claims, and that all such alternative implementations are intended to be encompassed within the scope of the appended claims.

[0134] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, comprising one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions shown in the blocks may occur out of the order shown in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or in the reverse order, depending on the functionality involved. It is also noted that each block in the block diagrams and / or flowchart diagrams, and combinations of blocks included in the block diagrams and / or flowchart diagrams, may be implemented by a special-purpose hardware-based system that performs the specified function(s) or operation(s), or executes a combination of special-purpose hardware and computer instructions.

[0135] While aspects have been described with respect to a computer system executing program code that directs the functions of the present invention, it should be understood that the present invention may alternatively be implemented as a program product that includes a computer-readable storage device having stored thereon program code that can be processed by a processor of the data processing system to cause the data processing system to perform the described functions. The computer-readable storage device may include volatile or non-volatile memory, optical or magnetic disks, etc., but excludes statutorily disallowed subject matter such as propagated signals per se, transmission media per se, and forms of energy per se.

[0136] As an example, a program product may include data and / or instructions that, when executed or otherwise processed on a data processing system, generate a logical, structural, or otherwise functionally equivalent representation (including a simulation model) of a hardware component, circuit, device, or system disclosed herein. Such data and / or instructions may include hardware description language (HDL) design entities or other data structures that conform to and / or are compatible with low-level HDL design languages ​​such as Verilog and VHDL, and / or high-level design languages ​​such as C or C++. Furthermore, the data and / or instructions may employ data formats used for the exchange of integrated circuit layout data and / or symbolic data formats (e.g., information stored in GDSII (GDS2), GL1, OASIS, map files, or any other suitable format for storing such design data structures).

Claims

1. 1. A method of communication in a data processing environment, comprising: receiving, at a communication interface of a second host data processing system, a host command in a first command set from a first host data processing system, said host command specifying a memory access to a memory coupled to said second host data processing system; translating the host command into a second command in a different second command set that emulates coupling of an additional function unit to the communication interface; presenting the second command to a host bus protocol interface of the second host data processing system; and initiating, by the host bus protocol interface, a host bus protocol memory access request specifying the memory access on a system fabric of the second host data processing system based on receipt of the second command.

2. the second command specifies an address; 2. The method of claim 1, wherein said method further comprises fixing a page table entry for said address referenced by said host command in a page frame table of said second host data processing system.

3. the communication interface is a first communication interface; the second host data processing system includes a second communication interface that is one of a plurality of communication participants coupled to the system fabric; the host command is a first host command; 2. The method of claim 1, further comprising: based on the second communications interface receiving the host bus protocol memory access request on the system fabric, issuing a second host command specifying the memory access.

4. 4. The method of claim 3, wherein issuing the second host command comprises issuing the second host command based on an address specified in the host bus protocol memory access request that belongs to a predetermined address range.

5. 4. The method of claim 3, wherein issuing the second host command comprises issuing the second host command to a third host data processing system.

6. 2. The method of claim 1, wherein said converting comprises converting a first address specified in said host command to a second address and placing said second address in said second command.

7. the communications interface includes a first mode of operation supporting attachment of an attached function unit to the second host data processing system, and a second mode of operation supporting coupling of the first host data processing system to the second host data processing system for memory sharing between the hosts; 10. The method of claim 1, further comprising configuring the communications interface in the second mode of operation to support memory sharing between hosts.

8. the host bus protocol memory access request is a read command; The method comprises: the second host data processing system accessing data specified by the host bus protocol memory access request in a system memory of the second host data processing system and returning the data to the communications interface; 2. The method of claim 1, further comprising: said communications interface of said second host data processing system issuing a read response to said first host data processing system, said read response including said data.

9. 9. The method of claim 8, further comprising: a communications interface of the first host data processing system receiving the read responses and translating the read responses into a second, different set of responses that emulate additional memory coupled to the communications interface.

10. the host bus protocol memory access request is a write command; The method comprises: the second host data processing system writing the data specified by the host bus protocol memory access request to a system memory of the second host data processing system; 2. The method of claim 1, further comprising: the communications interface of the second host data processing system issuing a write response to the first host data processing system indicating success of the write command before completion of the write, regardless of success or failure of the write.

11. a communication controller of a second host data processing system including a system fabric, said communication controller comprising: receiving a host command in a first command set from a first data processing system, the host command specifying a memory access to a memory coupled to the second host data processing system; translating the host command into a second command in a different second command set that emulates coupling of an additional function unit to the communications controller; submitting the second command to a host bus protocol interface; and initiating, via the host bus protocol interface, a host bus protocol memory access request specifying the memory access on the system fabric of the second host data processing system based on receipt of the second command.

12. 12. The communications controller of claim 11, wherein said converting comprises converting a first address specified in said host command to a second address and placing said second address in said second command.

13. the communications controller includes a first mode of operation that supports attachment of an attached function unit to the second host data processing system, and a second mode of operation that supports coupling of the second host data processing system to the first data processing system for memory sharing between the hosts; 12. The communications controller of claim 11, wherein the controller circuitry is configured to perform the conversion based solely on a configuration of the communications controller in the second mode of operation to support memory sharing between hosts.

14. the host bus protocol memory access request is a read command; the controller circuit receiving data specified by said host bus protocol memory access request from a system memory of said second host data processing system; 12. The communications controller of claim 11, further configured to: issue a read response including said data to said first data processing system.

15. 15. A system comprising the communication controller of claim 14, wherein the communication controller is a first communication controller; a second communication controller of the first data processing system coupled to the first communication controller of the second host data processing system, the second communication controller a controller circuit configured to receive the read responses and convert the read responses into a second, different set of responses that emulate an additional memory coupled to the second communication controller.

16. A system comprising the communication controller of claim 11, the host bus protocol memory access request is a write command; the controller circuit The system is further configured to: issue a write response to the first data processing system indicating success of the write command, regardless of success or failure of the write command.

17. A system comprising the communication controller of claim 11, the second host data processing system is coupled to the communication controller, the second host data processing system including a system memory storing a page frame table; the second command specifies an address; The system, wherein the second host data processing system is configured to pin, in the page frame table of the system memory, a page table entry for the address referenced by the host command.

18. the host command is a first host command; 12. A system comprising the communication controller of claim 11, wherein the communication controller is a first communication controller; a second host data processing system coupled to the communication controller, the second host data processing system including a second communication controller that is one of a plurality of communication participants coupled to the system fabric, the second communication controller issuing a second host command specifying the memory access based on receiving the host bus protocol memory access request on the system fabric.

19. 20. The system of claim 18, wherein the second communications controller issues the second host command based on an address specified in the host bus protocol memory access request belonging to a predetermined address range.

20. 20. The system of claim 18, wherein the second communications controller issues the second host command to a third host data processing system coupled to the second host data processing system.

21. A computer program comprising program code means adapted to perform the method according to any of claims 1 to 10 when the computer program is run on a computer.

Images