SYSTEM, APPARATUS AND METHOD FOR DIRECT DATA READ FROM MEMORY - Patent application

The interconnect device facilitates direct data reads from memory addresses with managed cache coherency, addressing inefficiencies in existing systems by reducing hops and bandwidth, thus lowering power consumption and costs.

JP2024544809APending Publication Date: 2024-12-05INTEL CORP
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
JP2023573681
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-12-22
Filing Date
2022-10-28
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Existing computing systems face significant delays and bandwidth consumption when processors read data from memory addresses managed by another processor, involving multiple hops and inefficient use of processor interconnects, leading to increased power consumption and costs.

Method used

An interconnect device enables processors to read data directly from memory addresses with cache coherency managed by another processor, handling coherence tasks and reducing the number of hops required, thereby minimizing delay and bandwidth usage.

Benefits of technology

Direct data transfer through the interconnect device reduces delays and bandwidth requirements, lowering power consumption and costs associated with processor interconnect sizing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024544809000001_ABST
    Figure 2024544809000001_ABST
Patent Text Reader

Abstract

In an embodiment, a system may include an interconnect device having first, second, and third ports; a first processor coupled to the first port; a second processor coupled to the second port; and a system memory coupled to the third port. The interconnect device may: receive a speculative read request from the first processor via the first port for a data element stored in the system memory, where coherence of the data element is managed by the second processor; receive a direct read request for the data element, merge the direct read request with the speculative read request, and transmit the data element directly to the first processor via the first port. Other embodiments are described and claimed.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] Embodiments relate to accessing memory in a computing environment. [Background technology]

[0002] A modern computer system may include any number of components, such as a Central Processing Unit (CPU), memory, a chipset, and / or many other devices (e.g., computer buses, networks, etc.), coupled together by an interconnect. The interconnect may transfer data between devices or components within a computer, as well as between computers. For example, an interconnect may be used to read data elements from a memory and provide the data elements to a processor. [Brief description of the drawings]

[0003] [Figure 1] FIG. 1 is a block diagram of an exemplary system according to an embodiment.

[0004] [Diagram 2] FIG. 2 is an illustration of an exemplary process according to an embodiment.

[0005] [Diagram 3] FIG. 2 is an illustration of an exemplary process according to an embodiment.

[0006] [Figure 4A] FIG. 2 is an illustration of an exemplary process according to an embodiment.

[0007] [Figure 4B] FIG. 1 is a block diagram of an exemplary system according to an embodiment.

[0008] [Diagram 5] FIG. 1 is a flow diagram of an example method in accordance with one or more embodiments.

[0009] [Figure 6] FIG. 1 is a block diagram of an exemplary system according to an embodiment.

[0010] [Figure 7] FIG. 1 is a block diagram of an exemplary system according to an embodiment.

[0011] [Figure 8] FIG. 1 is a block diagram of an exemplary system according to an embodiment.

[0012] [Figure 9] FIG. 1 is a block diagram of an exemplary system according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Some computing systems may implement interconnect protocols that provide the ability to dynamically add components such as memory and accelerators. For example, some systems may implement interconnects and switches according to the Compute Express Link (CXL) specification, such as the CXL 1.1 specification (or any future version, modification, variant, or replacement of the CXL specification). An exemplary device that may implement the CXL specification is a memory buffer device (e.g., a memory expander device), which may provide functionality for memory expansion and memory bandwidth expansion. A memory buffer device may include multiple ports and may allow multiple processors to access a shared memory device.

[0014] In some examples, shared memory may be implemented in a distributed manner where various portions of memory are associated with particular processors. For example, each processor may be assigned a particular memory address range and may manage cache coherency of data in that assigned address range. The processor managing the cache coherency may be referred to as the "home processor" (or "home agent") for that particular memory range. However, in such a situation, performing a memory read may involve multiple interactions or "hops" and therefore significant latency and bandwidth consumption. For example, assume that a first processor has a requirement to read data from a memory address assigned to a second processor. In this example, performing a read may require four hops: a first processor sending a first read command over the processor interconnect link to the second processor, the second processor sending a second read command over the CXL link to the memory buffer device, the memory buffer device sending data over the CXL link to the second processor, and the second processor then sending the data over the processor interconnect link to the first processor. Thus, a data read may involve significant latency (e.g., due to the multiple hops) and may require significant bandwidth, power, and cost (e.g., to transfer the data across the processor interconnect links and size the processor interconnect links accordingly).

[0015] In various embodiments described herein, an interconnect device may include circuitry or other logic to enable a processor to directly read data from a memory address that has cache coherency managed by another processor (i.e., the "home" processor). Additionally, the interconnect device may handle coherence tasks associated with a direct read, including maintaining directories and initiating back-invalidate flows. In some embodiments, a direct data transfer may not pass through the home processor and the processor interconnect. Thus, a direct read may be performed in relatively few hops (e.g., two hops), thereby involving less latency and bandwidth than conventional approaches. Additionally, some embodiments may reduce power consumption associated with memory reads and reduce costs associated with sizing the processor interconnect.

[0016] Figure 1 - An example system

[0017] 1, a block diagram of an exemplary system 100 is shown in accordance with one or more embodiments. The system 100 may include an interconnect device 120 coupled to a first processor 130 via a first port 121 and connected to a second processor 135 via a second port 122. Additionally, the interconnect device 120 may be coupled to a system memory 110 via a third port 123 and a fourth port 124. In some embodiments, the interconnect device 120 may be a type 3 CXL device (e.g., a memory buffer device) and a type 2 CXL device, etc. Additionally, the ports 121, 122, 123, 124 may implement a CXL protocol and may be coupled to a CXL interconnect along with corresponding devices 110, 130, 135. The first processor 130 may be connected to the second processor 135 via a processor interconnect 140. In some embodiments, the system 100 may represent a single cache coherency domain.

[0018] The system memory 110 may be implemented with any type of computer memory (e.g., Dynamic Random-Access Memory (DRAM), Static Random-Access Memory (SRAM), Non-Volatile Memory (NVM), a combination of DRAM and NVM, etc.). Each of the first and second processors 130, 135 may be a hardware processing device (e.g., a Central Processing Unit (CPU), a System on a Chip (SoC), etc.) and may include any number of processing circuits (also referred to herein as "processing cores" or "processing engines"). Additionally, the system memory 110 may be formed with a variety of memory devices.

[0019] As shown, interconnect device 120 may include direct read circuitry 125. In some embodiments, direct read circuitry 125 may enable a processor to directly read data from a memory address associated with another processor. For example, assume that a first processor 130 issues a request to read a data element 115 (e.g., a cache line) stored in system memory 110. Further assume that a second processor 135 is the home processor of data element 115 (e.g., second processor 135 is responsible for cache coherence of the address of data element 115). In some embodiments, interconnect device 120 may be a CXL device (e.g., a multi-headed memory buffer device, an accelerator device, an input / output device, etc.).

[0020] In some embodiments, the direct read circuitry 125 may include functionality for performing a direct read via a direct data transfer 155 of the data element 115 to the first processor 130. Additionally, the direct read circuitry 125 may perform coherence tasks associated with a direct read, including maintaining a directory and initiating a back-invalidation flow, etc. Additionally, in some embodiments, the processors 130, 135 and memory 120 may include circuitry or other logic for performing tasks associated with a direct read. In some embodiments, the direct data transfer 155 may not pass through the second processor 135 and the processor interconnect 140. Thus, the direct read may be performed in relatively few hops (e.g., two hops), thereby involving less latency and bandwidth compared to conventional approaches. The direct read functionality of the system 100 is further described below with reference to FIGS. 2-5.

[0021] It should be noted that although some examples are described herein with reference to the Compute Express Link (CXL) specification, embodiments are not limited in this respect. For example, it is contemplated that various embodiments may be used with other coherent interconnect technologies, such as the IBM XBus protocol, the Nvidia NVLink protocol, the AMD Infinity Fabric protocol, the Cache Coherent Interconnect for Accelerators (CCIX) protocol, or the Coherent Accelerator Processor Interface (OpenCAPI).

[0022] Figure 2 - First exemplary process

[0023] Referring now to FIG. 2, an exemplary diagram of a first direct read process 200 according to one or more embodiments is shown. In particular, FIG. 2 illustrates a sequence of data exchanges between a remote processor 201, a home processor 202, an interconnect device 203, and a memory 204. The devices 201, 202, 203, 204 may correspond to exemplary implementations of the devices 130, 135, 120, 110 (shown in FIG. 1), respectively. In some embodiments, the interconnect device 203 may be a CXL multi-headed memory buffer device (e.g., a memory expander device). The operations described below with reference to the remote processor 201 may be performed by a caching agent included in the remote processor 201. Additionally, the operations described below with reference to the home processor 202 may be performed by a home agent included in the home processor 202.

[0024] The process 200 may begin with a remote processor 201 requesting a data element (e.g., a cache line) from a memory 204. In particular, as shown in FIG. 2, the remote processor 201 may send (i.e., transmit) a speculative read message 210 ("MemSpecRd") to the interconnect device 203. The remote processor 201 may simultaneously send a Data Forward Only (DFO) message 220 ("RdOwnDFO") to the home processor 202. In some embodiments, the DFO message 220 requests that the requested data be returned only if it is found in a peer cache (i.e., a cache-to-cache forwarding). If the home processor 202 determines that the requested data is not found in any other peer cache, the home processor 202 may send a clean completion message 225 ("CmpO_Clean") to inform the remote processor 201 that it has exclusive ownership without the data. Thus, the remote processor 201 may send a direct memory read message 230 ("MemRd") to the interconnect device 203. Alternatively, if the requested data is found in a peer cache, the home processor 202 does not send a clean completion message 225 to the remote processor 201 (not shown in FIG. 2). Thus, the remote processor 201 may not be able to complete the direct read. Instead, the remote processor 201 may send a conventional (i.e., does not perform a direct read) read request to the home processor 202.

[0025] 2, in response to receiving speculative read message 210 from remote processor 201, interconnect device 203 may send memory read request 215 ("MRd") to memory 204, thereby causing memory 204 to send message 240 including the requested data to interconnect device 203. Further, upon receiving direct memory read message 230, interconnect device 203 may perform merging 235 of speculative read message 210 and direct memory read message 230. Interconnect device 203 may send message 250 including the requested data (i.e., received in message 240) directly to remote processor 201. In this manner, data may be forwarded directly to remote processor 201. In some embodiments, DFO message 220 and clean completion message 225 may be implemented using the CXL protocol. Additionally, the remaining messages 210, 215, 230, 240, 250, 220 shown in FIG. 2 may be implemented using a processor interconnect protocol (eg, the Intel® Ultra-Path Interconnect (UPI) protocol).

[0026] In some embodiments, remote processor 201 may not block snoops (e.g., from the back invalidation channel) while waiting for a response to direct memory read message 230 (i.e., message 250 containing the requested data). For example, blocking snoops in this way may cause a deadlock condition. Thus, upon receiving a snoop, remote processor 201 may release its exclusive ownership of the requested data and submit a conventional (i.e., not performing a direct read) read request to home processor 202, thereby providing ownership and data simultaneously.

[0027] In some embodiments, process 200 may be performed for an embodiment in which the home agent and memory controller are located on different dies of a silicon tile-based processor, and the caching agent is located on the same die as the memory controller. Such an embodiment may enable the remote agent to read directly from the native memory controller, thereby avoiding passing data over a die-to-die interface. In this manner, such an embodiment may reduce the power and die-to-die bandwidth required for the data flow of the co-located caching agent and memory controller.

[0028] Figure 3 - Second exemplary process

[0029] Referring now to FIG. 3, an exemplary diagram of a second direct read process 300 according to one or more embodiments is shown. As shown, FIG. 3 illustrates a sequence of data exchanges between remote processor 201, home processor 202, interconnect device 203, and memory 204 (also shown in FIG. 2). Devices 201, 202, 203, 204 may correspond to exemplary implementations of devices 130, 135, 120, 110 (shown in FIG. 1), respectively. In some embodiments, interconnect device 203 may be a CXL multi-headed memory buffer device. Operations described below with reference to remote processor 201 may be performed by a caching agent included in remote processor 201. Additionally, operations described below with reference to home processor 202 may be performed by a home agent included in home processor 202.

[0030] The process 300 may begin with the remote processor 201 sending a speculative read message 310 ("MemSpecRd") to the interconnect device 203. The remote processor 201 may simultaneously send a read data message 320 ("RdData") to the home processor 202. Upon receiving the read data message 320, the home processor 202 may send a completion message back to the remote processor 201 and may also send a direct memory read message 330 ("MemRd_Direct") to the interconnect device 203. Additionally, the home processor 202 may allocate or begin tracking the direct memory read message 330. In some embodiments, the direct memory read message 330 may indicate that the requested data will be sent directly from the memory 204 to the remote processor 201. Additionally, the direct memory read message 330 may include one or more fields for storing identification data about the requester (e.g., a requester identifier and address).

[0031] Upon receiving the direct memory read message 330, the interconnect device 203 may match the direct memory read message 330 with the speculative read message 310 using the included identification data. The interconnect device 203 may perform a merge 332 of the direct memory read message 330 and the speculative read message 310, thereby determining the requester that will receive the data. The interconnect device 203 may send a direct read acknowledgement message 335 ("Direct_ACK") to the home processor 202. Upon receiving the direct read acknowledgement message 335, the home processor 202 may deallocate or stop tracking the memory read message 330. In some embodiments, performing the merge 332 may include matching the requester identifier and address in the messages 310, 330, thereby determining the tag that will be returned to the requester. However, if the match does not hold, the merge fails and a conventional read request may be performed instead.

[0032] 3, in response to receiving speculative read message 310 from remote processor 201, interconnect device 203 may send memory read request 315 ("MRd") to memory 204, thereby causing memory 204 to send message 340 containing the requested data to interconnect device 203. After receiving message 340 and after performing merging 332, interconnect device 203 may send direct data message 350 ("Direct_Data") containing the requested data (i.e., received in message 340) directly to remote processor 201. In this manner, data may be transferred directly to remote processor 201. In some embodiments, direct memory read message 330, speculative read message 310, and direct data message 350 may be implemented using the CXL protocol.

[0033] FIG. 4A - Third exemplary process

[0034] 4A, an exemplary diagram of a third direct read process 400 according to one or more embodiments is shown. As shown, FIG. 4A illustrates a sequence of data exchanges between a first remote processor 401, a first home processor 402, an interconnect device 403, a memory 404, a second home processor 405, and a second remote processor 406. The devices 401, 402, 403, 404 may correspond to exemplary implementations of the devices 130, 135, 120, 110 (shown in FIG. 1), respectively. Furthermore, the devices 406, 405 may correspond to exemplary implementations of the devices 130, 135, respectively. In some embodiments, the interconnect device 403 may be a CXL multi-headed memory buffer device. The operations described below with reference to the remote processors 401, 406 may be performed by a caching agent included in the remote processors 401, 406. Additionally, the operations described below with reference to the home processors 402, 405 may be performed by a home agent included in the home processors 402, 405.

[0035] In some embodiments, remote processor 401 and home processor 402 may be included in a first virtual hierarchy (VH) 407. Additionally, remote processor 406 and home processor 405 may be included in a second virtual hierarchy (VH) 408. In some embodiments, each virtual hierarchy may represent one or more devices connected to interconnect device 403 via a particular link and may use a separate cache coherency domain. In some examples, cache coherency within each virtual hierarchy 407, 408 may be managed using a processor interconnect protocol (e.g., the Intel® Ultra-Path Interconnect (UPI) protocol). The interconnect device 403 may include circuitry for resolving coherency across multiple virtual hierarchies. For example, the interconnect device 403 may be a CXL memory buffer device that includes circuitry and / or other logic for coherency adjustment across different coherence domains of the first and second virtual hierarchies 407, 408. In some embodiments, the interconnect device 403 may use back-invalidate commands to resolve coherency.

[0036] 4A, process 400 may begin with a remote processor 401 in a first VH 407 sending a read message 410 to a home processor 402 in the first VH 407. Upon receiving the read message 410, the home processor 402 may send a memory read message 414 to the interconnect device 403. The interconnect device 403 may send a memory read request 416 to the memory 404, thereby causing the memory 404 to send a message 420 containing the requested data to the interconnect device 403.

[0037] Upon receiving message 420, the interconnect device 403 may send a data message 422 (including the requested data) to the home processor 402. Additionally, the interconnect device 403 may update a coherence directory to indicate that the requested data is exclusively owned by the first VH 407 (e.g., "Dir=E@VH407"). In some examples, the coherence directory may be stored in the interconnect device 203 and may be used to adjust or resolve coherency across the first and second VHs 407, 408. Upon receiving data message 422, the home processor 402 may send a completion message 424 to the remote processor 401.

[0038] 4A, a short time after a remote processor 401 in a first VH 407 sends a read message 410, a remote processor 406 in a second VH 408 sends a read message 430 to a home processor 405 in the second VH 408. Assume that the read messages 410, 430 are requesting to read the same memory address. Thus, the read messages 410, 430 may represent a conflict to read the same data by the first and second VHs 407, 408.

[0039] Upon receiving the read message 430, the home processor 405 may send a memory read message 434 to the interconnect device 403. The interconnect device 403 may determine a conflict with the first VH 407 for the requested data (e.g., with its internal coherency directory). Accordingly, the interconnect device 403 may send a back-invalidate snoop message 440 to request that the first VH 407 change the data from an exclusive coherency state to a shared coherency state or release ownership of the data. In some embodiments, the back-invalidate snoop message 440 may implement the CXL.memory protocol.

[0040] Upon receiving the back-invalidate snoop message 440, the home processor 402 may send a snoop message 442 to the remote processor 401. The remote processor 401 may change the data to a shared state and may send a response message 444 indicating that the data is now in a shared state. Upon receiving the response message 444, the home processor 402 may send a back-invalidate ACK 446, thereby informing the interconnect device 403 that the data is now in a shared state. In response, the interconnect device 403 may send a data message 454 (including the requested data) to the home processor 405. Additionally, the interconnect device 403 may update its internal coherence directory to indicate that the requested data is shared by the first VH 407 and the second VH 408 (e.g., "Dir=S@VH407, VH408"). Upon receiving the data message 454, the home processor 405 may send a completion message 456 to the remote processor 406.

[0041] In some embodiments, interconnect device 403 may require that all memory writes have ownership before the write occurs. This requirement may prevent dirty data from being written back to memory. For example, interconnect device 403 may implement the CXL.memory protocol in the back-invalidation channel to resolve coherence across different virtual hierarchies. This process may include flushing dirty data using Request with Data (RwD) commands. In some embodiments, the CXL.memory protocol may require that the snoop attribute be set to no-operation ("No-Op") for RwD commands in the affected memory region. In this manner, RwD commands may be prevented from causing back-invalidation snoops, thereby preventing the writing of dirty data.

[0042] FIG. 4B - An exemplary system

[0043] 4B, a block diagram of an example system 460 according to one or more embodiments is shown. The system 460 may implement the example process 400 (described above with reference to FIG. 4A). As shown, the system 460 may include a first processor 470, a second processor 471, a memory device 472, and an interconnect device 490. In some embodiments, the interconnect device 490 may be a CXL multi-headed memory buffer device. The memory device 472 may be a memory module (e.g., dynamic random access memory (DRAM), static random access memory (SRAM), etc.) or a single-headed memory buffer.

[0044] As shown, the first processor 470 may include a caching agent 472 and a home agent 474 coupled to a multi-level cache hierarchy (e.g., L1 / L2 / L3 caches) and multiple processing cores. Additionally, the second processor 471 may include a caching agent 475 and a home agent 476 coupled to a cache hierarchy and multiple processing cores. The processors 470, 471 may implement the CXL.cache and CXL.io communication protocols. Additionally, the interconnect device 490 may include a memory 492, a cache 496, and a final home agent 494. The caching agents 472, 475, the final home agent 494, and / or the home agents 474, 476 may be implemented, at least in part, in circuitry.

[0045] In some embodiments, process 400 (shown in FIG. 4A ) may be implemented using components of system 460. For example, process 400 may be performed using caching agent 472 in place of remote processor 401, home agent 474 in place of home processor 402, caching agent 475 in place of remote processor 406, home agent 476 in place of home processor 405, and interconnect device 490 in place of interconnect device 403.

[0046] In some embodiments, the processors 470, 471 may be directly coupled by a processor interconnect link 480. The first processor 470 may be coupled to the interconnect device 490 via a first link 482. The first link 482 may form or define a first virtual hierarchy (e.g., VH 407 shown in FIG. 4A). Additionally, the second processor 471 may be coupled to the interconnect device 490 via a second link 484. The second link 484 may form or define a second virtual hierarchy (e.g., VH 408 shown in FIG. 4A). As shown, the second processor 471 may be directly coupled to the memory device 472 via a third link 486. In some embodiments, the first and second links 482, 484 may implement the CXL.memory protocol.

[0047] In some embodiments, home agents 474 and 476 may include the capability to use two coherency management modes for different cache addresses: a hierarchical coherency management mode and a flat coherency management mode. In the hierarchical coherency management mode, home agents 474 and 476 may use a final home agent 494 to coordinate or resolve cache coherency across two different coherency domains used by home agents 474 and 476 (e.g., via links 482 and 484). Furthermore, in the flat coherency management mode, each of home agents 474 and 476 may separately manage cache coherency in its respective coherency domain. For example, home agent 476 may manage coherency in memory device 472 using the flat coherency management mode. In some embodiments, each of home agents 474 and 476 may include two snoop lists: a first snoop list that identifies cache locations for which a hierarchical coherency management mode is used, and a second snoop list that identifies cache locations for which a flat coherency management mode is used.

[0048] In some embodiments, the first processor 470 may be coupled to additional processors (not shown in FIG. 4B) to form a first group of processors. In such an example, the home agent 474 may manage coherency for the first group of processors and enable the first group of processors to use a hierarchical coherency management mode (e.g., coordinate cache coherency across different coherency domains with the final home agent 494). Similarly, the second processor 471 may be coupled to other processors (not shown in FIG. 4B) to form a second group of processors. In such an example, the home agent 476 may manage coherency for the second group of processors and enable the second group of processors to use a hierarchical coherency management mode.

[0049] FIG. 5 - Exemplary method

[0050] Referring now to Figure 5, a flow diagram of a method 500 according to one or more embodiments is shown. In various embodiments, the method 500 may be performed by processing logic (e.g., the processors 130, 135 and / or interconnect devices shown in Figure 1), which may include hardware (e.g., a processing device, circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. In firmware or software embodiments, the method 500 may be implemented by computer-executable instructions stored on a non-transitory machine-readable medium, such as an optical, semiconductor, or magnetic storage device. The machine-readable medium may store data that, when used by at least one machine, causes the at least one machine to fabricate at least one integrated circuit to perform the method.

[0051] Block 510 may include an interconnect device receiving, from a first processor, a speculative read request for a data element stored in a system memory, where coherence of the data element is managed by a second processor. For example, with reference to Figures 2-3, remote processor 201 may send speculative read message 210 (or speculative read message 310) to interconnect device 203.

[0052] Referring again to Figure 5, block 520 may include an interconnect device receiving a read request for the data element. For example, referring to Figure 2, remote processor 201 may send direct memory read message 230 to interconnect device 203. In another example, referring to Figure 3, home processor 202 may send direct memory read message 330 to interconnect device 203.

[0053] 5, block 530 may include the interconnect device merging the read request with the speculative read request. For example, referring to FIG. 2, the interconnect device 203 may perform merging 235 of the speculative read message 210 and the direct memory read message 230. In another example, referring to FIG. 3, the interconnect device 203 may perform merging 332 of the direct memory read message 330 and the speculative read message 310.

[0054] Referring again to Figure 5, block 540 may include the interconnect device transmitting the data element directly to the first processor. For example, referring to Figure 2, the interconnect device 203 may send a message 250 containing the requested data directly to the remote processor 201. In another example, referring to Figure 3, the interconnect device 203 may send a direct data message 350 containing the requested data directly to the remote processor 201. After block 540, the method 500 may be complete.

[0055] Figure 6 - Exemplary system

[0056] 6, an embodiment of an SoC design according to an embodiment is shown. As a specific illustrative example, SoC 600 may be configured for insertion into any type of computing device, ranging from portable devices to server systems. Here, SoC 600 includes two cores 606 and 607. Cores 606 and 607 may conform to an instruction set architecture such as an Intel® Architecture Core™ type processor, an Advanced Micro Devices (AMD) processor, a MIPS type processor, an ARM type processor design, or those of their customers, licensees or adopters. Cores 606 and 607 are coupled to a cache controller 608 associated with a bus interface unit 609 and an L2 cache 610 for communication with other parts of system 600 via interconnect 612.

[0057] The interconnect 612 provides communication channels to other components such as a subscriber identity module (SIM) 630 for interfacing with a SIM card, a boot ROM 635 for holding boot code for initializing and booting the SoC 600 executed by the cores 606 and 607, an SDRAM controller 640 for interfacing with external memory (e.g., DRAM 660), a flash controller 645 for interfacing with non-volatile memory (e.g., flash 665), a peripheral controller 650 (e.g., an eSPI interface) for interfacing with peripherals, a video codec 620 and a video interface 625 for displaying and receiving input (e.g., touch-enabled input), a GPU 615 for performing graphics-related computer processing, etc. In addition, the system shows peripherals for communication such as a Bluetooth module 670, a 3G modem 675, a GPS 680, and a Wi-Fi 685. The system also includes a power controller 655, which may implement the circuitry described herein for performing CXL-based power sharing according to an embodiment. As further shown in FIG. 6, the system 600 may additionally include interfaces, including, for example, a MIPI interface 692 to a display and / or an HDMI interface 695 that may be coupled to the same or a different display.

[0058] Figure 7 - Exemplary system

[0059]

[0031] Referring now to Figure 7, a block diagram of a system, such as an edge platform, is shown, according to another embodiment. As shown in Figure 7, a multiprocessor system 700 includes a first processor 770 and a second processor 780 coupled via an interconnect 750, which in an embodiment may be an optical interconnect in communication with optical circuitry (which may be included in or coupled to processor 770). As shown in Figure 7, each of processors 770 and 780 may be a multiple core processor, including representative first and second processor cores (i.e., processor cores 774a and 774b, and processor cores 784a and 784b).

[0060] 7, processors 770 and 780 further include point-to-point interconnects 777 and 787 that couple to switches 759 and 760 via interconnects 742 and 744 (which may be CXL buses). Switches 759, 760 in turn couple to pooled memories 755 and 765.

[0061] Still referring to FIG. 7 , the first processor 770 further includes a memory controller hub (MCH) 772 and point-to-point (PP) interfaces 776 and 778. Similarly, the second processor 780 includes an MCH 782 and PP interfaces 786 and 788. As shown in FIG. 7 , the MCHs 772 and 782 couple the processors to their respective memories, i.e., memory 732 and memory 734, which may be portions of system memory (e.g., DRAM) locally attached to the respective processors. The first processor 770 and the second processor 780 may be coupled to a chipset 790 via PP interconnects 776 and 786, respectively. As shown in FIG. 7 , the chipset 790 includes PP interfaces 794 and 798.

[0062] Further, chipset 790 includes an interface 792 that couples chipset 790 to a high performance graphics engine 738 by a PP interconnect 739. As shown in Figure 7, various input / output (I / O) devices 714 may be coupled to the first bus 716 along with a bus bridge 718 that couples the first bus 716 to a second bus 720. Various devices may be coupled to the second bus 720 including, for example, a keyboard / mouse 722, a communications device 726, and a data storage unit 728, such as a disk drive or other mass storage device, which in one embodiment may include code 730. Further, audio I / O 724 may be coupled to the second bus 720.

[0063] Figure 8 - Exemplary system

[0064] The embodiments described herein may be used within a variety of network architectures. Referring now to Figure 8, a block diagram of a network architecture according to another embodiment of the present invention is shown. As shown in Figure 8, the network architecture 800 includes a variety of computing platforms that may be located over a very wide area.

[0065] In the high level diagram of Figure 8, network architecture 800 includes an exemplary device 810, such as a smartphone, that may communicate over different radio access networks (RANs), including RAN 820 and RAN 830. RAN 820 may then be coupled to platform 825, which may be an edge platform, such as a fog / far / near edge platform, and may utilize embodiments herein. Other requests may be handled by far edge platform 835 coupled to RAN 830, which may also utilize embodiments.

[0066] As further shown in Figure 8, another near edge platform 840 may be coupled to the RANs 820, 830. Note that this near edge platform may be located closer to a data center 850 that may have a large amount of computing resources. Pushing messages to these more remote platforms will result in higher latency in handling requests on behalf of the edge device 810. It should be understood that one or more of the platforms shown in Figure 8 may incorporate embodiments for performing CXL based power sharing as described herein.

[0067] Figure 9 - Exemplary system

[0068]

[0036] Referring now to Figure 9, a block diagram of a system 900 according to another embodiment of the present invention is shown. As shown in Figure 5, the system 900 may be any type of computing device, and in one embodiment, may be a server system. In the embodiment of Figure 9, the system 900 includes multiple CPUs 910a,b, which in turn couple to respective system memories 920a,b, which in an embodiment may be implemented as DIMMs, such as Double Data Rate (DDR) memory, persistent memory, or other types of memory. It should be noted that the CPUs 910 may be coupled together via an interconnect system 915, such as an Intel® UltraPath Interconnect, or other processor interconnect technology.

[0069] There may be multiple interconnects 930a1-b2 to allow coherent accelerator devices and / or smart adapter devices to couple to the CPU 910 via potentially multiple communication protocols. In an embodiment, each interconnect 930 may be a given instance of CXL.

[0070] In the illustrated embodiment, each CPU 910 couples to a corresponding Field Programmable Gate Array (FPGA) / accelerator device 950a,b (which may include a GPU in one embodiment). In addition, the CPU 910 also couples to a smart NIC device 960a,b. The smart NIC device 960a,b then couples to a switch 980a,b (e.g., a CXL switch according to an embodiment), which in turn couples to a pooled memory 990a,b, such as a persistent memory. In an embodiment, the switch 980 may implement the circuitry described herein to perform the methods described herein. Of course, the embodiments are not limited to switches, and the techniques described herein may be performed by other entities of the system, including the CPU 910.

[0071] The following sections and / or examples relate to further embodiments.

[0072] In example 1, a system for data transfer may include: an interconnect device having first, second, and third ports; a first processor coupled to the first port; a second processor coupled to the second port; and a system memory coupled to the third port. The interconnect device may: receive a speculative read request for a data element stored in the system memory from the first processor via the first port, where coherence of the data element is managed by the second processor, receive a direct read request for the data element, merge the direct read request with the speculative read request, and transmit the data element directly to the first processor via the first port.

[0073] In Example 2, the subject matter of Example 1 may optionally include the interconnect device: in response to receiving the speculative read request, sending a memory read request to the system memory via the third port; and receiving the data element from the system memory via the third port.

[0074] In Example 3, the subject matter of Examples 1-2 may optionally include, where the interconnect device is to receive a read request directly from the first processor via the first port.

[0075] In Example 4, the subject matter of Examples 1-3 may optionally include the interconnect device: receiving a direct read request from the second processor via the second port; and transmitting an acknowledgment of the direct read request to the second processor via the second port.

[0076] In Example 5, the subject matter of Examples 1-3 may optionally include, in which the interconnect device merges the direct read request with the speculative read request based on a comparison of at least one identifier included in the direct read request and the speculative read request.

[0077] In Example 6, the subject matter of Examples 1-5 may optionally include: the first processor sending a read data message to the second processor; the second processor sending a completion message to the first processor in response to receiving the read data message; and the first processor sending a read request directly to the interconnect device in response to receiving the completion message.

[0078] In Example 7, the subject matter of Examples 1-6 may optionally include, where the first processor and the second processor are coupled by a processor interconnect link, and where the data element does not pass through the processor interconnect link.

[0079] In Example 8, the subject matter of Examples 1-7 may optionally include, wherein the interconnect device comprises a multi-headed memory buffer device.

[0080] In Example 9, the subject matter of Examples 1-8 may optionally include, wherein the direct read request and the speculative read request comprise a Compute Express Link (CXL) protocol message.

[0081] In example 10, an apparatus for data transfer may include: a first interconnect port for coupling to a first processor; a second interconnect port for coupling to a second processor; a third interconnect port for coupling to a system memory; and a direct read circuit. The direct read circuit may be to: receive a speculative read request from the first processor for a data element stored in the system memory, where coherence of the data element is managed by the second processor, receive a direct read request for the data element, merging the direct read request with the speculative read request, and transmit the data element directly to the first processor via the first interconnect port.

[0082] In Example 11, the subject matter of Example 10 may optionally include, wherein the apparatus comprises a multi-head memory buffer device.

[0083] In Example 12, the subject matter of Examples 10-11 may optionally include the direct read circuitry: in response to receiving the speculative read request, sending a memory read request to the system memory; and receiving the data element from the system memory.

[0084] In Example 13, the subject matter of Examples 10-12 may optionally include, where the direct read circuitry is to receive a direct read request from the first processor via the first interconnect port.

[0085] In Example 14, the subject matter of Examples 10-13 may optionally include the direct read circuitry: receiving a direct read request from the second processor via the second interconnect port; and transmitting an acknowledgment of the direct read request to the second processor.

[0086] In Example 15, the subject matter of Examples 10-14 may optionally include, where the direct read circuitry is to merge the direct read request with the speculative read request based on a comparison of at least one identifier included in the direct read request and the speculative read request.

[0087] In Example 16, the subject matter of Examples 10-15 may optionally include, wherein the direct read request and the speculative read request comprise a Compute Express Link (CXL) protocol message.

[0088] In Example 17, a system for data transfer may include: a first processor associated with a first coherency domain; a second processor associated with a second coherency domain; and an interconnect device coupled to the first processor, the second processor, and a system memory, where each of the first processor and the second processor has coherency circuitry for selecting from a hierarchical coherency management mode and a flat coherency management mode, where the hierarchical coherency management mode includes coordination of coherency across the first coherency domain and the second coherency domain by the interconnect device; and where the flat coherency management mode includes management of coherency in a single coherency domain by the coherency circuitry.

[0089] In Example 18, the subject matter of Example 17 may optionally include the interconnect device: receiving a direct read request for the data element from the first processor; adjusting coherency of the data element across the first coherency domain and the second coherency domain; sending a back-invalidation snoop message to modify a coherency state of the data element in at least one of the first coherency domain and the second coherency domain; and transmitting the data element directly to the first processor via the first interconnect port.

[0090] In Example 19, the subject matter of Examples 17-18 may optionally include the interconnect device having a Compute Express Link (CXL) multi-headed memory buffer device, and the interconnect device updating an internal directory to indicate a coherency state of the data elements in at least one of the first coherency domain and the second coherency domain.

[0091] In Example 20, the subject matter of Examples 17-19 may optionally include each of the first processor and the second processor having: a first list for identifying cache locations associated with a hierarchical coherency management mode; and a second list for identifying cache locations associated with a flat coherency management mode.

[0092] In Example 21, an apparatus for data transfer may include: means for receiving a speculative read request for a data element stored in a system memory; means for receiving a direct read request for the data element; means for merging the direct read request with the speculative read request; and means for directly transmitting the data element to a first processor via a first port.

[0093] In Example 22, the subject matter of Example 21 may optionally include: means for transmitting a memory read request in response to receiving the speculative read request; and means for receiving a data element.

[0094] In Example 23, the subject matter of Examples 21-22 may optionally include: means for receiving a direct read request; and means for transmitting an acknowledgment of the direct read request.

[0095] In Example 24, the subject matter of Examples 21-23 may optionally include means for merging the direct read request with the speculative read request based on a comparison of at least one identifier included in the direct read request and the speculative read request.

[0096] In Example 25, the subject matter of Examples 21-24 may optionally include, wherein the direct read request and the speculative read request comprise a Compute Express Link (CXL) protocol message.

[0097] In various embodiments described herein, an interconnect device may include circuitry or other logic to enable a processor to directly read data from a memory address that has cache coherency managed by another processor (i.e., the "home" processor). Additionally, the interconnect device may handle coherency tasks associated with a direct read, including maintaining directories and initiating back-invalidation flows. In some embodiments, a direct data transfer may not pass through the home processor and the processor interconnect. Thus, a direct read may be performed in relatively few hops (e.g., two hops), thereby involving less latency and bandwidth than conventional approaches. Additionally, some embodiments may reduce power consumption associated with memory reads and reduce costs associated with sizing the processor interconnect.

[0098] It should be noted that while Figures 1-9 depict various exemplary implementations, other variations are possible. For example, the examples depicted in Figures 1-9 are provided for illustrative purposes and are not intended to limit any embodiment. In particular, although the embodiments may be shown in simplified form for clarity, the embodiments may include any number and / or arrangement of components. For example, it is contemplated that some embodiments may include any number of components in addition to those depicted, and that different arrangements of the depicted components may occur in a particular implementation. Additionally, it is contemplated that details in the examples depicted in Figures 1-9 may be used anywhere in one or more embodiments.

[0099] It should be understood that various combinations of the above examples are possible. The embodiments may be used in many different types of systems. For example, in one embodiment, a communication device may be arranged to perform various methods and techniques described herein. Of course, the scope of the present invention is not limited to communication devices, and instead other embodiments may be directed to other types of apparatus for processing instructions, or one or more machine-readable media that include instructions that, in response to being executed on a computing device, cause the device to perform one or more of the methods and techniques described herein.

[0100] References throughout this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one implementation encompassed by the present invention. Thus, appearances of the phrases "one embodiment" or "in an embodiment" do not necessarily refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic may be provided in other suitable forms other than the particular embodiment illustrated, and all such forms may be encompassed within the scope of the claims of this application.

[0101] While the present invention has been described with respect to a limited number of embodiments, those skilled in the art will recognize modifications and variations therefrom, and it is intended that the appended claims cover all such modifications and variations that fall within the true spirit and scope of the present invention.

Claims

1. an interconnect device having first, second and third ports; a first processor coupled to the first port; a second processor coupled to the second port; and a system memory coupled to the third port; the interconnect device comprising: receiving a speculative read request for a data element stored in the system memory from the first processor via the first port, where coherence of the data element is managed by the second processor; receiving a direct read request for the data element; merging the direct read request with the speculative read request; and directly transmitting the data element to the first processor via the first port. This means the system.

2. The interconnect device comprises: in response to receiving the speculative read request, sending a memory read request to the system memory via the third port; and receiving the data element from the system memory via the third port; The system of claim 1 .

3. The interconnect device comprises: receiving the direct read request from the first processor via the first port; The system of claim 1 .

4. The interconnect device comprises: receiving the direct read request from the second processor via the second port; and Transmitting an acknowledgment of the direct read request to the second processor via the second port. The system of claim 1 .

5. The interconnect device comprises: merging the direct read request with the speculative read request based on a comparison of at least one identifier including the direct read request and the speculative read request; The system of claim 1 .

6. The first processor will send a read data message to the second processor; the second processor, in response to receiving the read data message, will send a completion message to the first processor; and the first processor, in response to receiving the completion message, sends the direct read request to the interconnect device. The system of claim 1 .

7. 2. The system of claim 1, wherein the first processor and the second processor are coupled by a processor interconnect link, and the data element does not pass through the processor interconnect link.

8. The system of claim 1 , wherein the interconnect device comprises a multi-headed memory buffer device.

9. The system of claim 1 , wherein the direct read request and the speculative read request comprise Compute Express Link (CXL) protocol messages.

10. a first interconnect port for coupling to the first processor; a second interconnect port for coupling to a second processor; a third interconnect port for coupling to a system memory; and receiving a speculative read request from the first processor for a data element stored in the system memory, where coherence of the data element is managed by the second processor; receiving a direct read request for the data element; merging the direct read request with the speculative read request; and Transmitting the data element directly to the first processor via the first interconnect port. Direct readout circuit for An apparatus comprising:

11. The apparatus of claim 1 , wherein the apparatus comprises a multi-headed memory buffer device.

12. The direct read circuitry comprises: sending a memory read request to the system memory in response to receiving the speculative read request; and Receive the data element from the system memory. The apparatus according to claim 10,

13. The direct read circuitry comprises: receiving the direct read request from the first processor via the first interconnect port; The apparatus according to claim 10,

14. The direct read circuitry comprises: receiving the direct read request from the second processor via the second interconnect port; and Transmitting an acknowledgment of the direct read request to the second processor. The apparatus according to claim 10,

15. The direct read circuitry comprises: merging the direct read request with the speculative read request based on a comparison of at least one identifier including the direct read request and the speculative read request; The apparatus according to claim 10,

16. The apparatus of claim 10 , wherein the direct read request and the speculative read request comprise Compute Express Link (CXL) protocol messages.

17. a first processor associated with the first coherency domain; a second processor associated with the second coherency domain; and an interconnect device coupled to the first processor, the second processor, and the system memory; Equipped with wherein each of the first processor and the second processor has coherency circuitry for selecting between a hierarchical coherency management mode and a flat coherency management mode; wherein the hierarchical coherency management mode comprises coordination of coherency across the first coherency domain and the second coherency domain by the interconnect device; and wherein the flat coherency management mode comprises management of coherency in a single coherency domain by the coherency circuitry. system.

18. The interconnect device comprises: receiving a direct read request for a data element from the first processor; coherency of the data elements across the first coherency domain and the second coherency domain; sending a back-invalidation snoop message to modify the coherency state of the data element in at least one of the first coherency domain and the second coherency domain; and Transmitting the data element directly to the first processor via the first interconnect port. The system of claim 17 .

19. 20. The system of claim 18, wherein the interconnect device comprises a Compute Express Link (CXL) multi-headed memory buffer device, and wherein the interconnect device updates an internal directory to indicate a coherency state of the data elements in at least one of the first coherency domain and the second coherency domain.

20. Each of the first processor and the second processor: a first list for identifying cache locations associated with the hierarchical coherency management mode; and a second list for identifying cache locations associated with the flat coherency management mode; 20. The system of claim 17, comprising:

Citation Information

Patent Citations

  • Systems and methods for providing remote prefetch buffers

    JP2009537921A

  • Lifetime of a read transaction tracker in a coherent interconnect system

    JP2020506482A

  • Data Processing

    JP2020514878A

  • Cache control in the presence of speculative read operations

    JP2021520545A

  • Apparatus and method for routing access requests in an interconnect

    US11086802B1