SYSTEM, APPARATUS, AND METHOD FOR DYNAMICALLY PROVIDING A COHERENT MEMORY DOMAIN - Patent application

Abstract

To provide systems, apparatus and methods for dynamically providing coherent memory domains.SOLUTION: A switch 210 includes: a table to store a plurality of entries, each entry identifying a memory domain of a system and a coherency status of the memory domain; and a control circuit coupled to the table. The control circuit is configured to receive a request to change a coherency status of a first memory domain of the system, and dynamically update a first entry of the table for the first memory domain to change the coherency status between a coherent memory domain and a non-coherent memory domain, in response to the request.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] FIELD Embodiments relate to controlling coherency in a computing environment. [Background technology]

[0002] In modern enterprise systems, memory can be implemented in a distributed manner, with different memory ranges assigned to specific devices. In such systems, it is possible to statically specify the processing entities and memory ranges that form a coherence domain. However, this approach does not scale, especially when trying to increase the number of coherent entities, as coherency communications can result in undesirable latencies. Increasing the number of coherent entities can increase these coherency communications many times over, leading to bottlenecks and other performance issues. [Brief explanation of the drawings]

[0003] [Figure 1] FIG. 1 is a block diagram of a portion of a data center architecture according to an embodiment.

[0004] [Figure 2] FIG. 2 is a block diagram of a switch according to an embodiment.

[0005] [Figure 3] FIG. 1 is a flow diagram of a method according to an embodiment.

[0006] [Figure 4] FIG. 4 is a flow diagram of a method according to another embodiment.

[0007] [Figure 5] FIG. 2 is a block diagram of a system according to another embodiment of the present invention.

[0008] [Figure 6]FIG. 1 is a block diagram of an embodiment of an SoC design according to an embodiment.

[0009] [Figure 7] FIG. 2 is a block diagram of a system according to another embodiment of the present invention.

[0010] [Figure 8] FIG. 1 is a block diagram of a network architecture according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] In various embodiments, a system may have a dynamically configurable memory space that includes multiple independent memory domains, each of which is dynamically created and updated. Furthermore, each of these independent memory domains may be dynamically controlled to be coherent or non-coherent, and the coherency status may be dynamically updated. To this end, switching circuitry within the system, such as a switch coupling multiple processors, devices, memories, etc., may be configured to dynamically assign memory ranges to a given memory domain. Furthermore, if the memory domain is indicated to have a coherent status, the switch may maintain and implement a coherency mechanism. Thus, the switching circuitry may dynamically handle incoming memory requests differently depending on whether the request is destined for a coherent or non-coherent memory domain. Furthermore, the switching circuitry may handle coherency operations differently, for example, depending on traffic conditions within the system. For example, a coherent memory domain may be assigned and associated with one or more fallback rules to correspond to a different coherency mechanism to be used when high traffic conditions exist.

[0012] Although embodiments are not limited in this respect, an exemplary cloud-based edge architecture may communicate using interconnects and switches according to a Compute Express Link (CXL) specification, such as the CXL 1.1 specification, or any future version, modification, variation, or replacement for the CXL specification. Additionally, while the exemplary embodiments described herein relate to CXL-based technology, embodiments may be used with other coherent interconnect technologies, such as the IBM XBus protocol, the Nvidia NVLink protocol, the AMD Infinity Fabric protocol, the Cache Coherent Interconnect for Accelerators (CCIX) protocol, or the Coherent Accelerator Processor Interface (OpenCAPI).

[0013] Many systems provide a single coherent memory domain such that all computing devices (e.g., multiple processor sockets) and add-on devices (such as accelerators or others) reside within the same coherent domain. Such a configuration can be effective for enabling shared computing and shared memory between processors. However, increasing the number of coherent agents also increases the amount of coherence traffic. As an example, adding four processor sockets to a system to go from a four-socket system to an eight-socket system can triple the coherence traffic. This can have a negative impact on latency, and further increasing the number of sockets further increases this traffic. This is especially true when considering add-on devices and accelerators that may be part of this single coherent memory domain.

[0014] Thus, embodiments enable dynamic and fine-grained control of memory coherency. In embodiments, shared coherence domain-based protocols can communicate over the CXL interconnect in a flexible and scalable manner. As a result, multiple servers or racks can communicate over the CXL switch using CXL.memory with cache or CXL.mem semantics. In embodiments, applications can dynamically and independently enforce coherency using CXL.cache semantics.

[0015] If coherency is disabled for a memory device attached via a CXL link, the memory device can be non-coherent and local only. As an example, an add-on memory or add-on accelerator with an add-on memory expansion card can be configured (1) in a "device bias" mode, such that it is non-coherent with respect to any other entity and is used exclusively by the device, or (2) in a "host bias" mode, such that it is globally coherent with respect to the rest of the platform.

[0016] In a cloud server implementation, such as a multi-tenant data center, the system may have multiple coherency domains, such as per-tenant coherency. As an example, each of multiple (possibly many different) tenants may be associated with a memory domain (or multiple memory domains). Note that these separate memory domains may be isolated from one another such that a first tenant assigned to a first memory domain cannot access a second memory domain assigned to a second tenant, and vice versa. In other cases, there may be a more flexible relationship between tenants and memory domains. In an embodiment, coherency domains are managed on a per-tenant basis.

[0017] One exemplary implementation may relate to a database server or database management system configured to operate on a cloud-based architecture. In such a system, multiple nodes may be implemented. In this case, at least some of the nodes may have a segment called main memory that is read-only and therefore does not require coherence. This main memory may consume a large percentage (e.g., 50%) of the total memory capacity used by the database. While other sections of the database may require coherence for certain transactions, embodiments may provide a fine-grained and flexible mechanism for applications to define coherence requirements. Thus, this dynamic and flexible approach provided by embodiments differs from fixed, predefined hard partitioning at the node level or memory region level.

[0018] To achieve this configuration, embodiments provide a mechanism that exposes an application or other requestor the ability to dynamically configure and update the coherency status, among other aspects, of a memory domain. For example, when allocating a memory region such as main memory that does not require coherency, an application can specify a memory allocation request as follows: cxl-mmap([A,B], allocate, 800GB, NULL<coherence>, NULL<callback>). According to this exemplary memory allocation request, the requestor provides information about the memory range request type (allocation request), the amount of space requested, the coherency status, and indicators for callback information (all of which are inactive for this particular request).

[0019] Meanwhile, while allocating the memory region to be used for a transaction, an application can specify coherence and further define the entities that are allowed access to this coherent memory domain (e.g., in terms of process address space identifiers (PASIDs) (e.g., PASID2, PASID3, and PASID5)). This is shown in the following memory allocation request: cxl-mmap([C,D], allocate, 100GB, PASID2, PASID3, PASID5, NULL<callback>). Note that to further provide per-tenant coherency, a memory domain can be associated with a tenant ID that can be mapped to one or more PASIDs. Note that in some implementations, a "tenant" can be defined as one instance of a whole process. Embodiments may enable defining a coherent domain as one of the following two options: (1) an ID (tenant ID) that includes a PASID tuple; or (2) PASID granularity (identifiable by tenant ID and PASID).

[0020] After the transaction is complete, coherence can also be turned off using the same memory allocation request, but using a modify indicator instead of an allocate indicator: cxl-mmap([C,D],modify,100GB,NULL<coherence>,NULL<callback>). The same mechanism can be used to later turn coherence on, for example to update coherence for only PASID5: cxl-mmap([C,D],modify,100GB,PASID5,NULL<callback>).

[0021] As further shown above, these memory allocation and update requests may include an extension called "callback" that can be used to specify CXL-style callback rules. These rules may implement fallback operations for handling coherency when one or more links are saturated. This is similar to a backoff mechanism for locking, where, for example, if a lock is not acquired, another code path or option is taken. As an example, a callback option may request the use of a software multi-phase commit protocol to enforce coherence when a switch generates a callback signal indicating that a coherence operation has saturated the interconnect: cxl-mmap([C,D],modify,100GB,PASID5,CALL-BACK CODEPATH *swcommitprotocol(C,D,PASID5)).

[0022] Another option for callback could be Quality of Service, where if the interconnect becomes saturated, a given PASID (e.g., PASID2) receives the following high-priority / specialized switch credits (e.g., PASID2 is performing the main coherence request operations, while PASID3 and PASID5 simply gather statistical analysis or perform garbage collection): cxl-mmap([C,D],modify,100GB,PASID5,CALL-BACK QOS PASID2).

[0023] Referring now to FIG. 1, a block diagram of a portion of a data center architecture according to an embodiment is shown. As shown in FIG. 1, system 100 can be a collection of components implemented as one or more servers in a data center. As shown, system 100 includes switch 110, such as a CXL switch according to an embodiment. In another implementation, switch 110 can be another type of coherent switch. In either case, however, switch 110 is implemented as a coherent switch rather than an Ethernet-type switch. Acting as a fabric, switch 110 enables various components to communicate with each other, including one or more central processing units (CPUs) 120, 160, one or more specialized function units, such as graphics processing unit (GPU) 150, and network interface circuit (NIC) 130. More specifically, these devices, which can each be implemented as one or more integrated circuits, are dedicated to performing functions that communicate with other functions in other devices via one of several CXL communication protocols. For example, CPU 120 can communicate with NIC 130 via the CXL.io communication protocol. Meanwhile, CPUs 120, 160 may communicate with GPU 150 via the CXL.mem communications protocol. And, by way of example, CPUs 120, 160 may communicate with each other and from CPU 160 to GPU 150 via the CXL.cache communications protocol. Switch 110 may include control circuitry that allows different memory domains to be dynamically allocated and updated (including coherency status) for devices and applications or services. For example, different processes may require coherency over certain memory ranges, while other processes may not require coherency at all.

[0024] 1, system memory may be formed by various memory devices. In the illustrated embodiment, pooled memory 160 is coupled to switch 110. Various components may access pooled memory 160 through switch 110. Additionally, portions of system memory may be directly coupled to particular components. As shown, memory device 170 is coupled to various regions directly to corresponding CPUs 120, 160, NIC 130, and GPU 150. 0‐3 is distributed.

[0025] 1, depending on memory allocation requests issued by processes, various coherent and non-coherent memory domains may be maintained within memory 170. While the embodiment of FIG. 1 is shown at this high level, it should be understood that various modifications and alternatives are possible.

[0026] Through an interface according to an embodiment, software (eg, a system stack) can dynamically specify these types of memory domains. In one embodiment, a memory domain consists of a set of memory regions with an address range, a list of PASIDs associated with the memory domain, and a coherency type (e.g., coherent, non-coherent, read-only, etc.). Device-level (e.g., GPU and CPU) memory domains can also be defined. In other cases, a memory domain can be mapped within a single address range, where a tenant can have multiple memory domains.

[0027] Circuitry within the switch may implement the coherency domain described above. To this end, the circuitry may be configured to intercept snoops and other CXL.cache flows and determine whether they need to pass through the switch. If not, it may return a corresponding CXL.cache response informing the snoop requestor that the address for that request is not hosted on the target platform or device.

[0028] It should be noted that the dynamic coherent memory domains described herein can be implemented without any modification to any coherency agents (such as a caching agent (CA) in a CPU).

[0029] Referring now to FIG. 2, a block diagram of a switch according to an embodiment is shown. As shown in FIG. 2, switch 200 includes various circuits, including ingress circuitry 212 through which incoming requests are received and egress circuitry 219 through which outgoing communications are transmitted. For purposes of describing the dynamic coherency mechanisms herein, switch 210 further includes a configuration interface 214 that may expose the functionality herein to applications, including the ability to dynamically instantiate and update coherent memory domains. To determine whether an incoming request is for a coherent domain, coherency circuitry 220 may utilize information in system address decoder 218, which may decode the received system address in the request.

[0030] 2, coherency circuitry 220 includes caching agent (CA) circuitry 222, which may perform snooping and other coherency operations. More specifically, when control circuitry 224 determines that a request should be processed coherently, it may register CA circuitry 222 to perform the coherency operations. This determination may be based, at least in part, on information maintained in telemetry circuitry 226, which may track traffic through the system, including interconnect bandwidth levels.

[0031] 2, a rules database 230 is provided within switch 210 that may store information regarding different memory domains. As shown, rules database 230 includes multiple entries, each associated with a given memory domain. As shown, each entry includes multiple fields, including a rule ID field, a memory range field, a PASID list field, a device list field, a callback field, and a coherency status field. These different fields may be populated in response to memory allocation requests and may be further updated in response to additional requests, such as for updates.

[0032] Embodiments may be applied to multi-tenant applications in cloud and edge computing and cloud-native applications, including many fine-grained services that do not have global coherence. For purposes of further illustration, multiple independent CXL coherence domains associated with different tenants may be isolated in system memory. For example, you may have an application that deploys containers or virtual machines that specifies the following domains: Domain 1 - VMs A, B, C = Compute devices S1, S2, S3, A3 sharing memory range [x,y] Domain 2 - VM D, E = Computing devices S3, S4, S5, A4 that share memory range [z,t] Domain 3 - Shared memory between VMs C and D - All compute devices App A generates snoop-at-mark X1[x,y], and the CXL switch only snoops S1, S2, S3, and A3.

[0033] As shown in Figure 2, these different memory domains shared between platforms are not coherent across all computing devices. As shown above by domains 1, 2, and 3, a snoop target set is specified for each memory range. Furthermore, some areas of memory, such as the main memory of a database, which may account for the majority of memory capacity usage, may be read-only. There is no need to snoop or maintain coherence for such defined areas.

[0034] According to this configuration, switch 210 can provide coherency quality of service (QoS) between and within coherent domains. In this manner, switch 210 exposes interfaces that can be used (1) by infrastructure owners to specify what coherent QoS (in terms of priority or coherent transactions per second) is associated with each coherent domain, and (2) by coherent domain owners to specify what level of QoS is associated between coherency flows between each participant in the domain.

[0035] Active telemetry coherency saturation awareness is provided via telemetry circuitry 226, allowing the software stack to recognize how accesses to different objects within a coherent domain may incur performance degradation. In one embodiment, the telemetry circuitry 226 may track the saturation of each participant in the domain and the various paths between the various objects, and notify each based on the monitoring rules provided.

[0036] In one embodiment, to enforce flow monitoring and quality of service, switch 210 may include a content addressable memory (CAM)-type structure, which can be tagged with an object ID to apply access tracking and QoS enforcement. To this end, system address decoder 216 tracks different objects and maps coherency requests (e.g., read requests) to those objects. Therefore, for a particular coherency request, switch 210 may use SAD 216 to determine which coherent domain and object it belongs to, identify the QoS it implements and specifies, and determine when to process the request. Requests determined not yet to be processed may be queued. When the request is processed, if the domain is coherent, it may proceed. If it is not coherent, switch 210 may execute a "fake" flow and reply to the originator with the expected response if the target does not have a line. Furthermore, switch 210 may send the request directly to the target via egress circuit 219. As an example, when bogus flow, the switch may return a global observation signal (eg, ACK GO) (indicating to the originator that no one has the line).

[0037] The switch 210 may implement new coherent domain registrations via the configuration interface 214 . In one embodiment, this interface allows for the specification of an address domain and the identification of memory ranges that belong to that memory domain, where it is assumed that the physical memory range (0 to N) maps to all different addressable memories in the system, and the interface also allows for the specification of elements within the memory domain, a list of process address IDs (PASIDs) that belong to the memory domain, and optionally a list of devices within the memory domain. The configuration interface 214 may also allow for the modification or removal of memory domains.

[0038] Coherency circuitry 220 may be configured to intercept CXL. cache requests and determine whether to intercept them. To do this, control circuitry 224 may use system address decoder 218 to determine, for the request, whether there is any coherency domain mapped within a particular address space that matches the memory address in the request. If no coherent domain is found, the request exits exit circuitry 219 toward its final target.

[0039] If one or more domains are found, coherency circuit 220 may check each domain to see if the PASID included in the request maps to it. If so, the request exits through exit circuit 219 and proceeds to its final target. If not, coherency circuit 220 may drop the snoop or memory CXL.cache request. Coherency circuit 220 implements a coherency response corresponding to that particular CXL.cache request, such as an invalid response.

[0040] Referring now to Figure 3, a flow diagram of a method according to an embodiment is shown. As shown in Figure 3, method 300 is a method for generating and updating memory properties in response to a memory allocation request. Thus, method 300 may be performed by switch circuitry, such as coherency circuitry within a switch according to an embodiment. Thus, method 300 may be performed by hardware circuitry, firmware, software, and / or combinations thereof.

[0041] As shown, method 300 begins with receiving a memory allocation request at a switch (block 310). By way of example, an application, such as a VM, a process, or any other software entity, may issue this request, which may include a variety of information. Exemplary information in the request may include memory range information, coherency status, address space identifier information, etc., but embodiments are not limited thereto.

[0042] Control then passes to diamond 320, where it is determined whether an entry already exists in the memory domain table for the memory range of this memory allocation request. If not, control passes to block 330, where an entry in this table may be created. By way of example, the entry may include the fields described above in connection with FIG. 2. Alternatively, if it is determined that an entry already exists, control passes to block 340, where the entry may be updated. For example, the coherency status may be changed, such as making a coherent domain non-coherent after a transaction completes, or deleting a memory domain upon application termination. While shown at this high level in the embodiment of FIG. 3, various modifications and alternatives are possible.

[0043] Referring now to Figure 4, a flow diagram of a method according to another embodiment is shown. As shown in Figure 4, method 400 is a method for handling received memory requests within a switch. Accordingly, method 400 may be performed by various circuits within the switch. Accordingly, method 400 may be performed by hardware circuits, firmware, software, and / or combinations thereof.

[0044] The method 400 begins by receiving a memory request in a switch (block 410). For purposes of explanation, the memory request is for a data read. The read request includes an address where the requested data resides. Next, in block 420, a memory domain table may be accessed based on the address of the memory request to identify an entry in the table associated with the memory domain that includes the address, for example.

[0045] At diamond 425, it is determined whether the memory request is for a coherent memory domain. This determination may be based on a coherency status indicator present in the coherency status field of the corresponding entry in the memory domain table. If not, control proceeds to block 430, where the memory request is forwarded to the destination location without further processing within the switch because the request is destined for a non-coherent domain.

[0046] 4, if it is determined that the request is for a coherent memory domain, control proceeds to diamond 440, where it is determined whether the memory request is associated with a snoop. This determination may be based on whether the request is for a read. If so, snoop processing may be performed. Other memory requests, such as write requests, may be handled directly without snoop processing (block 445).

[0047] Control then passes to diamond 450, where it is determined whether snooping is permitted. This determination may be based on one or more system parameters, such as interconnect status. If it is determined that snooping is not permitted, such as because there is high interconnect traffic, control passes to block 460. In block 460, the memory request may be handled in response to the callback information. More specifically, a corresponding entry in the memory domain table may be accessed to determine a fallback handling mechanism that may be used to handle the snooping. In this manner, interconnect traffic reduction may be achieved.

[0048] Continuing with reference to FIG. 4, if it is determined at diamond 450 that the snoop operation is permitted, control proceeds to block 470, where a snoop operation is performed to determine the presence and status of the requested data in various distributed caches and other memory structures. Next, at block 480, the memory request may be handled based on the snoop results. For example, if the latest copy of the data is determined to be valid, a read request may be executed. Alternatively, if dirty data is indicated, a read completion may be provided using the dirty data. While the embodiment of FIG. 4 is shown at this high level, various modifications and alternatives are possible.

[0049] Referring now to FIG. 5, a block diagram of a system according to another embodiment of the present invention is shown. As shown in FIG. 5, system 500 may be any type of computing device and, in one embodiment, may be a server system, such as an edge platform. In the embodiment of FIG. 5, system 500 includes multiple CPUs 510a, b coupled to respective system memories 520a, b. The system memories may, in embodiments, be implemented as dual in-line memory modules (DIMMs), such as double data rate (DDR) memory, persistent, or other types of memory. Note that the CPUs 510 may be coupled to each other via an interconnect system 515, such as the Intel® UltraPath Interconnect, or other processor interconnect technology.

[0050] There may be multiple interconnects 530a1-b2 to allow coherent accelerator devices and / or smart adapter devices to couple to CPU 510 via potentially multiple communication protocols. In one embodiment, each interconnect 530 may be a given instance of CXL.

[0051] In the illustrated embodiment, each CPU 510 may be coupled to a corresponding field programmable gate array (FPGA) / accelerator device 550a,b (which may include a graphics processing unit (GPU) in one embodiment). The CPUs 510 may also be coupled to smart NIC devices 560a,b. The smart NIC devices 560a,b may also be coupled to switches 580a,b (e.g., CXL switches according to embodiments). The switches may be coupled to pooled memory 590a,b, such as persistent memory. In embodiments, the switches 580 may perform fine-grained and dynamic coherency management of independent coherent (and non-coherent) memory domains, as described herein. Of course, embodiments are not limited to switches, and the techniques described herein may be performed by other entities in the system.

[0052] Referring now to FIG. 6 , an embodiment of an SoC design according to an embodiment is shown. As a specific illustrative example, SoC 600 may be configured for insertion into any type of computing device, ranging from a mobile device to a server system. Here, SoC 600 includes two cores 606 and 607. Cores 606 and 607 may conform to instruction set architectures such as those designed by Intel® Architecture Core™-type processors, Advanced Micro Devices (AMD), MIPS-type processors, or ARM-type processors, or those designed by their customers, licensees, or adopters. Cores 606 and 607 are coupled to a bus interface unit 609 and a cache controller 608 associated with an L2 cache 610 for communication with the rest of system 600 via interconnect 612. As shown, bus interface unit 609 includes coherency circuitry 611 that may perform coherency operations as described herein.

[0053] The interconnect 612 provides a communication channel to other components, such as a subscriber identity module (SIM) 630 for interfacing with a SIM card, a boot ROM 635 for holding boot code for initializing and booting the SoC 600 executed by the cores 606 and 607, an SDRAM controller 640 for interfacing with external memory (e.g., DRAM 660), a flash controller 645 for interfacing with non-volatile memory (e.g., flash 665), a peripheral controller 650 (e.g., an eSPI interface) for interfacing with peripherals, a video codec 620 and video interface 625 for displaying and receiving input (e.g., touch-enabled input), and a GPU 615 for performing graphics-related computer processing. The system also shows peripherals for communication, such as a Bluetooth module 670, a 3G modem 675, a GPS 680, and Wi-Fi 685. The system further includes a power controller 655. As further shown in FIG. 6, the system 600 may further include interfaces, including, for example, a MIPI interface 692 to a display, and / or an HDMI interface 695 that may also be coupled to the same display or another display.

[0054] Referring now to Figure 7, a block diagram of a system according to another embodiment of the present invention, such as an edge platform, is shown. As shown in Figure 7, a multiprocessor system 700 includes a first processor 770 and a second processor 780 coupled via a point-to-point interconnect 750. As shown in Figure 7, each of processors 770 and 780 may be a multi-core processor, including representative first and second processor cores (i.e., processor cores 774a and 774b, and processor cores 784a and 784b).

[0055] 7 embodiment, processors 770 and 780 further comprise point-to-point interconnects 777 and 787 that couple via interconnects 742 and 744 (which may be CXL buses) to switches 759 and 760 that may perform fine-grained and dynamic coherency management of independent coherent (and non-coherent) memory domains, as described herein. Switches 759, 760, in turn, couple to pooled memories 755 and 765. In this manner, switches 759, 760 may perform traffic monitoring and dynamic control of coherency traffic based on rules provided by applications executing on processors 770 and 780, for example, including reconfiguring fallback mechanisms for particular coherency traffic based on interconnect congestion levels exceeding a given threshold, as described herein.

[0056] 7, the first processor 770 further includes a memory controller hub (MCH) 772 and point-to-point (PP) interfaces 776 and 778. Similarly, the second processor 780 includes an MCH 782 and PP interfaces 786 and 788. As shown in FIG. 7, the MCHs 772 and 782 couple the processors to their respective memories, specifically memory 732 and memory 734, which may be portions of system memory (e.g., DRAM) locally attached to the respective processors. The first processor 770 and the second processor 780 may be coupled to a chipset 790 via PP interconnects 776 and 786, respectively. As shown in FIG. 7, the chipset 790 includes PP interfaces 794 and 798.

[0057] 7, various input / output (I / O) devices 714 may be coupled to the first bus 716, along with a bus bridge 718 that couples the first bus 716 to a second bus 720. Various devices may be coupled to the second bus 720, for example, a keyboard / mouse 722, a communications device 726, a data storage unit 728, such as a disk drive or other mass storage device, which in one embodiment may include code 730. Additionally, audio I / O 524 may be coupled to the second bus 720.

[0058] The embodiments described herein can be used within a variety of network architectures. To this end, many different types of computing platforms in a networked architecture, coupling between a given edge device and a data center, can perform the fine-grained and dynamic coherency management of independent coherent (and non-coherent) memory domains described herein. Referring now to FIG. 8, a block diagram of a network architecture according to another embodiment of the present invention is shown. As shown in FIG. 8, network architecture 800 includes a variety of computing platforms that can be located quite widely and have different latencies in communicating with different devices.

[0059] 8, network architecture 800 includes an exemplary device 810, such as a smartphone, that may communicate over different radio access networks (RANs), including RAN 820 and RAN 830. RAN 820, in turn, may be coupled to platform 825, which may be an edge platform, such as a fog / far / near edge platform, and may employ embodiments herein. Other requests may be handled by far edge platform 835 coupled to RAN 830, which may also employ embodiments.

[0060] As further shown in Figure 8, another near-edge platform 840 may be coupled to the RANs 820, 830. This near-edge platform may be located near a data center 850, which may have many computing resources. Pushing messages to these more remote platforms incurs more latency in handling requests on behalf of the edge device 810. It should be understood that all of the platforms shown in Figure 8 may incorporate embodiments described herein to perform fine-grained and dynamic coherency management of independent coherent (and non-coherent) memory domains.

[0061] The following examples relate to further embodiments.

[0062] In one example, an apparatus includes a table storing a plurality of entries each identifying a memory domain of a system and a coherency status of the memory domain, and a control circuit coupled to the table, wherein the control circuit can receive a request to change the coherency status of a first memory domain of the system and dynamically update a first entry in the table for the first memory domain to change the coherency status between a coherent memory domain and a non-coherent memory domain.

[0063] In one example, the control circuitry receives a memory allocation request for a second memory domain of the system and writes a second entry in the table for the second memory domain, the second entry indicating a coherency status of the second memory domain as one of the coherent memory domain or the non-coherent memory domain.

[0064] In one example, the first entry includes memory region information, one or more process address identifiers belonging to the first memory domain, one or more attributes related to the first memory domain, and callback information.

[0065] In one example, the callback information includes at least one fallback rule for handling coherency for memory requests when an interconnect congestion level exceeds a threshold.

[0066] In one example, the apparatus further comprises telemetry circuitry that maintains telemetry information including the interconnect congestion level.

[0067] In one example, the apparatus handles coherency for memory requests according to at least one fallback rule when an interconnect congestion level exceeds a threshold.

[0068] In one example, the device includes a coherent switch that receives a first memory request for a first location within the first memory domain and performs coherency processing before the coherency status change request, and receives a second memory request for another location within the first memory domain after the coherency status change request and forwards the second memory request to a destination of the second memory request without performing the coherency processing.

[0069] In one example, the control circuitry receives a memory allocation request for a second memory domain of the system that includes a primary data store for a database application, the memory allocation request indicating a coherency status of the second memory domain as a non-coherent memory domain, and in response to the memory allocation request, the control circuitry writes a second entry in the table for the second memory domain, the second entry indicating the coherency status of the second memory domain as the non-coherent memory domain.

[0070] In another example, a method includes receiving a memory request at a switch of a system, the switch coupled between a requester and a target memory; determining whether an address of the memory request is within a coherent memory domain; performing a snoop operation on the memory request and handling the memory request based on the snoop operation if the address of the memory request is within the coherent memory domain; and directing the memory request from the switch to the target memory without performing the snoop operation if the address of the memory request is not within the coherent memory domain.

[0071] In one example, the method further includes determining an interconnect congestion level.

[0072] In one example, the method further includes, if the interconnect congestion level is greater than a threshold, handling the memory request according to callback information associated with the coherent memory domain, the callback information being stored in a memory domain table.

[0073] In one example, the method further includes determining whether the address is within the coherent memory domain based on memory range information stored in a memory domain table.

[0074] In one example, the method further includes receiving a memory allocation request for a first coherent memory domain and storing an entry for the first coherent memory domain in a memory domain table, the entry including memory region information, one or more process address identifiers belonging to the first coherent memory domain, one or more devices within the first coherent memory domain, and callback information for identifying at least one fallback rule for handling memory requests for the first coherent memory domain when an interconnect congestion level exceeds a threshold.

[0075] In one example, the method further includes allocating a first memory domain to a first tenant in response to a first memory allocation request for a coherent memory domain associated with a first plurality of devices and a first memory range of the system, and allocating a second memory domain to a second tenant in response to a second memory allocation request for a non-coherent memory domain associated with a second plurality of devices and a second memory range of the system, wherein the first memory domain is isolated from the second memory domain.

[0076] In another example, a computer-readable medium comprising instructions performs the method of any of the above examples.

[0077] In a further example, a computer-readable medium comprising data is used by at least one machine to manufacture at least one integrated circuit that performs the method of any one of the above examples.

[0078] In another further example, an apparatus comprises means for performing the method of any one of the above examples.

[0079] In another example, a system includes a plurality of processors, a plurality of accelerators, a system memory dynamically partitioned into a plurality of memory domains including at least one coherent memory domain and at least one non-coherent memory domain, and a switch coupling at least some of the processors and at least some of the accelerators via a Compute Express Link (CXL) interconnect, wherein the switch can dynamically generate the at least one coherent memory domain in response to a first memory allocation request and dynamically generate the at least one non-coherent memory domain in response to a second memory allocation request.

[0080] In one example, the switch dynamically updates the at least one coherent memory domain to another non-coherent memory domain in response to a memory update request.

[0081] In one example, the switch includes a CXL switch including a memory domain table having a plurality of entries, each of the plurality of entries storing memory region information, at least one of one or more process address identifiers or at least one of one or more tenant identifiers belonging to the memory domain, and one or more devices within the memory domain.

[0082] In one example, at least some of the plurality of entries further store at least one fallback rule for handling memory requests when an interconnect congestion level exceeds a threshold.

[0083] In one example, the CXL switch further comprises telemetry circuitry that maintains telemetry information including the interconnect congestion level.

[0084] In one example, the CXL switch receives the first memory allocation request, the first memory allocation request including a memory range for the at least one coherent memory domain, a coherency indicator, one or more process address identifiers belonging to the at least one coherent memory domain, one or more devices within the at least one coherent memory domain, and at least one fallback rule for handling coherency for memory requests when a congestion level on one or more of the CXL interconnects exceeds a threshold.

[0085] It should be understood that various combinations of the above examples are possible.

[0086] It should be noted that the terms “circuit” and “circuitry” are used interchangeably herein. These terms, along with the term “logic,” are used herein to refer to analog circuits, digital circuits, hardwired circuits, programmable circuits, processor circuits, microcontroller circuits, hardware logic circuits, state machine circuits, and / or any other type of physical hardware component, alone or in any combination. Embodiments may be used in many different types of systems. For example, in one embodiment, a communications device may be configured to perform the various methods and techniques described herein. Of course, the scope of the invention is not limited to communications devices. Instead, other embodiments may be directed to other types of apparatus for processing instructions, or to one or more machine-readable media containing instructions that, when executed by a computer processing unit, cause the device to perform one or more of the methods and techniques described herein.

[0087] Some embodiments may be implemented in code and stored on a non-transitory storage medium that stores instructions that can be used to program a system to execute the instructions. Each embodiment may also be implemented in data and stored on a non-transitory storage medium. The non-transitory storage medium, when used by at least one machine, causes the at least one machine to manufacture at least one integrated circuit that performs one or more operations. Further embodiments may be implemented in a computer-readable storage medium that, when manufactured as an SoC or other processor, includes information that configures the SoC or other processor to perform one or more operations. The storage medium may include, but is not limited to, a floppy disk, an optical disk, a solid state drive (SSD), a compact disk read-only memory (CD-ROM), a compact disk re-writable (CD-RW), any type of disk including a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM) such as a dynamic random access memory (DRAM) or a static random access memory (SRAM), a semiconductor device such as an erasable programmable read-only memory (EPROM), a flash memory, an electrically erasable programmable read-only memory (EEPROM), a magnetic or optical card, or any other type of medium suitable for storing electronic instructions.

[0088] While the present invention has been described with respect to a limited number of embodiments, those skilled in the art will appreciate numerous variations and modifications therefrom, and it is intended in the appended claims to cover all such variations and modifications which fall within the true spirit and scope of the invention. [Other possible items] [Item 1] a table storing a plurality of entries each identifying a memory domain of the system and a coherency status of said memory domain; and control circuitry coupled to the table to receive a request to change the coherency status of a first memory domain of the system, the control circuitry dynamically updating a first entry in the table for the first memory domain to change the coherency status between a coherent memory domain and a non-coherent memory domain. [Item 2] 2. The apparatus of claim 1, wherein the control circuitry receives a memory allocation request for a second memory domain of the system and writes a second entry in the table for the second memory domain, the second entry indicating a coherency status of the second memory domain as one of the coherent memory domain or the non-coherent memory domain. [Item 3] The device described in item 1, wherein the first entry includes memory region information, one or more process address identifiers belonging to the first memory domain, one or more attributes related to the first memory domain, and callback information. [Item 4] Item 4. The apparatus of item 3, wherein the callback information includes at least one fallback rule for handling coherency for memory requests when an interconnect congestion level exceeds a threshold. [Item 5] Item 5. The apparatus of item 4, further comprising a telemetry circuit that maintains telemetry information including the interconnect congestion level. [Item 6] Item 1 . The apparatus of item 1, wherein the apparatus handles coherency for memory requests according to at least one fallback rule when an interconnect congestion level exceeds a threshold. [Item 7] 2. The apparatus of claim 1, further comprising a coherent switch that receives a first memory request for a first location within the first memory domain before the request to change the coherency status and performs coherency processing, and receives a second memory request for another location within the first memory domain after the request to change the coherency status and forwards the second memory request to a destination of the second memory request without performing the coherency processing. [Item 8] Item 10. The apparatus of item 1, wherein the control circuitry receives a memory allocation request for a second memory domain of the system that includes a primary data store for a database application, the memory allocation request indicating a coherency status of the second memory domain as a non-coherent memory domain, and in response to the memory allocation request, the control circuitry writes a second entry in the table for the second memory domain, the second entry indicating the coherency status of the second memory domain as the non-coherent memory domain. [Item 9] At least one computer-readable storage medium having instructions stored thereon, the instructions, when executed by a machine, receiving a memory request at a switch of the system, the switch being coupled between the requester and the target memory; determining whether the address of the memory request is within a coherent memory domain; if the address of the memory request is within the coherent memory domain, performing a snoop operation on the memory request and handling the memory request based on the snoop operation; if the address of the memory request is not within the coherent memory domain, directing the memory request from the switch to the target memory without performing the snooping operation; At least one computer-readable storage medium that causes the machine to perform a method including: [Item 10] Item 10. At least one computer-readable storage medium according to item 9, wherein the method further comprises determining an interconnect congestion level. [Item 11] Item 11. The at least one computer-readable storage medium of item 10, wherein the method further includes, if the interconnect congestion level is greater than a threshold, handling the memory request according to callback information associated with the coherent memory domain, the callback information being stored in a memory domain table. [Item 12] 10. The method of claim 9, further comprising determining whether the address is within the coherent memory domain based on memory range information stored in a memory domain table. [Item 13] Item 10. At least one computer-readable storage medium according to item 9, wherein the method further includes receiving a memory allocation request for a first coherent memory domain and storing an entry for the first coherent memory domain in a memory domain table, the entry including memory region information, one or more process address identifiers belonging to the first coherent memory domain, one or more devices within the first coherent memory domain, and callback information for identifying at least one fallback rule for handling memory requests for the first coherent memory domain when an interconnect congestion level exceeds a threshold. [Item 14] The above method is allocating a first memory domain to a first tenant in response to a first memory allocation request for a coherent memory domain associated with a first plurality of devices and a first memory range of the system; allocating a second memory domain to a second tenant in response to a second memory allocation request for a non-coherent memory domain associated with a second plurality of devices and a second memory range of the system; 10. The at least one computer-readable storage medium of claim 9, further comprising: [Item 15] a plurality of processors; Multiple accelerators and a system memory that is dynamically partitioned into multiple memory domains, including at least one coherent memory domain and at least one non-coherent memory domain; a switch coupled to at least some of the plurality of processors and at least some of the plurality of accelerators via a Compute Express Link (CXL) interconnect, the switch dynamically generating the at least one coherent memory domain in response to a first memory allocation request and dynamically generating the at least one non-coherent memory domain in response to a second memory allocation request. [Item 16] Item 16. The system of item 15, wherein the switch dynamically updates the at least one coherent memory domain to another non-coherent memory domain in response to a memory update request. [Item 17] Item 16. The system of item 15, wherein the switch includes a CXL switch, and the CXL switch includes a memory domain table having a plurality of entries, each of the plurality of entries storing memory region information, at least one of one or more process address identifiers or one or more tenant identifiers belonging to the memory domain, and one or more devices within the memory domain. [Item 18] Item 18. The system of item 17, wherein at least some of the plurality of entries further store at least one fallback rule for handling memory requests when an interconnect congestion level exceeds a threshold. [Item 19] 20. The system of claim 18, wherein the CXL switch further comprises a telemetry circuit that maintains telemetry information including the interconnect congestion level. [Item 20] 20. The system of claim 18, wherein the CXL switch receives the first memory allocation request, the first memory allocation request including a memory range for the at least one coherent memory domain, a coherency indicator, one or more process address identifiers belonging to the at least one coherent memory domain, one or more devices within the at least one coherent memory domain, and at least one fallback rule for handling coherency for memory requests when a congestion level on one or more of the CXL interconnects exceeds a threshold.

Claims

1. a table storing a plurality of entries each identifying a memory domain of the system and a coherency status of said memory domain; control circuitry coupled to the table for receiving a request to change the coherency status of a first memory domain of the system; Equipped with The control circuitry dynamically updates a first entry in the table for the first memory domain to change the coherency status between a coherent memory domain and a non-coherent memory domain.

2. 2. The apparatus of claim 1, wherein the control circuitry receives a memory allocation request for a second memory domain of the system and writes a second entry in the table for the second memory domain, the second entry indicating a coherency status of the second memory domain as one of the coherent memory domain or the non-coherent memory domain.

3. 3. The apparatus according to claim 1, wherein the first entry includes memory region information, one or more process address identifiers belonging to the first memory domain, one or more attributes related to the first memory domain, and callback information.

4. The apparatus of claim 3 , wherein the callback information includes at least one fallback rule for handling coherency for memory requests when an interconnect congestion level exceeds a threshold.

5. The apparatus of claim 4 , further comprising a telemetry circuit that maintains telemetry information including the interconnect congestion level.

6. The apparatus of claim 1 , wherein the apparatus handles coherency for memory requests according to at least one fallback rule when an interconnect congestion level exceeds a threshold.

7. 7. The apparatus of claim 1, further comprising a coherent switch configured to receive a first memory request for a first location within the first memory domain before the request to change the coherency status, perform coherency processing, receive a second memory request for a different location within the first memory domain after the request to change the coherency status, and forward the second memory request to a destination of the second memory request without performing the coherency processing.

8. 8. The apparatus of claim 1, wherein the control circuitry receives a memory allocation request for a second memory domain of the system that includes a primary data store for a database application, the memory allocation request indicating a coherency status of the second memory domain as a non-coherent memory domain, and in response to the memory allocation request, the control circuitry writes a second entry in the table for the second memory domain, the second entry indicating the coherency status of the second memory domain as the non-coherent memory domain.

9. To the machine, receiving a memory request at a switch of the system, the switch being coupled between a requester and a target memory; determining whether the address of the memory request is within a coherent memory domain; if the address of the memory request is within the coherent memory domain, performing a snoop operation on the memory request and handling the memory request based on the snoop operation; if the address of the memory request is not within the coherent memory domain, directing the memory request from the switch to the target memory without performing the snooping operation; A program that executes.

10. The program product of claim 9 , further causing the machine to perform the step of determining an interconnect congestion level.

11. 11. The computer program product of claim 10, further causing the machine to execute a procedure for handling the memory request according to callback information associated with the coherent memory domain if the interconnect congestion level is greater than a threshold, the callback information being stored in a memory domain table.

12. 12. The computer-readable medium of claim 9, further comprising: a program for causing the machine to execute a procedure for determining whether the address is within the coherent memory domain based on memory range information stored in a memory domain table.

13. 13. The program of claim 9, further causing the machine to perform the steps of receiving a memory allocation request for a first coherent memory domain and storing an entry for the first coherent memory domain in a memory domain table, the entry including memory region information, one or more process address identifiers belonging to the first coherent memory domain, one or more devices in the first coherent memory domain, and callback information for identifying at least one fallback rule for handling memory requests for the first coherent memory domain when an interconnect congestion level exceeds a threshold.

14. The machine, allocating a first memory domain to a first tenant in response to a first memory allocation request for a coherent memory domain associated with a first plurality of devices and a first memory range of the system; allocating a second memory domain to a second tenant in response to a second memory allocation request for a non-coherent memory domain associated with a second plurality of devices and a second memory range of the system; Then run The program of claim 9 , wherein the first memory domain is isolated from the second memory domain.

15. A computer-readable storage medium storing the program according to any one of claims 9 to 14.

16. a plurality of processors; Multiple accelerators and a system memory that is dynamically partitioned into a plurality of memory domains, including at least one coherent memory domain and at least one non-coherent memory domain; a switch coupling at least some of the processors and at least some of the accelerators via a Compute Express Link (CXL) interconnect; Equipped with The system, wherein the switch dynamically creates the at least one coherent memory domain in response to a first memory allocation request and dynamically creates the at least one non-coherent memory domain in response to a second memory allocation request.

17. 17. The system of claim 16, wherein the switch dynamically updates the at least one coherent memory domain to become another non-coherent memory domain in response to a memory update request.

18. 18. The system of claim 16 or 17, wherein the switch includes a CXL switch, and the CXL switch includes a memory domain table having a plurality of entries, each of the plurality of entries storing memory region information, at least one of one or more process address identifiers or one or more tenant identifiers belonging to a memory domain, and one or more devices within the memory domain.

19. 20. The system of claim 18, wherein at least some of the plurality of entries further store at least one fallback rule for handling memory requests when an interconnect congestion level exceeds a threshold.

20. 20. The system of claim 19, wherein the CXL switch further comprises a telemetry circuit that maintains telemetry information including the interconnect congestion level.

21. 20. The system of claim 19, wherein the CXL switch receives the first memory allocation request, the first memory allocation request including a memory range for the at least one coherent memory domain, a coherency indicator, one or more process address identifiers belonging to the at least one coherent memory domain, one or more devices within the at least one coherent memory domain, and at least one fallback rule for handling coherency for memory requests when a congestion level on one or more of the CXL interconnects exceeds a threshold.

Images