Cache access fabric

Abstract

Examples described herein relate to a cache fabric that includes a set of routers of a first tier and a plurality of cache controller clusters of a second tier. A router of the set of routers is accessible via an interface to receive a memory access request from a processor and select from a set of cache controllers based on a cluster identifier and a memory address, and provide the memory access request to the selected set of cache controllers. The selected set of cache controllers may receive memory access requests and serve memory access requests from the cache device, or forward memory access requests to a second cache controller or second cache device associated with the cache device.

Description

CLAIM OF PRIORITY

[0001] This application claims priority under 35 U.S.C. § 365(c) to U.S. Application No. 18 / 207,602, filed June 8, 2023, the entire contents of which are hereby incorporated by reference herein. BACKGROUND

[0002] As the number of cores in a server continues to grow, the fabric that communicatively couples the cores to cache devices has an increasing bandwidth requirement. A mesh fabric can allow cores to connect to cache devices to access the entire cache. For example, a mesh fabric allows connected cores to access a last level cache (LLC). BRIEF DESCRIPTION OF DRAWINGS

[0003] Figure 1A AND Figure 1B An example system is depicted.

[0004] Figure 2 An example system is depicted.

[0005] Figure 3A AND Figure 3B An example system is depicted.

[0006] Figure 4 An example process is depicted.

[0007] Figure 5 An example system is depicted. DETAILED DESCRIPTION

[0008] The power usage of a mesh fabric can increase as the mesh fabric connects more and more processor cores to cache devices. To at least potentially reduce or limit the power usage of a cache fabric that connects processor cores to one or more cache devices, various examples provide a cache fabric having a first tier of a set of routers and a second tier of a plurality of cache controller clusters. A router of the set of routers can be accessed via an interface that is to receive a memory access request from a processor and select from a set of cache controllers based on a cluster identifier and a memory address, and provide the memory access request to the selected set of cache controllers. The selected set of cache controllers can receive the memory access request from a cache device and service the memory access request, or forward the memory access request to a second cache controller associated with the cache device or a second cache device.

[0009] Figure 1AAn example overview of a system is depicted. Processor 100 can include one or more processor cores configured to process a specific instruction set. In some examples, the instruction set can facilitate complex instruction set computing (CISC), reduced instruction set computing (RISC), or computing via a very long instruction word (VLIW). Processor 100 can process a different instruction set, which can include instructions to facilitate the emulation of other instruction sets. Processor 100 can also include other processing devices, such as a digital signal processor (DSP). Processor 100 can be part of a central processing unit (CPU) or a graphics processing unit (GPU).

[0010] A register file (not shown) can additionally be included in processor 100 and can include different types of registers to store different types of data (e.g., integer registers, floating point registers, status registers, and instruction pointer registers). Some registers can be general registers. In some examples, one or more of the cores can include a memory controller (not shown) to provide memory access requests communications to memory devices and other components (e.g., router 110). In some examples, a memory access request can specify one or more of the following: a virtual or physical address associated with a memory device (e.g., memory 130) to be read or written or a cache cluster identifier to identify a cluster among cache clusters 120.

[0011] In some examples, one or more processors 100 can be coupled with one or more interface buses 102-0 through 102-L, where L is an integer, to transmit communications signals, such as address, data, or control signals, between the cores and one or more routers of router 110. The one or more interface buses 102-0 through 102-L can include a processor bus, such as a version of the direct media interface (DMI) bus. However, the processor bus is not limited to the DMI bus, and can include one or more peripheral component interconnect interfaces (e.g., peripheral component interconnect express (PCIe), universal chiplet interconnect (UCIe)), memory buses or interfaces (e.g., compute express link (CXL) or other types of interfaces.

[0012] The cache cluster 120 can include one or more cache controllers and caches (e.g., one or more registers, one or more cache devices (e.g., a level one cache (LI), a level two cache (L2), a level three cache (L3), a last level cache (LLC), a volatile memory device, a non-volatile memory device, or a persistent memory) (not shown) that can share using cache coherency techniques between the processors 100. If data referenced by a memory access request is stored in a cache device (e.g., a cache hit), the cache device can provide the data to the requesting processor. If data referenced by a memory access request is not stored in a cache device (e.g., a cache miss), the cache controller or cache device can forward the request to a memory controller of the memory device 130. A home agent (HA) or a caching and home agent (CHA) (not shown) can perform data coherency between the cache devices and the memory devices. In some examples, for a particular addressable memory range accessible to a particular CPU, the HA can attempt to implement data coherency between the memory devices and the caches. The memory device 130 can include one or more volatile or non-volatile memory devices. The memory 130 can provide data to a requesting processor via the cache cluster 120 or another connection.

[0013] Figure 1A Communication between components of the system 1000 can use conductors that provide chip-to-chip communications, board-to-board communications, and / or package-to-package communications. Chip-to-chip communications can utilize embedded multi-die interconnect bridges (EMIBs) or interposers. Figure 1A Components of the system 1000 can be enclosed in one or more semiconductor packages. A semiconductor package can include a metal, plastic, glass, and / or ceramic housing that encloses one or more semiconductor devices or integrated circuits and provides communication between and among the semiconductor devices or integrated circuits. Various examples can be implemented in a die, in a package, or between multiple packages, in a server, or between multiple servers.

[0014] Figure 1BAn example system is depicted. One or more cores of processor 100 can provide a memory access request to an associated router of routers 112-0 to 112-M, where M is an integer. In some examples, a source core can compute a destination cache controller and cache device that manages a particular address associated with the memory access request. By way of example, routers 112-0 to 112-M can perform a decode of the memory access request to identify a cluster among clusters 122-0 to 122-N, where N is an integer, that corresponds to a cluster of cache controllers and cache devices that potentially store data associated with a memory address provided by the memory access request (e.g., clusters 122-0 to 122-N, where N is an integer). The routers can utilize a crossbar, one or more multiplexers, or other interface to forward the memory access request to the identified cluster.

[0015] Clusters 120 can provide interconnects between outputs of routers 110 and cache controllers and cache devices and potentially memory devices. One or more of clusters 122-0 to 122-N can include one or more cache controllers and cache devices (CCs). The clusters can be mapped to a subset of memory addresses based on a hash function or other allocation to assign memory addresses or spans of contiguous memory addresses to a particular cluster. In some examples, a receiver cache controller of a CC can manage a particular address associated with a memory access request. The cache controller of the CC can receive the memory access request and determine whether the cache controller or another cache controller manages memory addresses in a range that includes the memory address associated with the memory access request. Based on the cache controller managing memory addresses in the range that includes the memory address associated with the memory access request, the cache controller can determine whether to store or not store data associated with the memory access request in an associated cache. For a cache hit, the cache controller can provide the data to the requester core. In the case of a cache hit, the data can traverse through a cluster (e.g., one of 124-0 to 124-N) to the path to the core via the interconnect, and then through an originating router (e.g., one of 112-0 to 122-M) to the requester core. For a cache miss, the cache controller can provide the memory access request or a portion thereof to a home agent or cache and home agent (not shown) to retrieve the data and provide it to the source core. The home agent or cache and home agent (not shown) can perform data coherency between the cache device and the memory device so that the data provided to the requester core is a current version.

[0016] Figure 2Example router circuitry is depicted. Decoder 202 can receive memory access requests and select an output line to connect to a target cluster based on an address associated with the memory access request based on configuration from an operating system (OS), system administrator, coordinator, or other entity. For example, decoder 202 can access a configuration that specifies a cluster and output line for a particular virtual or physical memory address range or value. In some examples, a core can specify a memory address and cluster number, and decoder 202 can select an output line to a target cluster in crossbar circuitry 204 based on the address and cluster number.

[0017] Crossbar circuitry 204 can connect one or more of a plurality of cores to a plurality of different cache controllers in different clusters. For example, crossbar circuitry 204 can connect one or more of 4 cores to 4 different cache controllers in different clusters. Crossbar circuitry 204 can include one or more 4x4 crossbars, four 2x2 crossbars, or other configurations. Crossbar circuitry 204 can queue received memory access requests before output to a target cluster and output memory access requests based on ordering (e.g., ordering of received requests from different transmissions such as first-in-first-out), arbitration (e.g., round robin selection from queues or source cores, round robin selection from queues or source cores without head-of-line blocking, releasing higher priority memory access requests before lower priority memory access requests), or other policies. Crossbar circuitry 204 can queue received responses to memory access requests (e.g., data or other control messages) before output to a core based on ordering (e.g., first-in-first-out), arbitration (e.g., releasing higher priority memory access requests before lower priority memory access requests), or other policies.

[0018] Figure 3AAn example cache controller system is depicted. Input 302 can include U2C (e.g., non-core to core) circuitry to provide communications (e.g., memory access requests, data, snoops, responses, interrupts, or control signals) to a router or crossbar via an interface for forwarding to a processor. Input 302 can include C2U (e.g., core to non-core) circuitry to provide communications (e.g., memory access requests, data, snoops, responses, interrupts, or control signals) from a router or crossbar via an interface to connection 304. The non-core (e.g., system agent) can include one or more of a memory controller, a shared cache (e.g., last level cache (LLC)), a cache coherency manager, an arithmetic logic unit, a floating point unit, a core or processor interconnect, a cache / home agent (CHA), interface circuitry (e.g., fabric, memory, device), and / or a bus or link controller. The non-core can provide one or more of a direct memory access (DMA) engine connection, a non-cache coherent master connection, data cache coherency between cores and arbitrate cache requests, or an advanced microcontroller bus architecture (AMBA) capability.

[0019] Connection 304 can provide communications between different cache controllers of a cluster using an interface. For example, one or more interfaces can operate in a manner compliant with an inter-integrated circuit (I2C) protocol, a serial peripheral interface (SPI), or other protocol. At an ingress, a cache controller and cache 308 can receive memory access requests or snoops from a processor. A snoop can be used to determine whether cache coherency is maintained between cache devices and identify whether a cache device stores content identified by a memory address. At an egress, a cache controller and cache 308 can forward a memory access request to another cache controller and cache in the same cluster, provide a memory access request to a home agent or cache and home agent, or provide data from cache 312 to be sent to a processor that issued a memory access request for the data. Cache controller 310 can include or access HA or CHA circuitry to perform data coherency checks to identify and request and receive modified data from other cache devices or memory devices. Cache controller 310 can utilize scoreboard circuitry to track received memory access requests, attempt to maintain an order of operations, and track a status of a memory access request or snoop (e.g., complete, forwarded, uncompleted, timed out (e.g., a response is not received), etc.).

[0020] Figure 3B An example cache controller system is depicted. Operation of a cache controller and cache (e.g., LLC) can be similar to operation of the system of Figure 3A .

[0021] Figure 4 An example process is depicted. The process can be performed by a processor using a set of routers and a plurality of cache controller clusters. At 402, the processor can issue a memory access request to an associated router. In some examples, the memory access request is associated with one or more of: a read or write request, a memory address, and a target cluster. At 404, the router can route the memory access request to the target cluster. In some examples, the router can be one of a plurality of routers, where one or more of the plurality of routers are coupled to different cache controllers in different clusters. For example, the router can select a particular output line connected to the target cluster to forward the memory access request to a cache controller in the target cluster.

[0022] At 406, it can be determined whether the cache controller in the target cluster can service the memory access request. For example, the cache controller in the target cluster can service the memory access request based on the memory access request being associated with an address assigned to the cache controller. Based on the cache controller in the target cluster servicing the memory access request, the process can proceed to 408. At 408, the cache controller can service the memory access request by providing data from a cache device associated with the cache controller to the requesting processor based on a cache hit in the cache device.

[0023] At 406, based on the cache controller in the target cluster not servicing the memory access request, the process can proceed to 410. At 410, the cache controller can forward the memory access request to a home agent (HA) or cache and home agent (CHA) to manage access to data for the memory access request from a memory or storage device based on a cache miss in a cache device associated with the cache controller. In some examples, at 410, the cache controller in the target cluster can use an output line in the target cluster to forward the memory access request to a second cache controller in the same cluster for the second cache controller to service the memory access request. For example, the target cluster can include one or more output lines that communicatively couple different cache controllers together.

[0024] Figure 5A system is depicted. As described herein, the system can use multiple routers and multiple cluster systems to forward memory access requests. The system 500 includes a processor 510, which provides processing, operation management, and instruction execution for the system 500. The processor 510 can include any type of microprocessor, central processing unit (CPU), graphics processing unit (GPU), XPU, processing core, or other processing hardware that provides processing for the system 500, or a combination of processors. An XPU can include one or more of the following: a CPU, a graphics processing unit (GPU), a general purpose GPU (GPGPU), and / or other processing units (e.g., an accelerator or a programmable or fixed-function FPGA). The processor 510 controls the overall operation of the system 500, and can be or include one or more programmable general- purpose or special-purpose microprocessors, digital signal processors (DSPs), programmable controllers, application specific integrated circuits (ASICs), programmable logic devices (PLDs), and the like, or a combination of such devices. The processor 510 can include one or more processor sockets.

[0025] In some examples, the interface 512 and / or the interface 514 can include a switch (e.g., a CXL switch) that provides device interfaces between the processor 510 and other devices (e.g., the memory subsystem 520, the graphics 540, the accelerator 542, the network interface 550, etc.).

[0026] In one example, the system 500 includes an interface 512 coupled to the processor 510, which can represent a higher speed interface or a high throughput interface for system components that require higher bandwidth connections, such as the memory subsystem 520 or the graphics interface component 540 or the accelerator 542. The interface 512 represents interface circuitry, which can be a standalone component or integrated onto the processor die. In some examples, the interface 512 and / or the interface 512 can include multiple routers and multiple cluster systems to forward memory access requests to the memory subsystem 520 and / or the storage subsystem 580, as described herein.

[0027] The accelerator 542 can be a programmable or fixed-function offload engine accessible or usable by the processor 510. For example, an accelerator in the accelerator 542 can provide compression (DC) capabilities, cryptographic services such as public key encryption (PKE), cryptography, hashing / authentication capabilities, decryption, or other capabilities or services. In some embodiments, additionally or alternatively, an accelerator in the accelerator 542 provides field selection controller capabilities as described herein. In some cases, the accelerator 542 can be integrated into a CPU socket (e.g., into a connector of a motherboard or circuit board that includes a CPU and provides an electrical interface with the CPU). For example, the accelerator 542 can include a single or multi-core processor, a graphics processing unit, a logic execution unit single or multi-level cache, a functional unit usable to independently execute a program or thread, an application-specific integrated circuit (ASIC), a neural network processor (NNP), programmable control logic, and programmable processing elements such as a field programmable gate array (FPGA). The accelerator 542 can provide multiple neural networks, CPU, processor cores, general purpose graphics processing units, or can make graphics processing units available for use by an artificial intelligence (AI) or machine learning (ML) model. For example, an AI model can use or include any one or combination of the following: a reinforcement learning scheme, a Q-learning scheme, a deep Q-learning or asynchronous advantage algorithm (A3C), a combined neural network, a recurrent combined neural network, or other AI or ML model. Multiple neural networks, processor cores, or graphics processing units can be made available for use by an AI or ML model.

[0028] Memory subsystem 520 represents the main memory of system 500 and provides storage for code to be executed by processor 510 or data values to be used in executing routines. Memory subsystem 520 can include one or more memory devices 530 such as read-only memory (ROM), flash memory, one or more varieties of random access memory (RAM) such as DRAM, or other memory devices, or combinations of such devices. Memory 530 stores and hosts, among other things, an operating system (OS) 532 to provide a software platform for executing instructions in system 500. In addition, applications 534 can execute on the software platform of OS 532 from memory 530. Applications 534 represent programs with their own operational logic to perform an execution of one or more functions. Applications 534 and / or processes 536 can alternatively or additionally refer to virtual machines (VMs), containers, microservices, processors, or other software. Processes 536 represent agents or routines that provide auxiliary functions to OS 532 or one or more applications 534 or combinations. OS 532, applications 534, and processes 536 provide software logic to provide functionality for system 500. In one example, memory subsystem 520 includes a memory controller 522 that is a memory controller to generate and issue commands to memory 530. Memory controller 522 can be a physical part of processor 510 or a physical part of interface 512. For example, memory controller 522 can be an integrated memory controller integrated onto a circuit with processor 510.

[0029] In some examples, OS 532 can be Linux®, Windows® Server or Personal Computer, FreeBSD®, Android®, MacOS®, iOS®, VMware vSphere, openSUSE, RHEL, CentOS, Debian, Ubuntu, or any other operating system. The OS and drivers can execute on one or more processors sold or designed by Intel®, ARM®, AMD®, Qualcomm®, IBM®, Nvidia®, Broadcom®, Texas Instruments®, and others. In some examples, OS 532 and / or drivers can configure multiple routers and multiple clustered systems to forward memory access requests to memory subsystem 520 or storage subsystem 580 as described herein.

[0030] While not specifically shown, it should be understood that system 500 can include one or more buses or bus systems between devices, such as a memory bus, graphics bus, interface bus, and the like. The buses or other signal lines can communicatively or electrically couple components together, or both. The buses can include physical communication lines, point-to-point connections, bridges, adapters, controllers, or other circuitry or combinations. The buses can include one or more of a system bus, a peripheral component interconnect (PCI) bus, a HyperTransport, or industry standard architecture (ISA) bus, a small computer system interface (SCSI) bus, a universal serial bus (USB), or an Institute of Electrical and Electronics Engineers (IEEE) standard 1394 bus (FireWire).

[0031] In one example, system 500 includes interface 514, which can be coupled to interface 512. In one example, interface 514 represents interface circuitry, which can include standalone components and integrated circuitry. In one example, multiple user interface components or peripheral components, or both, are coupled to interface 514. Network interface 550 provides system 500 with the ability to communicate with remote devices (e.g., servers or other computing devices) over one or more networks. Network interface 550 can include an Ethernet adapter, wireless interconnection components, cellular network interconnection components, USB (universal serial bus), or other wired or wireless standard or proprietary interfaces. Network interface 550 can transfer data to devices in the same data center or rack or to remote devices, which can include sending data stored in memory.

[0032] In some examples, network interface 550 can be implemented as a network interface controller, network interface card, host fabric interface (HFI), or host bus adapter (HBA), and such examples can be interchangeable. Network interface 550 can be coupled to one or more servers using buses, PCIe, CXL, or DDR. Network interface 550 can be embodied as part of a system on a chip (SoC) including one or more processors, or included on a multi-chip package that also contains one or more processors.

[0033] Some examples of network equipment 550 are part of or utilized by infrastructure processing units (IPUs) or data processing units (DPUs). An xPU can refer to at least an IPU, a DPU, a GPU, a GPGPU, or other processing unit (e.g., an accelerator device). An IPU or DPU can include a network interface with one or more programmable pipelines or fixed function processors to perform offloading of operations that can have been performed by a CPU. An IPU or DPU can include one or more memory devices. In some examples, an IPU or DPU can perform virtual switch operations, manage storage transactions (e.g., compression, encryption, virtualization), and manage operations performed on other IPUs, DPUs, servers, or devices.

[0034] In one example, system 500 includes one or more input / output (I / O) interfaces 560. Peripheral interface 570 can include any hardware interface not specifically mentioned above. A peripheral generally refers to a device that relies on system 500 for connection. The slave connection is a connection that system 500 provides an operating software platform or hardware platform or both on which to execute.

[0035] In one example, system 500 includes a storage subsystem 580 that stores data in a non-volatile manner. In one example, at least certain components of storage 580 can overlap with components of memory subsystem 520 in certain system implementations. Storage subsystem 580 includes a storage device 584, which can be or include any conventional medium for storing large amounts of data in a non-volatile manner, such as one or more magnetic, solid-state, or optical based disks or combinations thereof. Storage device 584 holds code or instructions and data 586 in a persistent state (e.g., the value is retained despite power interruptions to system 500). Storage device 584 can generally be considered a “memory,” although memory 530 is generally an execution or operating memory for providing instructions to processor 510. Although storage device 584 is non-volatile, memory 530 can include volatile memory (e.g., the value or state of data is indeterminate if power to system 500 is interrupted). In one example, storage subsystem 580 includes a controller 582 that interfaces with storage device 584. In one example, controller 582 is a physical part of interface 514 or processor 510, or can include circuitry or logic in processor 510 and interface 514.

[0036] In examples, system 500 can be implemented using an interconnected compute chassis of processors, memory, storage devices, network interfaces, and other components. High speed interconnects can be used, such as: Ethernet (IEEE 802.3), Remote Direct Memory Access (RDMA), InfiniBand, Internet Wide Area RDMA Protocol (iWARP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Quick UDP Internet Connections (QUIC), RDMA over Converged Ethernet (RoCE), Peripheral Component Interconnect Express (PCIe), Intel QuickPath Interconnect (QPI), Intel UltraPath Interconnect (UPI), Intel On-Chip System Fabric (IOSF), Omni-Path, Compute Express Link (CXL), HyperTransport, High-Speed Fabric, NVLink, Advanced Microcontroller Bus Architecture (AMBA) Interconnect, OpenCAPI, Gen-Z, Infinity Fabric (IF), Cache Coherent Interconnect for Accelerators (CCIX), 3GPP Long Term Evolution (LTE) (4G), 3GPP 5G, and variants thereof. Data can be copied or stored to virtualized storage nodes, or accessed using protocols such as structure-based Non-Volatile Memory Express (NVMe) over Fabrics (NVMe-oF) or NVMe.

[0037] In some examples, system 500 can be implemented using an interconnected compute node of processors, memory, storage devices, network interfaces, and other components. High speed interconnects can be used, such as PCIe, Ethernet, or optical interconnects (or combinations thereof).

[0038] Embodiments herein can be implemented in various types of computing and networking devices, such as switches, routers, racks, and blade servers, such as those employed in data center and / or server farm environments. Servers used in data centers and server farms include array server configurations, such as rack-based servers or blade servers. These servers are communicatively interconnected via various networking facilities, such as dividing a collection of servers into local area networks (LANs) with appropriate switching and routing facilities between the LANs to form a private intranet. For example, cloud hosting facilities can typically employ large data centers with multiple servers. Blades include individual computing platforms configured to perform server-type functions, i.e., “servers on a card.” Thus, each blade includes components common to conventional servers, including a main printed circuit board (motherboard) that provides internal wiring (e.g., buses) for coupling appropriate integrated circuits (ICs) and other components mounted to the board.

[0039] Various examples can be implemented using hardware elements, software elements, or combinations thereof. In some examples, hardware elements can include devices, components, processors, microprocessors, circuits, circuit elements (e.g., transistors, resistors, capacitors, inductors, etc.), integrated circuits, ASICs, PLDs, DSPs, FPGAs, memory units, logic gates, registers, semiconductor device, chips, microchips, chip sets, and so forth. In some examples, software elements can include software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, APIs, instruction sets, computing code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof.

[0040] Some examples can be implemented using or as articles of manufacture or at least one computer-readable medium. A computer-readable medium can include a non-transitory storage medium to store or record data which can be made available to store or record logic. In some examples, a non-transitory storage medium can include one or more types of computer-readable storage media capable of storing electronic data, including volatile memory or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writeable or re-writeable memory, and the like. In some examples, the logic can include various software elements, such as software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, APIs, instruction sets, computing code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof.

[0041] One or more aspects of at least one example can be implemented by representative instructions stored on at least one machine-readable medium which represents various logic within the processor, which when read by a machine, computing device or system causes the machine, computing device or system to fabricate logic to perform the techniques described herein. Such representations, known as "IP cores" can be stored on a tangible, machine readable medium and supplied to various customers or manufacturing facilities to load into the fabrication machines that actually make the logic or processor.

[0042] The occurrence of the phrase "one example" or "an example" within the specification does not necessarily refer to the same example or embodiment, although it can. Any aspect of the disclosure described herein can be combined in any combination with any other aspect or like aspect described herein, whether or not the combination is expressly described herein. The division, omission or inclusion of a block function in the description of the examples depicted in the drawings does not infer that the hardware components, circuits, software and / or elements for implementing the function necessarily be divided, omitted, or included in embodiments.

[0043] Some examples can be described using the expression “coupled to” and “connected to” along with their derivatives. These terms are not necessarily intended as synonyms for each other. For example, a description using the terms “connected” and / or “coupled” can indicate that two or more elements are in direct physical or electrical contact with each other. The term “coupled,” however, can also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other.

[0044] The terms “first,” “second,” and the like, do not denote any order, quantity, or importance, but are used to distinguish one element from another. The terms “a” and “an” herein do not denote a limitation of quantity, but rather denote the presence of at least one of the referenced item. The term “assertion” is used herein to refer to a state of a signal in which the signal is active, and it can be achieved by applying any logic level of either logic 0 or logic 1 to the signal. The terms “follow” or “after” can refer to immediately after the certain other event. Other sequences of operations can be performed according to alternative embodiments. Moreover, additional operations can be added or existing operations can be removed depending on the particular applications. Any combination of changes can be used and one of ordinary skill in the art having the benefit of this disclosure would understand many variations, modifications, and alternative embodiments.

[0045] Unless specifically stated otherwise, as apparent from the preceding discussion, it is appreciated that throughout the specification, discussions utilizing terms such as “at least one of,” “including,” “including at least,” “comprising,” “comprising at least,” “consisting of,” “consisting essentially of,” “consisting of at least,” “having at least,” “including at least,” “including one of,” “including one or more of,” or “including one of each of a plurality of items later specified,” utilize the

[0046] Illustrative examples of the devices, systems, and methods disclosed herein are provided below. An embodiment of the devices, systems, and methods can include any one or more, and any combination of, the examples described below.

[0047] The flow diagrams illustrated herein provide examples of sequences of various process actions. Flow diagrams can indicate operations to be taken by software or firmware routines and physical operations. In some embodiments, flow diagrams can illustrate states of a finite state machine (FSM) that can be implemented in hardware and / or software. Although shown in a particular sequence or order, unless otherwise specified, the order can be modified. Thus, the illustrated embodiments should be understood only as an example, and that the processes can be performed in a different order, and some actions can be performed in parallel. Additionally, one or more actions can be omitted in various embodiments; thus, not all actions are required in every embodiment. Other processing flows are possible.

[0048] Example 1 includes one or more examples and includes an apparatus comprising: a semiconductor package comprising: a plurality of cores, and a fabric to provide the plurality of cores with access to at least one cache device, wherein: the fabric comprises a set of routers and a plurality of sets of cache controllers, the fabric comprises a set of connections between the set of routers and the plurality of sets of cache controllers, and the fabric comprises a second set of connections between cache controllers of a set of cache controllers, and based on receiving a memory access request from a core of the plurality of cores: a router of the set of routers is to select a set of cache controllers from the plurality of sets of cache controllers based on the memory access request and forward the memory access request to the selected set of cache controllers, and a cache controller of the selected set of cache controllers is to: service the forwarded memory access request at a selected cache device of the at least one cache device based on data associated with the memory access request stored in the selected cache device, and forward the forwarded memory access request to a second cache controller of the selected set of cache controllers based on data associated with the memory access request managed by the second cache controller.

[0049] Example 2 includes one or more examples, wherein the set of connections and the second set of connections are physically arranged in different directions.

[0050] Example 3 includes one or more examples, wherein the memory access request comprises a memory address and an identifier of a set of cache controllers of the plurality of sets of cache controllers to which the memory access request is to be forwarded.

[0051] Example 4 includes one or more examples and includes the at least one cache device communicatively coupled to the fabric.

[0052] Example 5 includes one or more examples wherein, based on a cache miss in the at least one cache device, the second cache controller is to request data associated with the memory access request from a memory device.

[0053] Example 6 includes one or more examples wherein the at least one cache device includes one or more of: a level one cache (LI), a level two cache (L2), a level three cache (L3), or a last level cache (LLC).

[0054] Example 7 includes one or more examples and includes a server, wherein the server includes the plurality of cores and the fabric.

[0055] Example 8 includes one or more examples and includes a method comprising, in a fabric connected to a plurality of processor cores: based on receiving a memory access request from a core of the plurality of cores: selecting a set of cache controllers from a plurality of sets of cache controllers based on the memory access request, forwarding the memory access request to the selected set of cache controllers, servicing the forwarded memory access request at a selected cache device of the at least one cache device based on data associated with the memory access request stored in the at least one cache device, and forwarding the forwarded memory access request to a second cache controller of the selected set of cache controllers based on data associated with the memory access request managed by the second cache controller.

[0056] Example 9 includes one or more examples wherein: the fabric includes a set of routers and the plurality of sets of cache controllers, the fabric includes a set of connections between the set of routers and the plurality of sets of cache controllers, and the fabric includes a second set of connections between cache controllers of a set of cache controllers.

[0057] Example 10 includes one or more examples wherein the memory access request includes a memory address and an identifier of a set of cache controllers of the plurality of sets of cache controllers to which the memory access request is to be forwarded.

[0058] Example 11 includes one or more examples wherein, based on a cache miss in the at least one cache device, data associated with the memory access request is requested from a memory device.

[0059] Example 12 includes one or more examples and includes performing cache coherency to retrieve updated data associated with the memory access request.

[0060] Example 13 includes one or more examples where a package is to enclose the plurality of cores and the fabric.

[0061] Example 14 includes one or more examples where the router is to perform selecting a set of cache controllers from the plurality of sets of cache controllers based on the memory access request, and perform forwarding the memory access request to the selected set of cache controllers.

[0062] Example 15 includes one or more examples where selecting a set of cache controllers from the plurality of sets of cache controllers based on the memory access request includes selecting an output line to the selected set.

Claims

1. An apparatus comprising: a semiconductor package comprising: a plurality of cores, and a fabric to provide the plurality of cores with access to at least one cache device, wherein: the fabric comprises a set of routers and a plurality of sets of cache controllers, the fabric comprises a set of connections between the set of routers and the plurality of sets of cache controllers, and the fabric comprises a second set of connections between cache controllers of a set of cache controllers, wherein: based on receiving a memory access request from a core of the plurality of cores: a router of the set of routers is to select a set of cache controllers from the plurality of sets of cache controllers based on the memory access request and forward the memory access request to the selected set of cache controllers, and a cache controller of the selected set of cache controllers is to: service the forwarded memory access request at a selected cache device of the at least one cache device based on data associated with the memory access request stored in the at least one cache device, and forward the forwarded memory access request to a second cache controller of the selected set of cache controllers based on data associated with the memory access request managed by the second cache controller.

2. The apparatus of claim 1, wherein, the set of connections and the second set of connections are physically arranged in different directions.

3. The apparatus of any one of claims 1-2, wherein, the memory access request comprises a memory address and an identifier of a set of cache controllers of the plurality of sets of cache controllers to which the memory access request is to be forwarded.

4. The apparatus of any one of claims 1-3, comprising the at least one cache device communicatively coupled to the fabric.

5. The apparatus of any one of claims 1-4, wherein, based on a cache miss in the at least one cache device, the second cache controller is to request data associated with the memory access request from a memory device.

6. The apparatus of any one of claims 1-5, wherein, the at least one cache device comprises one or more of: a level one cache (LI), a level two cache (L2), a level three cache (L3), or a last level cache (LLC).

7. The apparatus of any of claims 1-6, comprising a server, wherein, the server comprises the plurality of cores and the fabric.

8. A method comprising: in a fabric connected to a plurality of processor cores: based on receiving a memory access request from a core of the plurality of cores: selecting a set of cache controllers from a plurality of sets of cache controllers based on the memory access request, forwarding the memory access request to the selected set of cache controllers, serving the forwarded memory access request at a selected cache device of at least one cache device based on data associated with the memory access request stored in the at least one cache device, and forwarding the forwarded memory access request to a second cache controller of the selected set of cache controllers based on data associated with the memory access request managed by the second cache controller.

9. The method of claim 8, wherein: The fabric includes a set of routers and the multiple sets of cache controllers, The fabric includes a set of connections between the set of routers and the multiple sets of cache controllers, and The fabric includes a second set of connections between cache controllers of the set of cache controllers.

10. The method of any one of claims 8-9, wherein, The memory access request includes a memory address and an identifier of a set of cache controllers of the multiple sets of cache controllers to which the memory access request is to be forwarded.

11. The method of any one of claims 8-10, wherein, Based on a cache miss in the at least one cache device, data associated with the memory access request is requested from a memory device.

12. The method of any of claims 8-11, comprising: performing cache coherency to retrieve updated data associated with the memory access request.

13. The method of any one of claims 8-12, wherein, A package for enclosing the multiple cores and the fabric.

14. The method of any one of claims 8-13, wherein, The routers perform selecting a set of cache controllers from the multiple sets of cache controllers based on the memory access request and forwarding the memory access request to the selected set of cache controllers.

15. The method of any one of claims 8-14, wherein, Selecting a set of cache controllers from the multiple sets of cache controllers based on the memory access request includes selecting an output line to the selected set.

16. An apparatus comprising: a plurality of cores, and circuitry to provide the plurality of cores with access to at least one cache device, wherein: the circuitry includes a set of routers and multiple sets of cache controllers, the circuitry includes a set of connections between the set of routers and the multiple sets of cache controllers, and the circuitry includes a second set of connections between cache controllers of the set of cache controllers, wherein: based on receiving a memory access request from a core of the plurality of cores: a router of the set of routers is to select a set of cache controllers from the multiple sets of cache controllers based on the memory access request and forward the memory access request to the selected set of cache controllers, and a cache controller of the selected set of cache controllers is to: service the forwarded memory access request at the selected cache device of the at least one cache device based on data associated with the memory access request stored in the at least one cache device, and forward the forwarded memory access request to a second cache controller of the selected set of cache controllers based on data associated with the memory access request managed by the second cache controller.

17. The apparatus of claim 16, wherein, The set of connections and the second set of connections are physically arranged in different directions.

18. The apparatus of any one of claims 16-17, wherein, The memory access request includes a memory address and an identifier of a set of cache controllers of the multiple sets of cache controllers to which the memory access request is to be forwarded.

19. The apparatus of any of claims 16-18, comprising the at least one cache device communicatively coupled with the circuitry.

20. The apparatus of any one of claims 16-19, wherein, Based on a cache miss in the at least one cache device, the second cache controller is to request data associated with the memory access request from a memory device.

21. The apparatus of any of claims 16-20, wherein, The at least one cache device comprises one or more of: a level one cache (LI), a level two cache (L2), a level three cache (L3), or a last level cache (LLC).

22. The apparatus of any of claims 16-21, comprising a server, wherein, The server comprises the plurality of cores and the circuitry. The server comprises the plurality of cores and the circuitry.

Images