Systems, methods, and apparatus for configurable memory management

Abstract

A system may include one or more memory devices configured to at least partially act as system memory; a memory interface; and at least one circuit configured to perform one or more operations, the one or more operations including: receiving a request to configure memory for one or more memory devices; connecting the memory device to the one or more memory devices using the memory interface; and configuring the memory device to operate as a system memory. In some aspects, using a memory interface includes an enabled memory interface, where the enabled memory interface allows a memory device to be added to one or more memory devices. In some aspects, using a memory interface and configuring a memory device are performed while the system remains in a powered-on state.

Description

[0001] Citation of Related Applications

[0002] This application claims priority to and the benefit of U.S. Provisional Patent Application Serial No. 63 / 606,591, filed on December 5, 2023, and U.S. Patent Application Serial No. 18 / 949,984, filed on November 15, 2024, which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates generally to memory management, and more particularly to systems, methods, and apparatus for configurable memory management. Background Art

[0004] Memory management is typically performed during startup of a computing system based on a predefined hardware configuration. For example, before startup of a computing system, a user can modify the hardware (e.g., memory) and set the hardware configuration, and the system will run with the set hardware configuration. If the user wants a different hardware configuration, for example, if more memory is needed, the user can shut down the system, update the hardware and / or configuration, and restart the system to accommodate the change.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the principles of the present invention and therefore it may contain information that does not constitute the prior art. Summary of the invention

[0006] In some aspects, the technology described herein relates to a system comprising: one or more memory devices configured at least in part as system memory; a memory interface; and at least one circuit configured to perform one or more operations, wherein the one or more operations include: receiving a request to configure memory for one or more memory devices; connecting a memory device to one or more memory devices using a memory interface; and configuring the memory device to operate as system memory. In some aspects, using the memory interface includes enabling the memory interface, wherein enabling the memory interface allows a memory device to be added to the one or more memory devices. In some aspects, the at least one circuit is further configured to perform one or more operations, including determining that the memory interface is available for configuring memory for one or more memory devices. In some aspects, the memory interface is a first interface; the request is a first request; and the at least one circuit is further configured to perform one or more operations, including: receiving a second request to configure memory for one or more memory devices; and configuring the second interface to modify an available state of one of the one or more memory devices. In some aspects, using the memory interface includes determining a bandwidth corresponding to the request; and enabling the memory interface based on the bandwidth. In some aspects, the memory interface includes one or more channels; and the at least one circuit is further configured to perform one or more operations, including configuring the one or more channels based on the request. In some aspects, receiving a request to configure memory includes receiving a request to modify an amount of memory bandwidth; and using the memory interface includes using the memory interface based on the amount of memory bandwidth. In some aspects, using the memory interface and configuring the memory device are performed while the system remains in a powered-on state.

[0007] In some aspects, the technology described herein relates to a method comprising: receiving a request to configure memory; determining that resources are available for the request to configure memory; configuring memory based on the request; and modifying the activation state of one or more memory interfaces. In some aspects, receiving the request to configure memory includes receiving a request for the amount of memory to be modified; and configuring memory includes modifying the activation state of one or more memory interfaces based on the amount of memory to be modified. In some aspects, determining that resources are available includes: determining that one or more memory interfaces are available for connecting one or more memory devices configured to operate at least partially as system memory. In some aspects, determining that resources are available includes: determining a bandwidth corresponding to the request, and determining that one or more memory interfaces correspond to the bandwidth; and configuring memory includes modifying the activation state of one or more memory interfaces based on the bandwidth. In some aspects, configuring memory includes enabling one or more channels of the memory interface based on the request. In some aspects, configuring memory is performed while the system configured with the memory remains in a powered-on state.

[0008] In some aspects, the technology described herein relates to a device, the device comprising: a memory interface; and at least one circuit configured to perform one or more operations, the one or more operations comprising: receiving a request to configure a memory of one or more memory devices configured at least in part as system memory; connecting the memory device to the one or more memory devices using the memory interface; and configuring the memory device to operate as system memory. In some aspects, using the memory interface comprises enabling the memory interface, wherein enabling the memory interface allows the memory device to connect to the one or more memory devices. In some aspects, using the memory interface comprises modifying the available state of a memory device in the one or more memory devices. In some aspects, receiving a request to configure the memory comprises receiving a request to increase the amount of memory bandwidth; and the at least one circuit is further configured to perform one or more operations, including enabling the one or more memory interfaces corresponding to the request to increase the amount of memory bandwidth. In some aspects, the memory interface comprises one or more channels; and the at least one circuit is further configured to perform one or more operations, including configuring the one or more channels based on the request. In some aspects, using the memory interface and configuring the memory devices are performed while the system remains in a powered-on state. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The drawings are not necessarily drawn to scale, and in all drawings, for illustrative purposes, elements of similar structure or function may generally be represented by the same reference numerals or portions thereof. The drawings are intended only to facilitate the description of the various embodiments described herein. The drawings do not describe every aspect of the teachings disclosed herein and do not limit the scope of the claims. In order to prevent the drawings from becoming obscure, not all components, connections, etc. may be shown, and not all components may have reference numerals. However, the mode of component configuration can be easily seen from the drawings. The drawings, together with the specification, illustrate example embodiments of the present disclosure and, together with the specification, are used to explain the principles of the present disclosure.

[0010] Figure 1 An embodiment of a memory device scheme according to an example embodiment of the present invention is shown.

[0011] Figure 2 Another embodiment of a memory device scheme according to an example embodiment of the present invention is shown.

[0012] Figure 3 Another embodiment of a memory device scheme according to an example embodiment of the present invention is shown.

[0013] Figure 4a An example of an auxiliary processing unit (xPU) configured with a system memory according to an example embodiment of the present disclosure is shown.

[0014] Figure 4b An example of an xPU configured with an extended memory according to an exemplary embodiment is shown.

[0015] Figure 5 An embodiment of an xPU with configurable memory according to an example embodiment of the present disclosure is shown.

[0016] Figure 6 Another embodiment of an xPU with configurable memory according to an example embodiment of the present disclosure is shown.

[0017] Figure 7 Another embodiment of an xPU with configurable memory according to an example embodiment of the present disclosure is shown.

[0018] Figure 8 An example flow chart of reconfiguring memory according to an example embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0019] Typically, a computing system may have an existing hardware configuration. For example, the system may be provided with certain hardware resources (e.g., computing processing unit (CPU) and graphics processing unit (GPU) cores, memory, storage, etc.). If more resources are needed, the hardware may be reconfigured (e.g., by adding additional memory) to accommodate the new requirements.

[0020] The performance of memory devices has increased over time. For example, in some embodiments, memory devices can be configured as additional memory or extended memory for a computing system. However, although new performance can be introduced, in some embodiments, the hardware may rely on existing methods for its configuration. For example, if a memory device is added to a system, it may need to be configured before the system is started in some embodiments. In some embodiments, if the hardware configuration is to be changed, the system may be shut down, thereby causing the system to be unavailable for a period of time due to the configuration being modified and the system being restarted (e.g., rebooted).

[0021] According to an embodiment of the present disclosure, a configurable memory management module may be added that allows a user to dynamically configure memory for a system. For example, the configurable memory management module may allow memory to be enabled / disabled and the size of a memory pool to be changed without restarting the system. For example, one or more interfaces to a memory device may be activated / deactivated so that a memory device can be added to / removed from a memory pool.

[0022] In some embodiments, the memory management module can be implemented in hardware and / or software. In some embodiments, the memory management module can be implemented on one or more circuits of the memory device. In some embodiments, the one or more circuits can include one or more field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and / or systems on a chip (SoCs).

[0023] Figure 1 An embodiment of a memory device scheme according to an example embodiment of the present invention is shown. Figure 1 The illustrated embodiment may include one or more host devices 100 and one or more memory devices 150 configured to communicate using one or more communication connections 110 .

[0024] In some embodiments, host device 100 may be implemented with any component or combination of components that may utilize one or more features of memory device 150. For example, the host may be implemented with one or more of a server, a storage node, a computing node, a central processing unit (CPU), a workstation, a personal computer, a tablet computer, a smart phone, and / or the like, or multiples and / or combinations thereof.

[0025] In some embodiments, the memory device 150 may include a communication interface 130, a memory 180 (some or all of which may be referred to as device memory), one or more computing resources 170 (which may also be referred to as computing resources), a device controller 160, and / or a device functional circuit 190. In some embodiments, the device controller 160 may control the overall operation of the memory device 150, including any operations, features, and / or the like described herein. For example, in some embodiments, the device controller 160 may parse, process, invoke, and / or perform similar operations on commands received from the host device 100.

[0026] In some embodiments, the device functional circuit 190 may include any hardware to implement the main functions of the memory device 150. For example, the device functional circuit 190 may include storage media, such as magnetic media (e.g., if the memory device 150 is implemented as a hard disk drive (HDD) or a tape drive), solid-state media (e.g., one or more flash memory devices), optical media, and / or similar storage media. For example, in some embodiments, the memory device may be implemented at least in part as a solid-state drive (SSD) based on non-AND (NAND) flash memory, a persistent memory (PMEM) (such as a cross-grid non-volatile memory), a memory with body resistance change, a phase change memory (PCM), or any combination thereof. In some embodiments, the device controller 160 may include a media conversion layer, such as a flash translation layer (FTL) for interfacing with one or more flash memory devices. In some embodiments, the memory device 150 may be implemented as a computational storage drive, a computational storage processor (CSP), and / or a computational storage array (CSA).

[0027] As another example, if the memory device 150 is implemented as an accelerator, the device functional circuitry 190 may include one or more accelerator circuits, memory circuits, and / or the like.

[0028] The computing resources 170 may be implemented with any component or combination of components that can perform operations on data that may be received, stored, and / or generated at the memory device 150. Examples of computing engines may include combinatorial logic, sequential logic, timers, counters, registers, state machines, complex programmable logic devices (CPLDs), FPGAs, ASICs, embedded processors, microcontrollers, central processing units (CPUs) such as complex instruction set computer (CISC) processors (e.g., x86 processors) and / or reduced instruction set computer (RISC) processors such as ARM processors, graphics processing units (GPUs), data processing units (DPUs), neural processing units (NPUs), tensor processing units (TPUs), and / or the like, which may run instructions stored in any type of memory and / or implement any type of execution environment such as containers, virtual machines, operating systems (such as Linux), extended Berkeley packet filter (eBPF) environments, and / or similar execution environments, or combinations thereof.

[0029] In some embodiments, memory 180 may be used, for example, by one or more of computing resources 170 to store input data, output data (e.g., computational results), intermediate data, transitional data, and / or the like. Memory 180 may be implemented, for example, with volatile memory such as dynamic random access memory (DRAM), static random access memory (SRAM), and / or similar volatile memory, as well as any other type of memory such as non-volatile memory.

[0030] In some embodiments, the memory 180 and / or computing resources 170 may include software, instructions, programs, codes, and / or the like, which may be executed, run, and / or the like using one or more computing resources (e.g., hardware (HW) resources). Examples may include software implemented in any language such as assembly language, C, C++, and / or the like, binary code, FPGA code, one or more operating systems, kernels, environments such as eBPF, and / or the like. The software, instructions, programs, codes, and / or the like may be stored in, for example, a repository in the memory 180 and / or computing resources 170. In some embodiments, the software, instructions, programs, codes, and / or the like may be downloaded, uploaded, side-loaded, pre-installed, built-in, and / or in a similar manner to the memory 180 and / or computing resources 170. In some embodiments, the memory device 150 may receive one or more instructions, commands, and / or the like to select, enable, activate, run, and / or the like software, instructions, programs, codes, and / or the like. Examples of computational operations, functions, and / or the like that may be implemented by the memory 180, computing resources 170, software, instructions, programs, code, and / or the like may include any type of algorithm for artificial intelligence (AI), machine learning (ML), neural networks, and / or the like, data movement, data management, data selection, filtering, encryption and / or decryption, compression and / or decompression, checksum calculations, hash value calculations, cyclic redundancy checks (CRCs), weight calculations, activation function calculations, training, inference, classification, regression, and / or the like.

[0031] In some embodiments, the communication interface 120 at the host device 100, the communication interface 130 at the memory device 150, and / or the communication connection 110 may use any type of interface, protocol, and / or the like, implement one or more interconnects, one or more networks, networks of networks (e.g., the Internet), and / or the like, or a combination thereof, and / or use any type of interface, protocol, and / or the like, utilize one or more interconnects, one or more networks, networks of networks (e.g., the Internet), and / or the like, or a combination thereof. For example, the communication connection 110 and / or one or more of the interfaces 120 and / or 130 may implement any type of wired and / or wireless communication media, interfaces, networks, interconnects, protocols, and / or the like, and / or be implemented utilize any type of wired and / or wireless communication media, interfaces, networks, interconnects, protocols, and / or the like, including Peripheral Component Interconnect Express (PCIe), NVMe, NVMe over Ethernet, and / or the like. Fabric (NVMe-oF), Compute Express Link (CXL) and / or coherent protocols (such as CXL.mem, CXL.cache, CXL.io and / or similar protocols, Gen-Z, Open Coherent Accelerator Processor Interface (OpenCAPI), Cache Coherent Interconnect for Accelerators (CCIX) and / or similar protocols), Advanced Extensible Interface (AXI), Direct Memory Access (DMA), Remote DMA (RDMA), RDMA over Converged Ethernet (ROCE), Advanced Message Queuing Protocol (AMQP), Ethernet, Transmission Control Protocol / Internet Protocol (TCP / IP), Fibre Channel, InfiniBand, Serial ATA (SATA), Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), iWARP, any generation of wireless networks including 2G, 3G, 4G, 5G, 6G and / or similar networks, any generation of Wi-Fi, Bluetooth, Near Field Communication (NFC) and / or the like, or any combination thereof. In some embodiments, the communication connection 110 may include one or more switches, hubs, nodes, routers and / or the like.

[0032] In some embodiments, the memory device 150 can be implemented in any physical form factor. Examples of form factors may include 3.5 inches, 2.5 inches, 1.8 inches, and / or the like, a memory device (e.g., a storage drive) form factor, an M.2 device form factor, an Enterprise and Data Center Standard Form Factor (EDSFF) (which may include, for example, E1.S, E1.L, E3.S, E3.L, E3.S2T, E3.L 2T, and / or the like), an Add-In Card (AIC) (e.g., a PCIe card (e.g., a PCIe expansion card) form factor, including half-height (HH), half-length (HL), half-height, half-length (HHHL), and / or the like), a Next Generation Small Form Factor (NGSFF), an NF1 form factor, a Compact Flash (CF) form factor, a Secure Digital (SD) card form factor, a Personal Computer Memory Card International Association (PCMCIA) device form factor, and / or the like, or a combination thereof. Any computing device disclosed herein can be connected to a system using one or more connectors, such as a SATA connector, a SCSI connector, a SAS connector, an M.2 connector, an EDSFF connector (e.g., 1C, 2C, 4C, 4C+, and / or the like), a U.2 connector (which may also be referred to as an SSD Form Factor (SSF) SFF-8639 connector), a U.3 connector, a PCIe connector (e.g., a card edge connector), and / or the like.

[0033] Any memory device disclosed herein may be used in conjunction with one or more personal computers, smart phones, tablet computers, servers, server chassis, server racks, data rooms, data centers, edge data centers, mobile edge data centers, and / or any combination thereof.

[0034] In some embodiments, memory device 150 may be implemented with any device that may include or have access to memory, storage media, and / or the like to store data that may be processed by one or more computing resources 170. Examples may include memory extension and / or buffer devices (such as CXL Type 2 and / or CXL Type 3 devices), as well as CXL Type 1 devices that may include memory, storage media, and / or the like.

[0035] In some embodiments, one or more memory devices 150 may form a memory pool. Typically, a memory pool is a logical grouping of memory devices that a system can access. In some embodiments, when one or more memory devices 150 are used as extended memory and / or system memory, one or more memory devices 150 may also form a memory pool. In some embodiments, a memory pool and / or a memory pool may be accessed as a pool and / or a separate memory device.

[0036] Figure 2 Another embodiment of a memory device scheme according to an example embodiment of the present invention is shown. Figure 2 The elements shown in the figure may be Figure 1 , where similar elements may be indicated by reference numerals ending with and / or containing the same numbers, letters, and / or similar symbols. In some embodiments, host device 100 may include system memory 210; and memory device 150 may include controller 160, memory medium 260, and / or storage medium 270. In some embodiments, controller 160 may be implemented on one or more circuits of memory device 150. In some embodiments, one or more circuits may include one or more FPGAs, ASICs, and / or SoCs.

[0037] In some embodiments, the memory medium 260 may be a relatively fast memory such as DRAM, and the storage medium 270 may be a slower non-volatile memory such as NAND flash memory. In some embodiments, the memory medium 260 may be used as a cache to store frequently accessed data in a faster memory. In some embodiments, the host device 100 may use a memory access request 220 to retrieve data from the memory medium 260. In particular, in some embodiments, in response to receiving the memory access request 220, the memory device 150 may send a request to the controller 160 to check the memory medium 260 for data corresponding to the request. In some embodiments, in response to a cache hit (e.g., data is found on the memory medium 260), data may be returned from the memory medium 260. In some embodiments, in response to a cache miss (e.g., data is not found on the memory medium 260), the controller 160 may copy data from the storage medium 270 to the memory medium 260 and return data from the memory medium 260.

[0038] In some embodiments, the memory device 150 may be advertised as system memory (e.g., device memory). In other words, the memory device 150 may appear as an additional memory node to the host device 100 and may be managed by the OS non-uniform memory architecture (NUMA) memory management. In some embodiments, when the memory device 150 is configured as an extended memory, the host device 100 may write data to a local device memory on the host device 100 or on the memory device 150. In some embodiments, since the memory device 150 may use the memory medium 260, the host device 100 may not experience a large amount of waiting time when accessing the memory medium 260 compared to accessing the local device memory by virtue of its fast speed. In some embodiments, this may provide additional memory capacity to the host device 100 at a lower cost than adding additional device memory to the host device 100.

[0039] In some embodiments, the demand for large memory capacity and / or bandwidth may increase due to the widespread adoption of large artificial intelligence (AI) models and data analysis and in-memory databases. Memory devices (such as memory devices 150) can be used as a substitute for system memory 210 to expand memory capacity because memory devices are generally cheaper than system memory. However, in some embodiments, there may be no memory bandwidth and / or capacity expansion capabilities with configurable hardware. For example, memory management is typically performed with static physical memory capacity, which is determined during boot based on a predefined static hardware configuration (e.g., double data rate (DDR), high bandwidth memory (HBM), etc.).

[0040] In some embodiments, the maximum transmission speed between the host device 100 and the memory configured as the extended memory on the memory device 150 can be determined by the number of ports and / or channels assigned to the extended memory. For example, if the port has 16 channels, each of which has a transmission speed of 4GB / s, the port can support a transmission speed of 64GB / s. In some embodiments, the hardware configuration can allocate a certain number of ports and / or channels to be used as extended memory. Therefore, for example, if two ports with 16 channels are allocated to be used as extended memory, the memory device 150 can support 128GB / s. However, in some embodiments, if more bandwidth is needed, the port can be configured so that more ports and / or channels can be allocated to be used as extended memory.

[0041] Figure 3 Another embodiment of a memory device scheme according to an example embodiment of the present invention is shown. Figure 3 The elements shown in the figure may be Figure 1 and Figure 2, where similar elements may be indicated by reference numerals ending with and / or containing the same numbers, letters, and / or similar symbols. In some embodiments, host device 100 may include one or more cores 310; and one or more memory devices 150 may be configured to connect to the host device using one or more interfaces (not shown). Figure 3 3. However, it should be understood that a single core 310 is shown for illustrative purposes and that the host may have multiple cores, as will be described in further detail below.

[0042] In some embodiments, the host device 100 can be physically connected to one or more memory devices 150. However, in some embodiments, not all interfaces can be activated (e.g., no power is supplied to the interface). Thus, one or more interfaces can be in an activated state 320, and one or more interfaces can be in a deactivated state 330. As will be described in further detail below, a configurable memory management module can allow an interface to be set to an activated / deactivated state and / or allow enabled / disabled memory devices to be added / removed from a memory pool without shutting down the system. In some embodiments, a memory device can remain connected to the system, but because the interface is in a deactivated state, this can allow the memory device to be unavailable to the system (e.g., changing the available state of the memory device).

[0043] Figure 4a and 4b An example of an xPU (e.g., a CPU, a GPU, an FPGA, a SoC, a neural processing unit (NPU), a visual processing unit (VPU), an ASIC, etc.) and corresponding memory according to an example embodiment of the present disclosure is shown. Figure 4a In the embodiment, an xPU (e.g., a central processing unit (CPU) or a graphics processing unit (GPU)) may be configured with one or more memory interfaces 410, which may be connected to a memory device such as Figure 2 Thus, in some embodiments, the xPU may be configured with a fixed amount of memory using one or more memory interfaces 410. In some embodiments, the number of memory interfaces may be fixed (e.g., all of the one or more memory interfaces 410 may be powered on). In some embodiments, a memory controller (not shown) may be located on the host to control the system memory.

[0044] exist Figure 4b In the embodiment, the xPU may be configured with one or more interfaces 420 that communicate with a memory such as a memory device. Figure 4bIn some embodiments, some of the one or more interfaces 420 may not be powered. For example, some of the interfaces may be available but not in use, and therefore not powered. Thus, in some embodiments, each memory interface may be powered or not powered. Figure 4a and Figure 4b In both, the system can be configured so that a process can be assigned multiple cores and a certain amount of memory. In some embodiments, if a hardware change is required, the system can be powered off so that the configuration can be updated. For example, if the process is associated with a virtual machine (VM), the virtual machine can be assigned multiple GPU and CPU cores, memory and / or storage, and when the hardware requirements change, the virtual machine can be shut down, the configuration updated, and the virtual machine restarted to adapt to the new configuration. In some embodiments, the controller can be on the memory device 150, such as Figure 2 In some embodiments, when the controller 160 is on the memory device 150, no controller may be required on the host.

[0045] Figure 5 An example of an xPU with configurable memory according to an example embodiment of the present disclosure is shown. Figure 5 The elements shown in the figure may be Figure 4b Elements similar to the elements shown in , where similar elements can be indicated by reference numerals ending with and / or containing the same numbers, letters and / or similar symbols. Figure 5 Logic 520 is included. In some embodiments, logic 520 can be a circuit that enables memory to be configurable, for example, by enabling or disabling a port. For example, if a VM has a hardware configuration (e.g., CPU cores and memory), logic 520 can be used to change the amount of memory available to the VM by enabling or disabling a port. Figure 5 Each interface 420 is shown in communication with a memory 510. However, the interfaces 420 need not be in a one-to-one relationship with the memory 510, and may be in a one-to-many, many-to-many, and / or many-to-one relationship.

[0046] In some embodiments, logic 520 may be included in the xPU design. In some embodiments, logic 520 may be implemented as hardware and / or software. In some embodiments, dynamic memory adjustment using configurable hardware capabilities may be provided in addition to existing software-driven static memory management.

[0047] Figure 6 Another example of an xPU with configurable memory according to an example embodiment of the present disclosure is shown. Figure 6 The elements shown in the figure may be Figure 5Elements similar to the elements shown in , where similar elements can be indicated by reference numerals ending with and / or containing the same numbers, letters and / or similar symbols. Figure 6 Dynamic configuration of different types of memory may be allowed. For example, memory 610 may correspond to DRAM, memory 620 may correspond to HBM, and memory 630 may correspond to a memory device configured as storage. Thus, in some embodiments, logic 520 may be used to configure different memory types and interfaces, thereby allowing the system to dynamically set memory bandwidth and / or capacity.

[0048] As used herein, "interface" may refer to a mechanism that allows data to be transmitted between devices. For example, an interface may use a physical layer to transmit a protocol between devices. In some embodiments, an interface may be hardware and / or software. As used herein, a "port" may refer to a connection between devices that allow devices to be connected to each other. In some embodiments, a port may have multiple channels (e.g., 16 channels) that provide a data path between devices. It should be understood that the terms "interface" and "port" may be used interchangeably because both refer to a mechanism that allows devices to communicate with each other.

[0049] In some embodiments, an algorithm for dynamic port management logic may be provided. For example, in some embodiments, an application and / or an operating system (OS) may communicate a memory requirement (e.g., higher or lower). In some embodiments, the system may check for available ports. In some embodiments, the system may configure memory by enabling / disabling memory ports.

[0050] In some embodiments, low power states (e.g., L0p) can be used to further configure memory bandwidth. For example, individual channels in a port can be enabled / disabled to allow for finer granularity of memory configuration. In some embodiments, the system can make changes to the configuration without disrupting real-time traffic.

[0051] Figure 7 Another example of an xPU with configurable memory according to an example embodiment of the present disclosure is shown. Figure 7 The elements shown in the figure may be Figure 5 and Figure 6 Elements similar to the elements shown in , where similar elements can be indicated by reference numerals ending with and / or containing the same numbers, letters and / or similar symbols. Figure 7 Included are interfaces 710 and 720 and a switch fabric 730. Interfaces 710 and 720 may communicate with corresponding interface 420. In some embodiments, switch fabric 730 may enable interface 420 and interfaces 710 and 720 to be interconnected.

[0052] In some embodiments, the switch fabric 730 may be an architecture that allows switches to connect to other switches and nodes to connect to other nodes. In some embodiments, the switch fabric 730 may be enabled in hardware and / or software. In some embodiments, the switch fabric 730 may enable data to be transferred between the interface 420 on the host and the interfaces 710 and 720 on the memories 510 and 530. In particular, the switch fabric 730 may enable the exchange between the host and the memories 510 and 530 so that there is no need to maintain a one-to-one relationship between the interface 420 and the interfaces 710 and 720, because the connection between the devices may be maintained by the switch fabric 730.

[0053] exist Figure 7 , memory 510 and memory 530 are shown. It should be understood that memories 510 and 530 do not need to be different memories and can be memories of the same type. In some embodiments, memory 510 can correspond to one type of memory device, and memory 530 can correspond to different types of memory devices. In addition, memory 510 can correspond to a memory for one purpose (e.g., as an extended memory), and memory 530 can correspond to a memory for different purposes. Within the scope of the present disclosure, memories 510 and 530 can be any type of memory device that communicates with the host.

[0054] Figure 8 An example flow chart for reconfiguring memory according to an example embodiment of the present disclosure is shown. For example, at box 810, according to an embodiment, a device may receive a request to configure memory. In some embodiments, the device may receive a request from an application or OS to increase or decrease the amount of memory. For example, for AI-related operations, additional memory may be required to perform some operations. However, the system may be limited by the amount of system memory and attached memory devices. In some embodiments, additional memory devices may be added and / or interfaces may be enabled or disabled to provide the requested memory without shutting down the system to enable configuration changes. In addition, since the memory pool may be limited by the total amount of system memory and storage devices, when the application and / or OS no longer needs the memory, the configuration may be changed to reduce the allocated memory, thereby freeing it for other operations.

[0055] At frame 820, according to an embodiment, the device may determine that resources are available for the request to configure the memory. For example, the device may check available ports. In some embodiments, some ports may be available, but the corresponding interfaces may not be enabled. In some embodiments, the host may have one or more enabled interfaces and one or more disabled interfaces, wherein the enabled interfaces allow the host to communicate with the memory device communicating with the interface, and the disabled interfaces prevent the host from communicating with the memory device. In some embodiments, the device may determine whether the port is available and / or used by another process.

[0056] At block 830, depending on the embodiment, memory may be configured based on the request. In some embodiments, based on the memory in the memory pool, the device may determine whether an interface may be activated or deactivated based on the memory required. For example, based on the request and the memory available in the memory pool, the device may determine the number of interfaces to activate and / or deactivate. If the bandwidth needs to be modified, the number of interfaces to activate and / or deactivate may be changed based on the bandwidth.

[0057] At box 840, according to an embodiment, the activation state of one or more memory interfaces can be modified. For example, in some embodiments, one or more interfaces can be enabled or disabled. In addition, channels for interfaces can be enabled or disabled. This can allow applications or OS to dynamically request more or less memory without stopping the system. For example, the amount of available memory and the amount of requested memory can be calculated. In some embodiments, one or more circuits (such as a management configuration module) can be used to determine whether to increase or decrease the amount of available memory and the amount of memory to be increased or decreased. In some embodiments, one or more memory interfaces can be enabled or disabled based on the amount of memory to be increased or decreased. In addition, in some embodiments, if more precise memory control is required, individual channels can be enabled or disabled to provide the amount of requested memory to be increased or decreased.

[0058] Therefore, in some embodiments, the memory can be dynamically configured to minimize system downtime. Figure 1 The memory devices 150 in the system are typically cheaper per GB than system memory, so the total cost of ownership can be reduced by using the memory devices to provide large memory capacity (e.g., extended memory).

[0059] In some embodiments, the cache medium is accessible by software using load and / or store instructions, and the storage medium is accessible by software using read and / or write instructions.

[0060] In some embodiments, the cache medium may be accessed using memory interfaces and / or protocols such as any generation of DDR (e.g., DDR4, DDR5, etc.), DMA, RDMA, Open Memory Interface (OMI), CXL, Gen-Z, and / or the like, while the storage medium may be accessed using interfaces and / or protocols such as Serial ATA (SATA), Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), NVMe, NVMe-oF, and / or the like.

[0061] Although some embodiments may be described in the context of cache media that may be implemented with cache media such as DRAM, in other embodiments, other types of media (e.g., storage media) may be used for cache media. For example, in some embodiments, some or all of memory media 260 may be implemented with media other than cache media that may have one or more relative characteristics (e.g., relative to storage media 270) that may make one or both of them more suitable for their respective functions. For example, in some embodiments, storage media 270 may have relatively high capacity, low cost, and / or similar characteristics, while some or all of memory media 260 may have relatively low access latency, which may make them relatively more suitable for use as a cache.

[0062] The memory device 150, as well as any other device disclosed herein, may be used in conjunction with one or more personal computers, smart phones, tablet computers, servers, server chassis, server racks, data rooms, data centers, edge data centers, mobile edge data centers, and / or any combination thereof.

[0063] Any functionality described herein, including any user functionality, device functionality, and / or the like (e.g., any control logic), may be implemented in hardware, software, firmware, or any combination thereof, including, for example, hardware and / or software combinational logic, sequential logic, timers, counters, registers, state machines, volatile memory (such as DRAM and / or SRAM), non-volatile memory (including flash memory, persistent memory (such as cross-grid non-volatile memory), memory with body resistance change, PCM and / or similar memory, and / or any combination thereof), complex programmable logic device (CPLD), FPGA, ASIC, central processing unit (CPU) including CISC processor (such as x86 processor) and / or RISC processor (such as ARM processor), graphics processing unit (GPU), neural processing unit (NPU), tensor processing unit (TPU), data processing unit (DPU), and / or similar devices, which run instructions stored in any type of memory. In some embodiments, one or more components may be implemented as a SoC.

[0064] Some embodiments disclosed above have been described in the context of various implementation details, such as devices implemented as memory devices that can use specific interfaces, protocols and / or similar memory devices, but the principles of the present disclosure are not limited to these or any other specific details. For example, some functions have been described as being implemented by certain components, but in other embodiments, functions can be distributed between different systems and components in different locations and have various user interfaces. Certain embodiments have been described as having specific processes, operations, etc., but these terms also cover embodiments in which specific processes, operations, etc. can be implemented with multiple processes, operations, etc., or embodiments in which multiple processes, operations, etc. can be integrated into a single process, step, etc. References to components or elements may refer only to a portion of a component or element. For example, references to blocks may refer to the entire block or one or more sub-blocks. Terms such as "first" and "second" used in the present disclosure and claims may be used only for the purpose of distinguishing the elements they modify, and may not indicate any spatial or temporal order unless it is obvious from the context. In some embodiments, references to elements may refer to at least a portion of an element, for example, "based on" may refer to "at least partially based on" and / or similar. References to the first element may not imply the presence of the second element. The principles disclosed herein have independent utility and can be implemented separately, and not every embodiment can utilize every principle. However, the principles can also be embodied in various combinations, some of which can amplify the benefits of the various principles in a synergistic manner. The various details and embodiments described above can be combined to produce additional embodiments according to the inventive principles disclosed in this patent.

[0065] In some embodiments, a portion of an element may refer to less than or all of the element. A first portion of an element and a second portion of an element may refer to the same portion of an original. A first portion of an element and a second portion of an element may overlap (e.g., a portion of the first portion may be the same as a portion of the second portion).

[0066] In the embodiments described herein, the operations are example operations and may involve various additional operations that are not explicitly shown. In some embodiments, some of the operations shown may be omitted. In some embodiments, one or more of the operations may be performed by components other than those shown herein. In addition, in some embodiments, the time sequence of the operations may be changed. Furthermore, the drawings are not necessarily drawn to scale.

[0067] The principles disclosed herein may have independent utility and may be implemented separately, and not every embodiment may utilize every principle. However, the principles may also be embodied in various combinations, some of which may amplify the benefits of the various principles in a synergistic manner.

[0068] In some embodiments, the latency of a memory device may refer to the delay between the memory device and the processor when accessing the memory. In addition, latency may include delays caused by hardware, such as the read and write speeds of accessing the memory device, and / or the structure of the arrayed memory device may cause various delays when reaching various elements of the array. For example, a first memory device in the form of a DRAM may have a faster read / write speed than a second memory device in the form of a NAND device. In addition, the latency of the memory device may change over time based on conditions (such as relative network load), as well as the performance of the memory device over time and environmental factors (such as temperature changes affecting delays in signal paths).

[0069] Although some example embodiments may be described in the context of specific implementation details, such as a processing system that may implement a NUMA architecture, memory devices and / or pools that may be connected to the processing system using an interconnect interface and / or protocol CXL and / or the like, the principles are not limited to these example details and may be implemented using any other type of system architecture, interface, protocol, and / or the like. For example, in some embodiments, one or more memory devices may be connected using any type of interface and / or protocol, wherein the interface and / or protocol includes a peripheral component interconnect express bus (PCIe), a non-volatile memory express bus (NVMe), NVMe over Fabric (NVMe oF), an advanced extensible interface (AXI), an ultrapath interconnect (UPI), Ethernet, a transmission control protocol / internet protocol (TCP / IP), a remote direct memory access (RDMA), RDMA over converged Ethernet (ROCE), Fibre Channel, InfiniBand, Serial ATA (SATA), a small computer system interface (SCSI), a serial attached SCSI (SAS), iWARP, and / or the like, or any combination thereof. In some embodiments, the interconnect interface may be implemented with one or more memory semantic and / or memory coherent interfaces and / or protocols, including one or more CXL protocols (such as CXL.mem, CXL.io, and / or CXL.cache), Gen-Z, Coherent Accelerator Processor Interface (CAPI), Cache Coherent Interconnect for Accelerators (CCIX), and / or the like, or any combination thereof. Any memory device may be implemented with one or more of any type of memory device interface, including DDR, DDR2, DDR3, DDR4, DDR5, LPDDRX, Open Memory Interface (OMI), NVLink, HBM, HBM2, HBM3, and / or the like.

[0070] In some embodiments, any of the memory devices, memory pools, hosts, and / or the like, or their components, may be implemented in any physical and / or electrical configuration and / or form factor, such as a stand-alone device, an add-on card such as a PCIe adapter or expansion card, a plug-in device that can be inserted into a connector and / or slot of a server chassis (e.g., a connector on a backplane and / or midplane of a server or other device), and / or the like. In some embodiments, any of the memory devices, memory pools, hosts, and / or the like, or their components, may be implemented in a form factor for a memory device (such as 3.5 inches, 2.5 inches, 1.8 inches, M.2, Enterprise and Data Center SSD Form Factor (EDSFF), NF1, and / or the like), using any connector configuration for an interconnect interface (such as a SATA connector, a SCSI connector, a SAS connector, an M.2 connector, a U.2 connector, a U.3 connector, and / or the like). Any device disclosed herein may be implemented and / or used in conjunction with a server chassis, a server rack, a data room, a data center, an edge data center, a mobile edge data center, and / or any combination thereof, in whole or in part. In some embodiments, any of the memory devices, memory pools, hosts, and / or the like, or components thereof, may be implemented as CXL Type 1 devices, CXL Type 2 devices, CXL Type 3 devices, and / or the like.

[0071] In some embodiments, any functionality described herein, including, for example, any logic for implementing tiering, device selection, and / or similar functionality), may be implemented in hardware, software, or a combination thereof, including combinatorial logic, sequential logic, one or more timers, counters, registers, and / or state machines, one or more CPLDs, FPGAs, ASICs, CPUs (such as CISC processors (such as x86 processors) and / or RISC processors (such as ARM processors)), GPUs, NPUs, TPUs, and / or the like, which execute instructions stored in any type of memory, or any combination thereof. In some embodiments, one or more components may be implemented as a SoC.

[0072] In the present disclosure, many specific details are set forth in order to provide a thorough understanding of the present disclosure, but the disclosed aspects can be practiced without these specific details. In other instances, well-known methods, processes, components, and circuits are not described in detail to avoid obscuring the subject matter disclosed herein.

[0073] References throughout this specification to "one embodiment" or "an embodiment" mean that the specific features, structures, or characteristics described in conjunction with the embodiment may be included in at least one embodiment disclosed herein. Therefore, the phrases "in one embodiment" or "in an embodiment" or "according to an embodiment" (or other phrases with similar meanings) that appear in various places throughout this specification may not necessarily all refer to the same embodiment. In addition, specific features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In this regard, as used herein, the word "exemplary" means "used as an example, instance, or illustration". Any embodiment described herein as "exemplary" should not be interpreted as necessarily being preferred or advantageous over other embodiments. In addition, specific features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In addition, depending on the context discussed herein, a singular term may include a corresponding plural form, and a plural term may include a corresponding singular form. Similarly, hyphenated terms (e.g., "two-dimensional", "pre-determined", "pixel-specific", etc.) may occasionally be used interchangeably with corresponding non-hyphenated versions (e.g., "two dimensional", "pre determined", "pixel specific", etc.), and uppercase terms (e.g., "Counter Clock", "Row Select", "PIXOUT", etc.) may be used interchangeably with corresponding non-uppercase versions (e.g., "counter clock", "row select", "pixout", etc.). Such occasional interchangeable usages should not be considered inconsistent with each other.

[0074] In addition, depending on the context discussed herein, singular terms may include corresponding plural forms, and plural terms may include corresponding singular forms. It should also be noted that the various drawings (including component drawings) shown and discussed herein are for illustrative purposes only and are not drawn to scale. For example, the dimensions of some elements may be exaggerated relative to other elements for clarity. In addition, if deemed appropriate, reference numerals are repeated in the drawings to indicate corresponding and / or similar elements.

[0075] The terms used herein are for the purpose of describing some example embodiments only and are not intended to limit the claimed subject matter. As used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. When used in this specification, the terms "comprises" and / or "comprising" specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0076] When an element or layer is referred to as being on, "connected to," or "coupled to" another element or layer, it may be directly on, connected to, or coupled to another element or layer, or there may be intermediate elements or layers. In contrast, when an element is referred to as being "directly on," "directly connected to," or "directly coupled to" another element or layer, there are no intermediate elements or layers. The same reference numerals always represent the same elements. As used herein, the term "and / or" may include any and all combinations of one or more of the associated listed items.

[0077] As used herein, the terms "first," "second," and the like are used as labels for the nouns that follow them and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) unless explicitly so defined. In addition, the same figure numbers may be used across two or more figures to refer to parts, components, blocks, circuits, units, or modules having the same or similar functions. However, this usage is only for simplicity of illustration and ease of discussion; it does not mean that the construction or architectural details of such components or units are the same in all embodiments, or that such commonly referenced parts / modules are the only way to implement some example embodiments disclosed herein.

[0078] The term "module" may refer to any combination of software, firmware, and / or hardware configured to provide the module-related functions described herein. For example, software may be embodied as a software package, code, and / or instruction set or instructions, and the term "hardware" as used in any embodiment described herein may include, for example, components, hardwired circuits, programmable circuits, state machine circuits, and / or firmware storing instructions run by programmable circuits, either alone or in any combination. Modules may be implemented together or individually as circuits forming part of a larger system, such as, but not limited to, integrated circuits (ICs), SoCs, components, and the like. The embodiments of the subject matter and operations described in this specification may be implemented in digital electronic circuits, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or a combination of one or more of them. The embodiments of the subject matter described in this specification may be implemented as one or more computer programs, such as one or more modules of computer program instructions, which are encoded on a computer storage medium for operation by a data processing device or for controlling the operation of a data processing device. Alternatively or additionally, program instructions may be encoded on an artificially generated propagation signal, such as a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device for operation by a data processing device. A computer storage medium may be or be included in a computer-readable memory device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination thereof. In addition, although a computer storage medium is not a propagation signal, a computer storage medium may be a source or destination of computer program instructions encoded in an artificially generated propagation signal. A computer storage medium may also be one or more separate physical components or media (e.g., multiple CDs, disks, or other memory devices), or be included therein. In addition, the operations described in this specification may be implemented as operations performed by a data processing device on data stored on one or more computer-readable memory devices or received from other sources.

[0079] Although this specification may include many specific implementation details, the implementation details should not be interpreted as a limitation on the scope of any claimed subject matter, but rather as a description of features specific to a particular embodiment. Certain features described in the context of a separate embodiment in this specification may also be implemented in combination in a single embodiment. On the contrary, the various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination. In addition, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from the claimed combination may be deleted from the combination in some cases, and the claimed combination may be directed to a sub-combination or a variation of the sub-combination.

[0080] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that the operations be performed in the particular order shown or in sequence, or that all of the operations shown be performed, to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0081] Thus, specific embodiments of the subject matter have been described herein. Other embodiments are within the scope of the following claims. In some cases, the actions set forth in the claims can be performed in a different order and still achieve the desired results. Additionally, the processes depicted in the accompanying drawings do not necessarily require the particular order or sequence shown to achieve the desired results. In certain embodiments, multitasking and parallel processing may be advantageous.

[0082] Although certain exemplary embodiments have been described and shown in the accompanying drawings, it should be understood that such embodiments are merely illustrative, and the scope of the present disclosure is not limited to the embodiments described or shown herein. The present invention may be modified in arrangement and detail without departing from the concept of the present invention, and such changes and modifications are considered to fall within the scope of the appended claims.

Claims

1. A system comprising: one or more memory devices configured to function at least in part as system memory; Memory interface; as well as At least one circuit is configured to perform one or more operations including: receiving a request to configure memory for the one or more memory devices; connecting a memory device of the one or more memory devices to a host device using the memory interface; and The memory device is configured to operate as system memory.

2. The system according to claim 1, wherein: Using the memory interface includes enabling the memory interface, wherein enabling the memory interface allows the memory device to be added as system memory.

3. The system according to claim 1, wherein: The at least one circuit is further configured to perform one or more operations including determining that the memory interface is usable to configure memory for the one or more memory devices.

4. The system of claim 1, wherein: The memory interface is a first interface; The request is a first request; as well as The at least one circuit is further configured to perform one or more operations including: receiving a second request to configure memory for the one or more memory devices; as well as The second interface is configured to modify an available state of one of the one or more memory devices.

5. The system according to claim 1, in, Using the memory interface includes: determining a bandwidth corresponding to the request; and The memory interface is enabled based on the bandwidth.

6. The system according to claim 1, in, The memory interface includes one or more channels; and The at least one circuit is further configured to perform one or more operations including configuring the one or more channels based on the request.

7. The system according to claim 1, wherein: Receiving a request to configure memory includes receiving a request to modify an amount of memory bandwidth; and using the memory interface includes using the memory interface based on the amount of memory bandwidth.

8. The system according to claim 1, wherein: Using the memory interface and configuring the memory device are performed while the system remains in a powered-on state.

9. A method comprising: receiving a request to configure a memory; determining resources available for the request to configure the memory; configuring the memory based on the request; as well as Modifies the activation state of one or more memory interfaces.

10. The method according to claim 9, wherein: Receiving a request to configure memory includes receiving a request for an amount of memory to be modified; and configuring the memory includes modifying an activation state of the one or more memory interfaces based on the amount of memory to be modified.

11. The method according to claim 9, wherein: Determining that resources are available includes determining that the one or more memory interfaces are available to connect one or more memory devices configured to operate at least partially as system memory.

12. The method according to claim 9, wherein: Determining the resources available include: determining a bandwidth corresponding to the request; and determining that the one or more memory interfaces correspond to the bandwidth; and Configuring the memory includes modifying an activation state of the one or more memory interfaces based on the bandwidth.

13. The method according to claim 9, wherein: Configuring the memory includes enabling one or more channels of a memory interface based on the request.

14. The method according to claim 9, wherein: Configuring the memory is performed while a system configured with the memory remains in a powered-on state.

15. A device comprising: Memory interface; as well as At least one circuit is configured to perform one or more operations including: receiving a request to configure memory for one or more memory devices configured at least in part as system memory; connecting a memory device of the one or more memory devices to a host device using a memory interface; and The memory device is configured to operate as system memory.

16. The device according to claim 15, wherein: Using the memory interface includes enabling the memory interface, wherein enabling the memory interface allows the memory device to connect to the host device.

17. The apparatus according to claim 15, wherein: Using the memory interface includes modifying an available state of the memory device of the one or more memory devices.

18. The apparatus according to claim 15, wherein: Receiving a request to configure memory includes receiving a request to increase an amount of memory bandwidth; And the at least one circuit is further configured to perform one or more operations including: enabling one or more memory interfaces corresponding to the request to increase the amount of memory bandwidth.

19. The device according to claim 15, in, The memory interface includes one or more channels; and The at least one circuit is further configured to perform one or more operations including configuring the one or more channels based on the request.

20. The apparatus of claim 15, wherein: Using the memory interface and configuring the memory device are performed while the device remains in a powered-on state.

Images