Memory management method and server
By allocating directly accessed DAX files to each compute node on the CXL memory extender, the near-end access extended memory is solved, and the problem of insufficient near-end extended memory in multi-CPU computing devices is improved, and the memory access speed and CPU performance are improved.
Patent Information
- Application Number
- CN202411158029.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-12-20
AI Technical Summary
In multi-CPU computing devices, when the nearest extended memory is insufficient, it is necessary to access the remote extended memory across the CPU, resulting in slow access speed and affecting CPU performance.
By allocating directly accessed DAX files to each compute node on the CXL memory extender, it realizes near-end access to extended memory, reduces data transmission complexity, and synchronizes the marking state when allocating and freeing storage space, avoiding duplicate allocation and conflict.
It improves memory access speed, ensures CPU performance, reduces memory conflicts and resource waste, and improves memory resource utilization and system consistency.
Smart Images

Figure CN119166333A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of server technology, and in particular to a memory management method and a server. Background Art
[0002] In application scenarios such as big data retrieval and artificial intelligence, computing devices such as servers and data centers require larger physical storage space to store the explosively growing data.
[0003] However, due to the limitation of the internal space of computing devices, it is difficult for computing devices to carry enough memory to meet the growing data processing needs. In some technical solutions, CXL memory expanders are used as external memory devices for computing devices to expand the memory capacity of computing devices without occupying the memory slots of computing devices. For devices connected to CXL memory expanders, the extended memory provided by CXL memory expanders is divided into two parts: proximal extended memory and distal extended memory. The sum of the capacities of proximal extended memory and distal extended memory is the total capacity of extended memory.
[0004] However, when the proximal extended memory of a CPU in a multi-CPU computing device is insufficient, it is necessary to access other remote extended memories across CPUs. The access speed of the remote extended memory is slow, and the CPU performance is easily affected. Summary of the invention
[0005] The embodiments of the present application provide a memory management method and a server to improve memory access speed and ensure CPU performance.
[0006] In a first aspect, an embodiment of the present application provides a memory management method, which is applied to a computing device, wherein a plurality of computing nodes in the computing device are respectively connected to a computing fast link CXL memory expander through a plurality of interfaces on the CXL memory expander, the method comprising:
[0007] Receive a storage space allocation request sent by a first computing node; the storage space allocation request includes a computing node number and an application space size;
[0008] Determine a first direct access DAX file corresponding to the first computing node according to the computing node number; the storage space of the first DAX file is a mapping of the extended memory accessible through the first interface in the computing device; the first interface is used to access all extended memories of the CXL memory expander;
[0009] Allocate target storage space for the first computing node based on the first DAX file and the requested space size;
[0010] The status of the space corresponding to the target storage space in the second DAX file is marked as invalid; the storage space of the second DAX file is a mapping of the extended memory accessible through the second interface in the computing device; the second interface is used to access all extended memories of the CXL memory expander; the second DAX file corresponds to a second computing node.
[0011] In the embodiment of the present application, since the first DAX file corresponds to the first computing node and the first DAX file can access all the extended memory of the CXL memory expander, the target storage space allocated to the first computing node based on the first DAX file is equivalent to the local space of the first computing node, and the first computing node's access to the target storage space can be regarded as proximal access, thereby reducing the complexity of data transmission, improving the memory access speed, and ensuring the performance of the CPU. In addition, after the storage space allocation is completed, the status of the space corresponding to the storage space in each DAX file is marked as invalid, avoiding repeated allocation of the same storage space, reducing memory conflicts, and ensuring that the first computing node can use the target storage space normally.
[0012] In a possible implementation, allocating a target storage space for the first computing node based on the first DAX file and the requested space size includes:
[0013] The target storage space is allocated to the available storage space using the state in the first DAX file.
[0014] In an embodiment of the present application, before allocating storage space, it is possible to first traverse and determine the unoccupied or unallocated storage space in the first DAX file, and then use the available storage space to allocate target storage space to the first computing node, thereby avoiding repeated occupation of storage space and improving memory stability.
[0015] In a possible implementation, allocating a target storage space for the first computing node based on the first DAX file and the requested space size includes:
[0016] Obtain a file descriptor of a first DAX file, and determine a first offset address and a space length in the first DAX file; the first offset address is a starting address of the target storage space in the first DAX file, and a space size corresponding to the space length is equal to the applied space size;
[0017] The file descriptor, the first offset address and the space length are sent to the first computing node.
[0018] In the embodiment of the present application, by directly operating the DAX file, the complex mapping and conversion process in traditional memory management is avoided. The computing node can quickly access the extended memory through the offset address and space length. Through precise offset address and space length control, storage conflicts can be avoided, and the memory area in the DAX file can be effectively managed, which helps to reduce the repeated allocation of the same storage space and avoid data overwriting and consistency issues.
[0019] In a possible implementation, the method further includes:
[0020] The status of the target storage space is marked as invalid in the first DAX file.
[0021] In an embodiment of the present application, after allocating the target storage space, marking the target storage space as invalid can effectively prevent the same storage space from being allocated multiple times to different computing nodes, avoid resource reuse or allocation conflicts, and ensure the uniqueness and consistency of memory allocation.
[0022] In a possible implementation, marking the status of the space corresponding to the target storage space in the second DAX file as invalid includes:
[0023] Determine a second offset address according to the first offset address; the second offset address is the starting address of the space corresponding to the target storage space in the second DAX file;
[0024] According to the second offset address and the space length, a status of the space corresponding to the target storage space is marked as invalid in the second DAX file.
[0025] In the embodiment of the present application, by marking the state of the space corresponding to the target storage space as invalid in the second DAX file, it is ensured that the space will not be misused or repeatedly allocated, and memory conflicts and resource contention are avoided. By synchronously marking the invalid state in different DAX files, the consistency of memory allocation can be maintained. Even if the space is shared between different computing nodes, the system can ensure that there will be no multiple allocations or data inconsistencies.
[0026] In a possible implementation, the method further includes:
[0027] Receiving a storage space release request sent by a first computing node; the storage space release request includes the computing node number, the first offset address and the space length;
[0028] releasing the data in the target storage space in the first DAX file according to the first offset address and the space length; and marking the state of the target storage space as valid in the first DAX file;
[0029] According to the storage space release request, the status of the space corresponding to the target storage space is marked as valid in the second DAX file.
[0030] In an embodiment of the present application, by marking the released storage space as valid in the DAX file, it is ensured that the released space can be reused, the utilization rate of memory resources is improved, and resource waste is reduced. In the process of releasing and restoring storage space, it is ensured that the status information in each DAX file is updated synchronously, data consistency is maintained, and memory allocation problems caused by inconsistent states are prevented. By explicitly marking the memory state as valid, potential memory conflict problems caused by inconsistent states after the release operation are avoided, ensuring that the space can be reliably allocated to other tasks or computing nodes after release, allowing dynamic release and reallocation of storage space, so that the system can more flexibly respond to the memory requirements of different computing nodes.
[0031] In a possible implementation, marking the status of the space corresponding to the target storage space as valid in the second DAX file according to the storage space release request includes:
[0032] Determine the second offset address according to the first offset address; the second offset address is the starting address of the space corresponding to the target storage space in the second DAX file;
[0033] According to the second offset address and the space length, the status of the space corresponding to the target storage space is marked as valid in the second DAX file.
[0034] In the embodiment of the present application, through the above steps, the storage space release request can be effectively processed, and the status of the target storage space can be correctly marked in the second DAX file, thereby optimizing memory resource management.
[0035] In a possible implementation, before receiving the storage space allocation request sent by the first computing node, the method further includes:
[0036] Obtaining extended memory information of the CXL memory expander;
[0037] The memory capacity accessed by each interface of the CXL memory expander is configured as the total capacity of the extended memory of the CXL memory expander according to the extended memory information.
[0038] In an embodiment of the present application, by configuring the memory capacity of each interface, each interface can effectively access all extended memory resources, thereby improving the memory utilization efficiency of the system, so that the computing node can call all extended memory in a proximal access manner through the corresponding interface.
[0039] In a possible implementation, configuring the memory capacity accessed by each interface of the CXL memory expander as the total capacity of the extended memory of the CXL memory expander according to the extended memory information includes:
[0040] Configure the interface number, physical starting address and address length of each interface, and configure the number, decoder starting address and decoding length of each host management device memory HDM decoding register group; each interface and each HDM decoding register group correspond one to one; the address length of each interface and the decoding length of each HDM decoding register correspond to the total capacity of the extended memory;
[0041] Creating a plurality of DAX files according to the interface number, the physical start address and the address length of each interface; wherein the plurality of DAX files correspond to the plurality of interfaces one by one;
[0042] Creating multiple virtual HDM decoding register groups according to the number of each HDM decoding register group, the decoder start address and the decoding length; each HDM decoding register group corresponds to the multiple virtual HDM decoding register groups one by one;
[0043] The respective HDM decoding register groups are mapped to the plurality of virtual HDM decoding registers.
[0044] In the embodiment of the present application, each interface and HDM decoding register group of the CXL memory expander can be accurately configured, so as to efficiently manage and access the extended memory resources.
[0045] In a possible implementation manner, the first computing node is a first processor, and the second computing node is a second processor.
[0046] In the embodiment of the present application, the first computing node and the second computing node are two processors in the same computing device. The first processor and the second processor can both access all the extended memory provided by the CXL memory expansion device through the corresponding DAX file, and can both access all the extended memory through proximal access, thereby reducing the latency of the processor accessing the extended memory.
[0047] In a second aspect, an embodiment of the present application provides a memory management device, which is applied to a computing device, wherein a plurality of computing nodes in the computing device are respectively connected to a computing fast link CXL memory expander through a plurality of interfaces on the CXL memory expander, and the device comprises:
[0048] A receiving module, configured to receive a storage space allocation request sent by a first computing node; the storage space allocation request includes a computing node number and an application space size;
[0049] a determination module, configured to determine a first direct access DAX file corresponding to the first computing node according to the computing node number; the storage space of the first DAX file is a mapping of the extended memory accessible through the first interface in the computing device; the first interface is used to access all extended memories of the CXL memory expander;
[0050] an allocation module, configured to allocate a target storage space to the first computing node based on the first DAX file and the requested space size;
[0051] A marking module is used to mark the status of the space corresponding to the target storage space in the second DAX file as invalid; the storage space of the second DAX file is a mapping of the extended memory accessible through the second interface in the computing device; the second interface is used to access all the extended memories of the CXL memory expander; the second DAX file corresponds to a second computing node.
[0052] In a possible implementation, the allocation module is specifically configured to:
[0053] Obtain a file descriptor of a first DAX file, and determine a first offset address and a space length in the first DAX file; the first offset address is a starting address of the target storage space in the first DAX file, and a space size corresponding to the space length is greater than or equal to the applied space size;
[0054] The file descriptor, the first offset address and the space length are sent to the first computing node.
[0055] In a possible implementation, the marking module is further used to:
[0056] The status of the target storage space is marked as invalid in the first DAX file.
[0057] In a possible implementation, the marking module is specifically used to:
[0058] Determine a second offset address according to the first offset address; the second offset address is the starting address of the space corresponding to the target storage space in the second DAX file;
[0059] According to the second offset address and the space length, a status of the space corresponding to the target storage space is marked as invalid in the second DAX file.
[0060] In a possible implementation manner, the device further includes:
[0061] A release module, configured to receive a storage space release request sent by a first computing node; the storage space release request includes the computing node number, the first offset address and the space length; and release the data in the target storage space in the first DAX file according to the first offset address and the space length;
[0062] The marking module is also used for:
[0063] In the first DAX file, the status of the target storage space is marked as valid; and in the second DAX file, according to the storage space release request, the status of the space corresponding to the target storage space is marked as valid.
[0064] In a possible implementation, the marking module is further used to:
[0065] Determine the second offset address according to the first offset address; the second offset address is the starting address of the space corresponding to the target storage space in the second DAX file;
[0066] According to the second offset address and the space length, the status of the space corresponding to the target storage space is marked as valid in the second DAX file.
[0067] In a possible implementation manner, the device further includes:
[0068] The configuration module is used to obtain the extended memory information of the CXL memory expander; and configure the memory capacity accessed by each interface of the CXL memory expander as the total extended memory capacity of the CXL memory expander according to the extended memory information.
[0069] In a possible implementation, the configuration module is specifically used to:
[0070] Configure the interface number, physical starting address and address length of each interface, and configure the number, decoder starting address and decoding length of each host management device memory HDM decoding register group; each interface and each HDM decoding register group correspond one to one; the address length of each interface and the decoding length of each HDM decoding register correspond to the total capacity of the extended memory;
[0071] Creating a plurality of DAX files according to the interface number, the physical start address and the address length of each interface; wherein the plurality of DAX files correspond to the plurality of interfaces one by one;
[0072] Creating multiple virtual HDM decoding register groups according to the number of each HDM decoding register group, the decoder start address and the decoding length; each HDM decoding register group corresponds to the multiple virtual HDM decoding register groups one by one;
[0073] The respective HDM decoding register groups are mapped to the plurality of virtual HDM decoding registers.
[0074] In a third aspect, an embodiment of the present application provides a server, comprising a processor, and a memory communicatively connected to the processor;
[0075] The memory is used to store computer-executable instructions;
[0076] The processor is used to execute the computer-executable instructions stored in the memory to implement the memory management method described in any embodiment of the first aspect above.
[0077] In a fourth aspect, an embodiment of the present application provides a computer storage medium, in which a code is stored. When the code is executed, a device executing the code implements the memory management method described in any one of the first aspects above.
[0078] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes code. When the code is executed, a device executing the code implements the memory management method described in any one of the first aspects above. BRIEF DESCRIPTION OF THE DRAWINGS
[0079] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0080] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application;
[0081] Figure 2 A schematic diagram of another application scenario provided for an embodiment of the present application;
[0082] Figure 3 A schematic diagram of the architecture of a server provided in an embodiment of the present application;
[0083] Figure 4 A schematic diagram of another server architecture provided in an embodiment of the present application;
[0084] Figure 5A schematic diagram of the architecture of a CXL memory expander provided in an embodiment of the present application;
[0085] Figure 6 A schematic diagram of a software architecture provided for an embodiment of the present application;
[0086] Figure 7 A flowchart of a memory management method provided in an embodiment of the present application;
[0087] Figure 8 A schematic diagram of the architecture of a server and a CXL memory expander provided in an embodiment of the present application;
[0088] Fig. 9 A schematic diagram of a process for allocating and releasing storage space provided in an embodiment of the present application. DETAILED DESCRIPTION
[0089] It should be noted that the embodiments described in this application are only part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0090] In order to make the following embodiments clear, the technical terms involved in this application are first introduced.
[0091] The Computer Express Link (CXL) protocol is an open interconnection protocol launched by Intel that enables high-speed and efficient interconnection between the central processing unit (CPU) and the graphics processing unit (GPU), field programmable gate array (FPGA) or other accelerators, thereby meeting the requirements of high-performance heterogeneous computing. The CXL protocol supports the CPU of a computing device to access other extended memory devices connected to the computing device, while other extended memory devices do not need to occupy the memory slots of the computing device, saving internal space of the computing device.
[0092] CXL memory expander (CXL Memory Expander), CXL memory expander integrates resources of multiple memory components based on pooling technology. In addition, CXL memory expander can provide multiple interfaces to connect one by one with multiple CPUs in computing devices, providing a centrally managed memory pool for computing devices. CXL memory expander supports basic address mapping, access control and other functions. In an embodiment of the present application, CXL memory expander can be Type2 or Type3. CXL memory expander can also be a multi-head logic device (Multi Head Single Logic Device, MHSLD), that is, CXL memory expander includes multiple interfaces.
[0093] The embodiment of the present application provides a memory management method and server. The CXL memory expander is used to provide extended memory for computing nodes, and the memory capacity of the proximal extended memory corresponding to each computing node is configured as the total capacity of the CXL memory expander, that is, each computing node can access the full capacity of the extended memory through proximal access. The part that originally belonged to the remote extended memory is adjusted to the proximal extended memory, so that the computing node proximal accesses the memory of the CXL memory expander, thereby improving the access speed of the extended memory.
[0094] The application scenarios of the embodiments of the present application are introduced below with reference to the accompanying drawings.
[0095] In a possible application scenario, Figure 1 As shown, the computing node can be an independent computing device (such as a server), and multiple computing nodes (a first server and a second server) are respectively connected to different interfaces (interface 1 and interface 2) of the CXL memory expander. Computing node 1 (such as the first server) is connected to the CXL memory expander through interface 1 (also called the first interface), and computing node 2 (such as the second server) is connected to the CXL memory expander through interface 2 (also called the second interface), and computing node 1 and computing node 2 are communicatively connected.
[0096] In another possible application scenario, such as Figure 2 As shown, the computing node can also be a processor in a computing device (such as a server), and multiple computing nodes (a first processor and a second processor) in the same computing device are respectively connected to different interfaces (interface 1 and interface 2) of the CXL memory expander. Computing node 1 (such as the first processor) is connected to the CXL memory expander through interface 1 (also called the first interface), and computing node 2 (such as the second processor) is connected to the CXL memory expander through interface 2 (also called the second interface), and computing node 1 and computing node 2 can be connected in communication, or they can be connected through the internal lines of the computing device.
[0097] exist Figure 1 and Figure 2In the application scenario shown, the extended memory accessed by computing node 1 through interface 1 is the proximal extended memory of computing node 1, and the extended memory accessed by computing node 1 through interface 2 across computing node 2 is the distal extended memory of computing node 1. Similarly, the extended memory accessed by computing node 2 through interface 2 is the proximal extended memory of computing node 2, and the extended memory accessed by computing node 2 through interface 1 across computing node 1 is the distal extended memory of computing node 2.
[0098] Among them, the memory capacity of the proximal extended memory of computing node 1 (also the distal extended memory of computing node 2) and the distal extended memory of computing node 1 (also the proximal extended memory of computing node 2) are all the extended memory capacities provided by the CXL memory expander. Computing node 1 can access the full capacity of the extended memory through interface 1, and of course can also access the full capacity of the extended memory through interface 2. The latency of computing node 1 accessing the memory proximal through interface 1 is shorter than the latency of computing node 1 and computing node 2 indirectly accessing the memory remotely through interface 2, and the latency of processor 2 accessing the proximal extended memory through interface 2 is shorter than the latency of processor 2 accessing the remote extended memory through interface 1. The memory that can be accessed proximal by a computing node can be called proximal extended memory, and the memory that can be accessed remotely by a computing node can be called remote extended memory.
[0099] Through the memory management method provided by the embodiment of the present application, each computing node can access all the extended memory spaces provided by the CXL memory expander through proximal access, thereby improving the speed at which the computing node accesses the extended memory. In addition, the capacity of the proximal extended memory and the distal extended memory of the same computing node are the same as the capacity of the extended memory, and the memory space of the proximal extended memory and the distal extended memory of the same computing node are the same as the memory space of the extended memory; that is, the proximal extended memory and the distal extended memory of the same processor both map all the spaces provided by the same CXL memory expander. Therefore, the same computing node can access the same piece of extended memory through proximal access or distal access, and its proximal access path and distal access path can back up each other. When the computing node cannot access the extended memory proximal, it can also access the extended memory remotely to improve the reliability of memory access.
[0100] In addition, by setting the memory capacity accessible by each interface to the total capacity of the extended memory, the extended memory is used as the shared memory of multiple computing nodes. When a computing node (such as computing node 1) needs to share data with another computing node (such as computing node 2), the computing node (such as computing node 1) directly writes the data into the extended memory, and the other computing node (such as computing node 2) can directly read the required shared data from the extended memory. There is no need for computing node 1 and computing node 2 to perform other interactive operations on the data to be shared, which simplifies the data exchange process between nodes and improves efficiency.
[0101] As an example, a Universal Backplane Connector (UBC) port of a computing node and a UBC interface of a CXL memory expander can be connected via a cable. The UBC port is an interface for connecting a processor to a CXL device to achieve high-speed, low-latency data transmission.
[0102] Below Figure 2 The scenario shown is an exemplary introduction. Specifically, the computing device may be a server, computing node 1 may be CPU1, and computing node 2 may be CPU2.
[0103] In a server including multiple CPUs, Figure 3 shown.
[0104] CPU1 and CPU2 can communicate through high-speed interconnection (such as Intel QuickPathInterconnect (QPI) or high-speed serial computer expansion bus standard Peripheral ComponentInterconnect Express (PCIE)). Each CPU is connected one-to-one with the interface of the CXL memory expander. The CPU can proximally access the memory space provided by the CXL memory expander based on the CXL protocol. This access is called CPU access to proximal extended memory. The CPU can also communicate with another CPU and remotely access the memory space provided by the CXL memory expander based on the other CPU and the interface connected to the other CPU. This access is called CPU access to remote extended memory.
[0105] Taking CPU1 as an example, CPU1 is directly connected to interface 1, and CPU1 can achieve near-end access to near-end extended memory through interface 1. CPU1 and CPU2 are interconnected at a high speed, and CPU2 is directly connected to interface 2. CPU1 can achieve remote access to remote extended memory through CPU2 and interface 2.
[0106] As an example, the interface of the CXL memory expander is connected to the CPU via CXL interconnect technology. In the server, the interface of the CXL memory expander (or UBC cable, or gold finger slot) is connected to the interface (including but not limited to UBC interface) or open expansion slot on the server motherboard through a bus (including but not limited to UBC cable). The CXL controller transfers data from the CXL memory expander to the corresponding CPU to realize the memory expansion function. This connection method can achieve high-speed, low-latency memory access, allowing the CPU to access data in the extended memory faster and improve the overall system performance.
[0107] In general, the connection between the CXL memory expander and the CPU is achieved through CXL interconnect technology, using the high-speed CXL bus to transmit data to ensure that the CPU can efficiently access the extended memory resources. This connection method can meet the needs of high-performance, low-latency memory access and is suitable for scenarios that require large memory capacity and high-performance computing.
[0108] CPU1 and CPU2 can process different tasks at the same time without interfering with each other. For CXL memory expanders, CPU1 and CPU2 can access extended memory through different interfaces to improve parallel processing capabilities. CXL memory expanders are connected to multiple CPUs in the server through multiple interfaces, so that the CPU corresponding to each interface can directly access the extended memory. In such a configuration, each interface is regarded as an independent channel for connecting to a specific CPU.
[0109] In order to further improve the memory access efficiency, the embodiment of the present application Figure 3 Based on the architecture shown in FIG. 1 , a server architecture diagram is further provided, such as Figure 4 shown.
[0110] Direct Access (DAX) files are a special type of file that allows applications to access file data directly in memory without going through the kernel cache. This can improve the speed and efficiency of data access and is particularly suitable for scenarios that require frequent reading and writing of large files, such as high-performance computing, big data processing, and other applications.
[0111] In an embodiment of the present application, the DAX file enables the CPU corresponding to each interface to access the extended memory in a low-latency, high-efficiency manner, just like accessing local memory. In addition, the DAX file corresponding to each interface can cover the capacity of the entire extended memory of the CXL memory expander, so that each CPU can access all extended memory resources. The CPU can directly access the extended memory provided by the CXL memory expander through the DAX files corresponding to each interface, bypassing the file system cache, thereby increasing the access speed to the extended memory. Reduce the complexity of data transmission and increase memory access speed.
[0112] like Figure 4 As shown, CPU1 implements proximal access to the extended memory through DAX1, and CPU2 implements proximal access to the extended memory through DAX2. When DAX1 or interface 1 is abnormal, CPU1 can also implement remote access to the extended memory through CPU2 and DAX2; when DAX2 or interface 2 is abnormal, CPU2 can also implement remote access to the extended memory through CPU1 and DAX1.
[0113] In the related art, only part of the extended memory of the CXL memory expander can be accessed through each interface, that is, the computing node can proximally access a part of the extended memory through the directly connected interface, and then remotely access another part of the extended memory through the indirectly connected interface (the indirectly connected interface refers to the interface connected through another computing node). Through the memory management method provided in the embodiment of the present application, the memory corresponding to each interface of the CXL memory expander is adjusted, so that the computing node can access the extended memory provided by the CXL memory expander through the corresponding interface, that is, the computing node can proximally access the complete extended memory through the directly connected interface, so that part of the extended memory that originally needs to be accessed remotely can also be accessed proximally, thereby improving the speed at which the computing node accesses the extended memory and eliminating the latency difference of the computing node accessing different parts of the extended memory.
[0114] Moreover, in the related art, if a certain interface fails or the connection between the computing node and the interface fails, the computing node cannot access the part of the extended memory corresponding to the interface. In the embodiment of the present application, each interface corresponds to a complete extended memory, and the access path to the extended memory corresponding to each interface can back up each other. Even if the computing node cannot access the extended memory through a certain interface, it can still access the complete extended memory through other interfaces, which solves the problem of limited access to the extended memory in the related art and improves the reliability of the extended memory access.
[0115] The present application embodiment also provides a structural diagram of a MHSLD type CXL memory expansion device, such as Figure 5 As shown, Figure 5 What is shown is a multi-head CXL memory expander. The CXL memory expander includes multiple (two or more) interfaces, a CXL controller and a memory. Among them, multiple connectors are connected to the CXL controller through interfaces respectively, and the CXL controller is connected to the memory. The memory is used to provide extended memory, and the interface is used to connect the computing nodes in the computing device. The CXL controller runs CXL firmware and multiple CXL management units. The multiple heads in the CXL expansion device correspond one-to-one to the multiple CXL management units, and the multiple heads in the CXL expansion device correspond one-to-one to the multiple interfaces. The CXL management unit is responsible for processing the communication and data exchange between the interface and the computing node.
[0116] Based on the hardware architecture provided in the above embodiment of the present application, the memory management method provided in the embodiment of the present application is introduced from the software architecture. Figure 6 As shown, Figure 6 A software architecture diagram provided for an embodiment of the present application.
[0117] The basic input and output system (BIOS) and the operating system (OS) are installed in the computing device to which the independent computing node or multiple computing nodes belong. BIOS is the first software to run when the computing node starts, and is responsible for hardware initialization, resource configuration, self-test, and loading OS. The operating system takes over the control of the system after startup, and is responsible for managing the resources, scheduling tasks, and providing services of the system. The OS is integrated with the CXL driver (CXL Driver) and the DAX driver. In an embodiment of the present application, during the operation of the BIOS, the configuration work of each interface of the CXL memory expander is implemented, and a corresponding configuration table is generated. During the OS loading process, the corresponding relationship between each computing node and each interface of the CXL memory expander can be established according to the configuration table.
[0118] The CXL controller of the CXL memory expander runs CXL firmware (CXL FW), which can work together with the BIOS to configure various interfaces. For example, in response to the mailbox command issued during the BIOS operation phase, the size of the extended memory provided by the CXL memory expander is fed back to the BIOS.
[0119] The CXL controller of the MHSLD type CXL memory expander includes multiple CXL management units, which correspond to interfaces one by one, and the memory managed by each CXL management unit corresponds to the entire CXL extended memory. The computing device uniformly addresses the CXL extended memory and the local system memory, allocates a physical address to the memory managed by each CXL management unit, and each CXL management unit configures the physical address allocated by the computing device into the corresponding decoder. This allows the computing node to access the extended memory provided by the CXL memory expander as if it were accessing local memory. In addition, in a computing device with multiple computing nodes, the computing node and the CXL management unit maintain cache consistency together to ensure that when the computing node modifies the data in the extended memory, other computing nodes can correctly access the updated extended memory. In addition, if an interface or memory module fails, the computing node can also implement fault isolation to prevent the entire system from being affected.
[0120] Below, we take the computing device as a server and the computing node as the CPU in the server as an example. Figure 4 The hardware architecture shown and Figure 6 The software architecture shown first introduces the process of powering on the server and CXL memory expander at the same time. The specific steps are as follows Figure 7 shown.
[0121] S701: The server runs BIOS, identifies the CXL device, configures bus resources for the CXL device, triggers link training of the CXL device, and sets the CXL device to enter CXL link mode.
[0122] After the server starts, it runs the BIOS and first performs a self-test (POST) to check the working status of the hardware, such as the CPU, memory, hard disk, and graphics card, to ensure that the hardware is working properly. After the POST is completed, the BIOS initializes the hardware, such as cache configuration, memory timing and voltage settings, and CPU frequency and voltage settings. In addition, the BIOS allocates bus resources, such as interrupt requests (IRQs), I / O ports, and memory address spaces for buses such as PCIe, USB, and SATA, and triggers the CXL memory expander to perform CXL link training.
[0123] During the BIOS initialization of CXL devices, if the system uses Intel's UltraPath Interconnect (UPI) architecture and the CXL devices are managed through the Integrated I / O (IIO) stack. The UPI root complex (RC) first identifies the IIO devices in the system and detects which IIO devices support CXL mode. For each IIO device, its PCIe expansion capability and PCI configuration space are detected to determine whether the CXL protocol is supported and which IIO stacks need to be enabled. The UPI RC configures the UPI link for each enabled IIO stack to ensure that the link parameters (such as frequency, width, etc.) meet the device requirements. The IIO RC performs link training to ensure the stability and performance of the physical link and prepare for communication in CXL mode.
[0124] Link training is a key step to ensure the stability and reliability of the communication link, especially for high-speed buses such as CXL. The process of link training is, for example: First, electrical training is performed to ensure that the signal quality and voltage level meet the standards and adjust the parameters of the signal transmitter and receiver. Then, timing training is performed to adjust the timing of the signal to ensure that the data arrives at the right place at the right time to prevent data loss or errors. Then, speed training is performed to gradually increase the link speed according to the capabilities of the hardware until the maximum speed is reached while ensuring error-free transmission. Finally, channel equalization is performed to ensure consistent signal strength across all channels and reduce signal distortion by adjusting the signal strength on different channels.
[0125] S702: The server runs BIOS to scan the interface of the CXL memory expander.
[0126] The server scans the interfaces of the CXL memory expander to discover the CXL management unit corresponding to each interface. The RC assigns a unique bus number to each IIO stack to distinguish different IIO devices. The RC sets the device ID and function ID for each IIO device, which are unique in the bus resources. The RC assigns interrupt vectors and configures interrupt routing to ensure that the IIO device can respond to interrupt requests.
[0127] S703: The server runs BIOS scanning and finds the CXL management unit, maps and initializes the configuration space register and the memory mapping space register.
[0128] The CXL management unit is a logical part of the CXL memory expander. The CXL management unit is responsible for handling the communication and data exchange with the server. The CXL memory expander may contain multiple CXL management units, each of which can be independently configured and perform different tasks. CXL management units correspond to interfaces one by one.
[0129] The server obtains the interface's Designated Vendor-Specific Extended Capability (DVSEC) configuration by identifying and enumerating the interface to which the server is connected and the CXL management unit corresponding to the interface. The server can map and access the configuration space register corresponding to the CXL management unit, which records the interface's related information, such as interface ID, interrupt settings, and power management options. By mapping and accessing the initialization configuration space register, the server can read and set various parameters of the device to ensure that the device can operate as expected.
[0130] The memory mapping registers correspond one-to-one with the interfaces and one-to-one with the CXL management units. The memory mapping registers are used to store the status information, control bits, and configuration parameters (including but not limited to the physical start address, address length, decoding start address, and decoding length) required for mapping memory, and support the CPU to configure and control the devices connected to the CXL bus through memory access instructions. The server allocates addresses for the memory mapping registers by configuring MEMBAR0 in the CXL downstream interface (CXL Downstream Port) and the CXL upstream interface (CXLUpstream Port).
[0131] S704: The server runs BIOS to send a command to the CXL management unit to obtain extended memory information.
[0132] The command is used to obtain information about the extended memory of the CXL memory expander (including but not limited to the total capacity, memory frequency, etc.). As an example, the command may be a mailbox command.
[0133] The mailbox command is a communication mechanism in the CXL protocol, used to implement information exchange between a host (such as a server) and a CXL device (such as a CXL memory expander).
[0134] As an example, during the BIOS operation, the server may send a CXL_MBOX_OP_IDENTIFY command to each CXL management unit, requesting each CXL management unit to provide information such as its memory capacity, characteristics, etc. After receiving this command, each CXL management unit prepares corresponding response data, which generally includes the total memory capacity, volatile memory capacity, etc. of the CXL memory expander, and finally returns the response data to the computing device.
[0135] The mailbox command mechanism simplifies the communication between the server and the CXL management unit, reducing the need for complex protocol stacks, while also allowing the CXL management unit to respond to server requests with low latency and high efficiency.
[0136] S705: The server runs BIOS to obtain response data fed back by the CXL management unit.
[0137] The CXL management unit obtains the capacity information of the extended memory provided by the CXL memory expander, such as the total memory capacity, volatile memory capacity, persistent memory capacity, partition alignment byte number, logical sector size (Logical Sector Addressing size, LSA size) and firmware version through the CXL FW. Each CXL management unit returns the capacity information to the BIOS as response data.
[0138] S706: The server runs the BIOS configuration interface and the HDM decoding register.
[0139] As an example, the configuration information of the server interface and HDM decoding register is shown in Table 1.
[0140] Table 1: Configuration information table
[0141]
[0142]
[0143] The server assigns a bus number, an interface number, and a memory capacity (physical starting address and address length) to each interface. The bus number corresponds to the interface one-to-one, the interface number also corresponds to the interface one-to-one, and the physical starting address also corresponds to the interface one-to-one. When the server needs to access the extended memory, the bus number and the interface number can be used to determine the only interface and then access the extended memory. In addition, the server sets the memory capacity corresponding to each interface to the total capacity of the extended memory provided by the CXL memory expander. As shown in Table 1, although the physical starting addresses of interface 1 and interface 2 are different, the address lengths corresponding to interface 1 and interface 2 are the same, which is the total capacity of the extended memory.
[0144] Host-managed Device memory (HDM) decoder, the number of HDM decoding register groups is the same as the number of interfaces. Each CXL management unit has an independent set of HDM decoding register groups (such as HDM decoder0 ~ HDM decoder n), which is used to control multiple decoding registers to decode different address segments respectively. The total decoding range of multiple HDM decoding registers in an HDM decoding register group is 0 ~ total capacity of extended memory (Total Capacity). The server configures different decoding start addresses and decoding lengths for each HDM decoding register. For different CXL management units, the decoding start addresses of HDM decoding register decoder n are different, and the decoding lengths of each HDM decoding register are the same. HDM decoding registers are usually integrated in the CXL memory expander to respond to the server's access instructions to the extended memory. The server can also configure other decoding parameters for the HDM decoding register, such as interleaving granularity, decoding granularity, decoding mode and data transmission mode, etc., and reasonable configuration can improve data access efficiency.
[0145] It should be noted that there may be two different HDM decoding registers with the same number in two independent HDM decoding register groups. For example, HDM decoding register group 1 includes HDM decoding register 1, and HDM decoding register group 2 also includes HDM decoding register 1, but HDM decoding register 1 in HDM decoding register group 1 and HDM decoding register 1 in HDM decoding register group 2 are two different HDM decoding registers.
[0146] After the server configuration is completed, the configuration information related to the interface and the HDM decoding register is written into the memory mapping register configured in step S703 to implement the configuration of the interface, the CXL management unit and the HDM decoding register.
[0147] S707: The server generates a CEDT table and transmits it to the OS startup process.
[0148] The CXL Early Discovery Table (CEDT) is used to store the relevant information configured by the server for the interface and HDM decoding register in the above steps, such as the physical start address, address length and interface number of each interface, the HDM number, decoding start address and decoding length of the HDM decoding register, the decoding parameters of the HDM decoding register and the physical address of the memory mapping register, etc. The BIOS sends the access address of the CEDT table to the OS, and when the OS starts, it understands the relevant configuration of the CXL memory expander for correct initialization and management.
[0149] S708: The server creates a virtualized device according to the CEDT table.
[0150] To achieve correct access to the CLX memory expander, the server also needs to create virtualized devices corresponding to each device in the CLX memory expander in the OS, and synchronize the configuration and mapping of the virtualized devices with the devices in the CXL memory expander. The virtualized devices include virtual HDM decoding registers and virtual memory mapping registers, such as Figure 8 shown.
[0151] According to the relevant information of the HDM decoding register group recorded in the CEDT table, the server adds and configures the virtual HDM decoding register group (struct cxl_decoder) in the OS. The virtual HDM decoding register group is the virtual decoding register group of the HDM decoding register group in the CXL memory expander in the OS. The virtual HDM decoding register group corresponds one-to-one with the HDM decoding register group. In addition, virtual memory mapping registers are added and configured in the OS. The virtual memory mapping registers are the virtual registers of the memory mapping registers in the CXL memory expander in the OS. The decoding ranges of the HDM decoding register group and the virtual HDM decoding register group are the same (i.e., decoding start address and decoding length), and the configuration information in the virtual memory mapping register and the memory mapping register (physical start address, interface number, bus number, HDM number, decoding start address, and decoding length) are also the same.
[0152] The server maps the virtual HDM decoding register set in the OS to the HDM decoding register set in the CXL memory expander, and maps the virtual memory mapping register to the memory mapping register in the CXL memory expander. It should be noted that the virtual HDM decoding register set and virtual memory mapping register in the OS correspond one-to-one to the HDM decoding register set and memory mapping register in the CXL memory expander.
[0153] In the above step S705, the BIOS implements the configuration of the relevant registers or decoders in the CXL memory expander, and further implements the synchronization configuration in the OS based on the relevant information in the CEDT table through step S707, and synchronizes the virtual expression in the OS with the software and hardware in the CXL memory expander to achieve consistency. Then, when accessing the extended memory, the total capacity of the extended memory can be directly accessed through the virtual expression in the OS.
[0154] S709: The server creates a DAX file.
[0155] The server calls the DAX driver to create DAX files corresponding to each interface in the OS. Figure 8 As shown, the same number of DAX files as the CXL memory expander interfaces can be seen in the OS, and the capacity mapped by each DAX file is the total extended memory capacity of the CXL memory expander.
[0156] For each identified interface, the DAX driver creates a corresponding DAX file for it and configures each DAX file to directly map to the total memory capacity of the extended memory of the CXL memory expander. That is, each DAX file can directly access the entire extended memory provided by the CXL memory expander.
[0157] Through DAX files, the operating system or application can directly access all extended memory without additional cache processing, greatly reducing access latency and increasing data processing speed.
[0158] In the embodiment of the present application, the server allocates a physical address space (which can be determined by the physical start address and address length of the interface) to each interface in the CXL memory expander, and the entire extended memory of the CXL memory expander can be accessed through the physical address space. That is, for a CXL memory expander including multiple interfaces, the physical address spaces corresponding to the multiple interfaces all correspond to the same complete extended memory space.
[0159] In a possible implementation, the OS of the server may manage multiple physical address spaces corresponding to the CXL memory expander through offset mapping based on a local CXL memory management daemon (LocalCXL Space Management Daemon, LMD).
[0160] After the LMD process is started, first check whether the number of DAX files in the operating system is 0. If it is 0, call the DAX driver to create DAX files with the same number of interfaces. When creating, use the "lspci|grep 1dfa" command to query the number of interfaces of the CXL memory expander. If it is not 0, check whether the DAX file can be shared and accessed. If the DXA file name is dax0.0, dax1.0. Write the corresponding data to the offset 0 position of dax0.0, and check whether the correct data can be read from the dax1.0 device. If the test passes, it proves that the DAX file is created successfully. If the test fails, perform a read and write consistency test on the dax0.0 or dax1.0 device separately. If dax0.0 (or dax1.0) cannot read and write normally, all space is forced to be allocated from dax1.0 (or dax0.0), and error0 is reported to ask the user to replace the physical device. If both dax0.0 and dax1.0 cannot read and write normally, error1 is reported to ask the user to replace the physical device. Set the capacity of the shared access space to the total capacity of the extended memory, and use the "cat / proc / iomem|grep dax0.0" command to obtain the total capacity of the dax0.0 device. Use the buddy&slab solution to manage the shared space in slices. After the DAX file is created, the LMD process enters the sleep mode and waits to be awakened by the user process.
[0161] In the above steps S701-S709, through the interface in the CXL memory expander and the memory mapping register and HDM decoding register corresponding to the interface, the resources and attributes of the CXL memory expander can be flexibly managed, so that each interface can directly map the total memory capacity of the extended memory of the CXL memory expander. In addition, a corresponding DAX file is created for each interface through the DAX driver, and all extended memories are directly mapped in the OS of the server, providing the ability to directly access all extended memories without additional cache processing. All extended memories provided by the CXL memory expander are used as proximal extended memories, and the CPU can access the total capacity of the extended memory in a proximal access manner through a directly connected interface, thereby improving the access speed of the extended memory. Moreover, the memories corresponding to each DAX file can back up each other, thereby improving the security of the data. At the same time, through the proximal access method, the CPU in the server can access all extended memories more quickly and reliably, thereby ensuring the integrity and reliability of the data.
[0162] Based on the above embodiments, the embodiments of the present application further introduce how an application in the user space accesses the extended memory provided by the CXL memory expander through a DAX file.
[0163] In a possible implementation, an application running in user space can access extended memory by directly mapping a DAX device. Take CPU1 (also called the first computing node) executing an application to request access to extended memory as an example. Fig. 9 The following describes the process of accessing extended memory:
[0164] S901: The LMD process receives a storage space allocation request sent by an application process.
[0165] The storage space allocation request is used to request the allocation of storage space from the extended memory provided by the CXL memory expander. The application process refers to the process of the CPU when running an application. When CPU1 is executing an application, it may need to store some intermediate data or application data. The application process sends a storage space allocation request to the LMD process to apply for storage space to store the relevant data of the running application.
[0166] The storage space allocation request includes the requested space size and the CPU number (also called computing node number) for executing the application process. The CPU number for executing the application process can also be understood as the CPU number for running the application program. The CPU number is used to identify a unique CPU.
[0167] S902: The LMD process parses the storage space allocation request.
[0168] The LMD process obtains the requested space size and the CPU number for executing the application process by parsing the storage space allocation request.
[0169] S903: The LMD process determines the DAX file 1 corresponding to CPU1.
[0170] In the embodiment of the present application, the CPU and the DAX file correspond one to one. Specifically, it can be managed through a mapping table of the CPU and the DAX file. The LMD process can determine the DAX file corresponding to the CPU according to the CPU number.
[0171] The LMD process determines that the CPU executing the application process is CPU1 based on the CPU number obtained by parsing the storage space allocation request, and then determines the DAX file 1 (also called the first DAX file) corresponding to CPU1. For CPU1, the access path provided by DAX file 1 is proximal access. Through DAX file 1, CPU1 can access the extended memory provided by the CXL memory expander through proximal access.
[0172] S904: The LMD process allocates a target storage space for the application process through the DAX file 1, and marks the target storage space as invalid.
[0173] The application space size in the storage space allocation request is used to indicate the size of the storage space to be allocated. The LMD process uses the DAX file 1 to allocate the target storage space of the application process with the application space size.
[0174] The LMD process can manage the status of the storage space of multiple DAX files (such as the first DAX file and the second DAX file). The storage space of the DAX file can be divided into two states: invalid and available. The storage space in the invalid state may be occupied or disabled; the storage space in the available state can be allocated to the application.
[0175] Before the application process, the LMD process first traverses the status of the storage space in DAX file 1, uses the storage space with the status of available to allocate the target storage space for the application process, and after allocating the target storage space, marks the status of this part of the storage space as invalid to avoid repeated allocation.
[0176] As an example, the LMD process opens DAX file 1, obtains the file descriptor of DAX file 1, and determines the offset address and space length of DAX file 1. The offset address is the starting address of the storage space allocated for the application process, and the space length is used to indicate the size of the storage space allocated for the application process. The offset address of DAX file 1 is also called the first offset address, which is the starting address of the target storage space in DAX file 1.
[0177] After the LMD process marks the status of the target storage space as invalid in DAX file 1, it will also mark the status of the space corresponding to the target storage space as invalid in other DAX files. The same piece of storage space in the CXL memory expander is mapped to different addresses in different DAX files. Taking DAX file 1 and DAX file 2 (also called the second DAX file) as an example, the same piece of storage space in the CXL memory expander is mapped to different addresses in DAX file 1 and DAX file 2. Therefore, the address of the target storage space in DAX file 1 is different from that in DAX file 2. The LMD process can determine the space corresponding to the target storage space in DAX file 2 after marking the status of the target storage space as invalid in DAX file 1, and then synchronously mark the status of the space corresponding to the target storage space as invalid in DAX file 2. This prevents the LMD process from repeatedly allocating the same piece of space when allocating space for other application processes, reduces memory conflicts, and improves the stability of extended memory access.
[0178] Specifically, the LMD process can convert the first offset address of the target storage space in DAX file 1 to obtain the second offset address of the target storage space in DAX file 2. The space length of the target storage space in DAX file 1 is the same as the space length in DAX file 2. Furthermore, the LMD process marks the allocated storage space as invalid in DAX file 2. The second offset address is the starting address of the target storage space in DAX file 2.
[0179] As an example, the target storage space allocated by the LMD process to the application process is storage space A. Storage space A is actually provided by the CXL memory expander, and the address mapped by storage space A in DAX file 1 is different from the address mapped in DAX file 2. After the LMD process marks storage space A as invalid in DAX file 1, it determines the address corresponding to storage space A in DAX file 2, and then marks storage space A as invalid in DAX file 2.
[0180] The above only takes DAX file 1 and DAX file 2 as examples. If DAX file N is also included, the LMD process determines the Nth offset address of the target storage space in DAX file N according to the first offset address of the target storage space in DAX file 1, and then marks the space status corresponding to the target storage space in DAX file N as invalid according to the Nth offset address and the space length.
[0181] It should be noted that when the status of a space in a DAX file changes, the LMD process will synchronously change the status of the same space in other DAX files to achieve spatial status synchronization of multiple DAX files.
[0182] S905: The LMD process returns storage space allocation information to the application process.
[0183] The storage space allocation information includes file descriptors, offset addresses, space lengths, and read / write permissions. After the LMD process sends the storage space allocation information to the application process, the application process parses the file descriptor, offset address, space length, and read / write permissions, and uses the mmap interface to map the extended memory provided by the CXL memory expander allocated by the LMD process to the application process, and then performs read / write operations on the extended memory.
[0184] In the above process, the LMD process uses all the extended memory mapped by the DAX file to allocate storage space for the application process, and the CPU can access all the memory provided by the CXL memory expander through the DAX file. In addition, after allocating space for the application process, the allocated space is marked as invalid to prevent the LMD process from repeatedly allocating the same space when allocating space for other application processes, reducing memory conflicts and improving the stability of extended memory access.
[0185] S906: The LMD process receives the storage space release request sent by the application process.
[0186] The storage space release request includes the CPU number, the first offset address and the space length. When the application process completes using the storage space, the application process sends a storage space release request to the LMD process to request the release of the storage space allocated by the LMD process for the application process, that is, to release the target storage space.
[0187] S907: The LMD process releases the target storage space according to the storage space release request.
[0188] After the LMD process is awakened by the storage space release request, it parses the storage space release request to determine the offset address (i.e., the first offset address), space length, and CPU number of the released space. The LMD process determines the corresponding DAX file according to the CPU number, i.e., DAX file 1, and releases the data in the storage space in DAX file 1 according to the first offset address and space length, and marks the storage space as valid in DAX file 1.
[0189] At the same time, the LMD process also marks the status of the target storage space corresponding to the space in other DAX files as valid. The LMD process maps the offset address and space length of the storage space in DAX file 1 to obtain the offset address and space length of the storage space in DAX file 2, and then marks the storage space as valid in DAX file 2.
[0190] After the operation is completed, the LMD process enters a dormant state. After the application process is finished, by requesting to release the space allocated by the LMD process, the system can effectively reclaim the used memory resources and avoid unnecessary occupation of extended memory resources.
[0191] The embodiments of the present application also provide corresponding computing devices and computer storage media for implementing the solutions provided by the embodiments of the present application.
[0192] The computing device includes a memory and a processor, the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes so that the device executes the memory management method described in any embodiment of the present application.
[0193] The computer storage medium stores code, and when the code is executed, the device executing the code implements the memory management method described in any embodiment of the present application.
[0194] The present application also provides a computer program product, which includes code. When the code is executed, the device executing the code implements the memory management method described in any embodiment of the present application. The "first" and "second" (if any) mentioned in the embodiments of the present application are only used as name identification and do not represent the first and second in order.
[0195] Through the description of the above implementation methods, it can be known that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a storage medium, such as a read-only memory (ROM) / RAM, a magnetic disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network communication device such as a router) to execute the methods described in each embodiment of the present application or some parts of the embodiments.
[0196] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.
[0197] The above description is merely an exemplary embodiment of the present application and is not intended to limit the protection scope of the present application.
Claims
1. A memory management method, characterized in that: Applied to a computing device, a plurality of computing nodes in the computing device are connected to a computing fast link CXL memory expander through a plurality of interfaces on the CXL memory expander, and the method comprises: Receive a storage space allocation request sent by a first computing node; the storage space allocation request includes a computing node number and an application space size; Determine a first direct access DAX file corresponding to the first computing node according to the computing node number; the storage space of the first DAX file is a mapping of the extended memory accessible through the first interface in the computing device; the first interface is used to access all extended memories of the CXL memory expander; Allocate target storage space for the first computing node based on the first DAX file and the requested space size; The status of the space corresponding to the target storage space in the second DAX file is marked as invalid; the storage space of the second DAX file is a mapping of the extended memory accessible through the second interface in the computing device; the second interface is used to access all extended memories of the CXL memory expander; the second DAX file corresponds to a second computing node.
2. The method according to claim 1, characterized in that The allocating target storage space to the first computing node based on the first DAX file and the requested space size includes: Obtain a file descriptor of a first DAX file, and determine a first offset address and a space length in the first DAX file; the first offset address is a starting address of the target storage space in the first DAX file, and a space size corresponding to the space length is greater than or equal to the applied space size; The file descriptor, the first offset address and the space length are sent to the first computing node.
3. The method according to claim 2, characterized in that The method further comprises: The status of the target storage space is marked as invalid in the first DAX file.
4. The method according to claim 3, characterized in that The step of marking the status of the space corresponding to the target storage space in the second DAX file as invalid comprises: Determine a second offset address according to the first offset address; the second offset address is the starting address of the space corresponding to the target storage space in the second DAX file; According to the second offset address and the space length, a status of the space corresponding to the target storage space is marked as invalid in the second DAX file.
5. The method according to any one of claims 2 to 4, characterized in that: The method further comprises: Receiving a storage space release request sent by a first computing node; the storage space release request includes the computing node number, the first offset address and the space length; releasing the data in the target storage space in the first DAX file according to the first offset address and the space length; and marking the state of the target storage space as valid in the first DAX file; According to the storage space release request, the status of the space corresponding to the target storage space is marked as valid in the second DAX file.
6. The method according to claim 5, characterized in that The step of marking the status of the space corresponding to the target storage space as valid in the second DAX file according to the storage space release request includes: Determine the second offset address according to the first offset address; the second offset address is the starting address of the space corresponding to the target storage space in the second DAX file; According to the second offset address and the space length, the status of the space corresponding to the target storage space is marked as valid in the second DAX file.
7. The method according to claim 1, characterized in that Before receiving the storage space allocation request sent by the first computing node, the method further includes: Obtaining extended memory information of the CXL memory expander; The memory capacity accessed by each interface of the CXL memory expander is configured as the total capacity of the extended memory of the CXL memory expander according to the extended memory information.
8. The method according to claim 7, characterized in that The step of configuring the memory capacity accessed by each interface of the CXL memory expander as the total capacity of the extended memory of the CXL memory expander according to the extended memory information includes: Configure the interface number, physical starting address and address length of each interface, and configure the number, decoder starting address and decoding length of each host management device memory HDM decoding register group; each interface and each HDM decoding register group correspond one to one; the address length of each interface and the decoding length of each HDM decoding register correspond to the total capacity of the extended memory; Creating a plurality of DAX files according to the interface number, the physical start address and the address length of each interface; wherein the plurality of DAX files correspond to the plurality of interfaces one by one; Creating multiple virtual HDM decoding register groups according to the number of each HDM decoding register group, the decoder start address and the decoding length; each HDM decoding register group corresponds to the multiple virtual HDM decoding register groups one by one; The respective HDM decoding register groups are mapped to the plurality of virtual HDM decoding registers.
9. The method according to claim 1, characterized in that: The first computing node is a first processor, and the second computing node is a second processor.
10. A server, characterized in that: comprising a processor, and a memory communicatively connected to the processor; The memory is used to store computer-executable instructions; The processor is used to execute the computer-executable instructions stored in the memory to implement the memory management method described in any one of claims 1-9.
Citation Information
Cited By
Register resource management method and device for GPU and medium
CN119477664A
Server memory expansion method and electronic equipment
CN121364952A
A server memory expansion method and electronic device
CN121364952B
