A method and device for optimizing DPU side memory, a DPU chip, an AI cluster acceleration card and a cloud computing host

Abstract

The application provides a method, device, DPU chip, AI cluster acceleration card and cloud computing host for optimizing DPU side memory, comprising: sending a memory optimization request to a host; receiving a response notification from the host; constructing a virtual memory exchange device according to the response notification; when a memory low water line threshold is triggered, exchanging out DPU side memory into the virtual memory exchange device; when memory in the virtual memory exchange device needs to be accessed, exchanging in corresponding memory in the successfully reserved host side memory address range into the DPU side memory. According to the technical scheme of the application, the memory resource dynamic optimization and expansion of the DPU can be realized at low cost, the disk IO delay is reduced, and the overall system performance is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer networks, and in particular to a method and device for optimizing DPU memory, a DPU chip, an AI cluster acceleration card and a cloud computing host. BACKGROUND

[0002] A DPU (Data Processing Unit) is a special-purpose processor designed around data-centric architecture, used to complete acceleration tasks for network, storage and security. With the rise of cloud computing, AI (Artificial Intelligence), big data and other technologies, the technical requirements for data centers are becoming higher and higher, and it is necessary to provide users with certain computing power, storage space and application services to cope with a large number of complex scene applications.

[0003] In the AI computing cluster scenario, the GPU (Graphics Processing Unit) is responsible for deep learning inference and training, while the DPU as a data processing unit undertakes data movement, storage acceleration, network communication optimization and other tasks. The AI cluster combining DPU and GPU can significantly improve computing efficiency, reduce CPU burden and optimize data transmission performance. Generally speaking, a DPU supports multiple GPUs to provide data transmission acceleration, network optimization, storage management and other functions. This 1:N (DPU:GPU) architecture design enables GPU computing resources to work more efficiently, reduces CPU burden and optimizes data transmission paths.

[0004] As the business scenarios supported by DPU become more complex, the resource occupation of CPU and DDR (Double Data Rate memory) on DPU is increasing, so the stability of the main core components of SOC (System on Chip) is very important. If the system resources are insufficient, it will cause business jitter, which will have a greater impact on business in the large cluster scenario.

[0005] Current DPU cards are generally configured with serial hard disks (SATA) for cost considerations, but the performance of the SATA protocol is not very high, and the maximum theoretical bandwidth can reach 600MB / s. The actual bandwidth is usually close to 500MB / s to 50MB / s. The bandwidth of DDR5 can reach 36GB / s, which is much lower than the bandwidth of DDR. When the system is running at full load and triggers the low water line, the performance of the system will be affected by the disk IO delay when the system memory is swapped to the disk, which will cause performance jitter, and serious IO delay may also cause business interruption. At present, in order to optimize performance, it is considered to update the disk on the DPU and adjust it to a disk of the NVME (Non-Volatile Memory Express, Non-Volatile Memory Host Controller Interface Specification) type with better performance, but in general, the type of disk is limited by the interface type of the hardware, so there are high requirements for the size and cost of the DPU. Or use the way of increasing the memory on the DPU to optimize, upgrade the memory, but this way usually also means higher requirements for the size and cost of the DPU. The essence of the above two ways is to upgrade the hardware, and it is still difficult to implement these two solutions for mature products.

[0006] Therefore, a technical solution is needed to dynamically optimize and expand the memory resources of the DPU at a low cost, while reducing disk IO delay and improving the overall performance of the system. SUMMARY

[0007] The present application aims to provide a method, device, DPU chip, AI cluster acceleration card and cloud computing host for optimizing DPU side memory, which can dynamically optimize and expand the memory resources of the DPU at a low cost, while reducing disk IO delay and improving the overall performance of the system.

[0008] According to an aspect of the present application, a method for optimizing DPU side memory is provided, which is used for a DPU and includes:

[0009] sending a memory optimization request to a host through a control channel;

[0010] receiving a response notification from the host, the response notification including an address range of successfully reserved host side memory;

[0011] According to the response notification, a virtual memory switching device is constructed using the successfully reserved host side memory, and the virtual memory switching device is used to implement the operation of writing DPU side memory data to the successfully reserved host side memory through a DMA engine;

[0012] When a memory low water line threshold is triggered, the DPU side memory is swapped to the virtual memory switching device;

[0013] When memory in the virtual memory swap device needs to be accessed, the corresponding memory in the successfully reserved address range of the host side memory is swapped into the DPU side memory.

[0014] According to some embodiments, sending a memory optimization request to the host through the control channel includes:

[0015] Obtaining a user memory optimization option;

[0016] According to the user memory optimization option, sending a memory optimization request to the host through the control channel,

[0017] The memory optimization request includes a request for a reserved memory space size, and the control channel includes a PCIe link.

[0018] According to some embodiments, according to the response notification, a virtual memory swap device is constructed using the successfully reserved host side memory, including:

[0019] Constructing virtual memory swap device structure related information, the virtual memory swap device structure related information including: a swap mapping table and a plurality of swap slots, wherein the virtual memory swap device is divided into the plurality of swap slots, and the swap mapping table is used to manage the entries of the plurality of swap slots;

[0020] Registering the virtual memory swap device with a memory swap module of a DPU side operating system kernel.

[0021] According to some embodiments, swapping the DPU side memory out to the virtual memory swap device includes:

[0022] According to the virtual memory swap device structure related information, swapping the DPU side memory out to the virtual memory swap device through the DMA engine, so that the DPU side memory writes to the corresponding memory in the successfully reserved address range of the host side memory;

[0023] Swapping the corresponding memory in the successfully reserved address range of the host side memory into the DPU side memory includes:

[0024] According to the virtual memory swap device structure related information, swapping the corresponding memory in the successfully reserved address range of the host side memory into the virtual memory swap device through the DMA engine.

[0025] According to some embodiments, the method further includes:

[0026] Obtaining a memory optimization shutdown request from a user;

[0027] According to the memory optimization closing request, it is checked whether there is data not written back in the current swap mapping table, if not, a memory release notification is sent to the host to release the memory host corresponding to the address range of the successfully reserved host-side memory, and the virtual memory swap device is unregistered and removed from the operating system kernel swap module on the DPU side to complete resource cleaning.

[0028] According to another aspect of the present application, a method for optimizing DPU-side memory is provided, the method is used for a host, comprising:

[0029] Receiving a memory optimization request from the DPU through a control channel;

[0030] According to the memory optimization request, reserving host-side memory of a corresponding size, and sending a response notification to the DPU, the response notification including an address range of successfully reserved host-side memory;

[0031] When the DPU swaps out DPU-side memory into a virtual memory swap device, receiving and storing DPU-side memory data swapped out by the DPU in the successfully reserved host-side memory, wherein the virtual memory swap device is used to realize writing the DPU-side memory data to the successfully reserved host-side memory through a DMA engine;

[0032] When the DPU needs to access memory in the virtual memory swap device, sending the DPU-side memory data stored in the successfully reserved host-side memory to the DPU-side memory.

[0033] According to some embodiments, the virtual memory swap device is constructed according to virtual memory swap device structure related information, the virtual memory swap device structure related information including a swap mapping table and a plurality of swap slots, wherein the virtual memory swap device is split into the plurality of swap slots, and the swap mapping table is used to manage entries of the plurality of swap slots.

[0034] According to some embodiments, the method further comprises:

[0035] Receiving a memory release notification from the DPU, thereby releasing the successfully reserved host-side memory.

[0036] According to some embodiments, according to the memory optimization request, reserving host-side memory of a corresponding size, and sending a response notification to the DPU, comprises:

[0037] According to the memory optimization request, checking the request memory size, if the request memory size is insufficient, sending a rejection request as the response notification to the DPU;

[0038] reserving a host side memory of a corresponding size, sending a rejection request as the response notification to the DPU if the reservation fails, and sending a success response and an address range of the successfully reserved host side memory as the response notification to the DPU if the reservation succeeds,

[0039] The memory optimization request includes a requested memory space size, and the control channel includes a PCIe link.

[0040] According to another aspect of the present application, an apparatus for optimizing DPU side memory is provided, comprising:

[0041] A request module is configured to send a memory optimization request to a host through a control channel.

[0042] A response acceptance module is configured to receive a response notification from the host, the response notification including an address range of successfully reserved host side memory.

[0043] A virtual device construction module is configured to construct a virtual memory switching device using the successfully reserved host side memory according to the response notification, so that the virtual memory switching device is used to implement, by a DMA engine, writing of DPU side memory data to memory corresponding to the address range in the successfully reserved host side memory.

[0044] A memory swap-out module is configured to swap out DPU side memory to the virtual memory switching device when a kernel triggers a memory low-water line threshold.

[0045] A memory swap-in module is configured to swap in corresponding memory in the successfully reserved host side memory address range to DPU side memory in the virtual memory switching device when the memory in the virtual memory switching device needs to be accessed.

[0046] According to another aspect of the present application, an apparatus for optimizing DPU side memory is provided, comprising:

[0047] A request reception module is configured to receive a memory optimization request from a DPU through a control channel.

[0048] A memory reservation module is configured to reserve a host side memory of a corresponding size according to the memory optimization request, and send a response notification to the DPU, the response notification including an address range of successfully reserved host side memory.

[0049] A memory swap-out response module is configured to receive and store DPU side memory data swapped out by the DPU in the successfully reserved host side memory when the DPU swaps out DPU side memory to a virtual memory switching device, wherein the virtual memory switching device is used to implement, by a DMA engine, writing of the DPU side memory data to the successfully reserved host side memory.

[0050] a memory swap-in response module, configured to send the DPU-side memory data stored in the successfully reserved host-side memory to the DPU-side memory when the DPU needs to access the memory in the virtual memory swap device.

[0051] According to another aspect of the present application, a DPU chip is provided, comprising a processor and a memory, and the memory stores a computer program, and the processor implements the method according to any one of the above when executing the computer program stored in the memory.

[0052] According to another aspect of the present application, an AI cluster acceleration card is provided, comprising: a DPU chip according to the above and a plurality of interfaces and a GPU, and the DPU chip communicates with the outside through the plurality of interfaces.

[0053] According to another aspect of the present application, a cloud computing host is provided, comprising: an AI cluster acceleration card according to the above and a central processing unit and a memory, the AI cluster acceleration card is used for data acceleration, network optimization and / or storage management, the central processing unit is used for processing data transferred by the AI cluster acceleration card, and the memory stores a computer program, and the processor implements the method according to any one of the above when executing the computer program stored in the memory.

[0054] According to another aspect of the present application, a non-transitory computer readable storage medium is provided, which stores computer readable instructions, and the instructions make the processor execute the method according to any one of the above when executed by the processor.

[0055] According to the embodiments of the present application, the technical scheme of the present application realizes cross-device memory collaborative management by taking the host (HOST) side memory as the remote swap space of the DPU, and significantly improves the memory use efficiency and performance of the DPU system. When the local memory of the DPU is tight, instead of relying on the slower SATA disk for data swap-out, the inactive memory page is directly migrated to the DDR memory of the HOST through the high-speed PCIe link and the DMA engine. This way greatly reduces the IO delay of memory swap, avoids the business jitter caused by disk writing, and guarantees the stable operation of the key task on the DPU. Secondly, the available memory capacity of the DPU is effectively expanded. With the larger memory resources of the HOST, the DPU can process larger-scale data and more complex tasks, breaking through the limitation of its own physical memory. This elastic memory expansion mechanism is particularly suitable for DPU applications in high-throughput scenarios such as network and storage, and improves the overall carrying capacity of the system.

[0056] It should be understood that the above general description and the following detailed description are only exemplary and cannot limit the present application. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0058] Figure 1 This diagram illustrates the prior art Linux OS SWAP memory reclamation.

[0059] Figure 2 A system block diagram is shown for a method of optimizing DPU-side memory according to an example embodiment.

[0060] Figure 3 A flowchart of a method for optimizing DPU-side memory is shown according to an example embodiment.

[0061] Figure 4 A flowchart of a method for optimizing DPU-side memory is shown according to another example embodiment.

[0062] Figure 5 The diagram illustrates an optimized operating system memory reclamation process according to an example embodiment of a method for optimizing DPU-side memory.

[0063] Figure 6 The illustration shows a schematic diagram of an implementation method for optimizing DPU-side memory according to an example embodiment.

[0064] Figure 7 A schematic diagram of an apparatus for optimizing DPU-side memory is shown according to an example embodiment.

[0065] Figure 8 A schematic diagram of an apparatus for optimizing DPU-side memory is shown according to another example embodiment.

[0066] Figure 9 The illustration shows a method for optimizing DPU-side memory according to an example embodiment, including a schematic diagram of the DPU-side and HOST-side memory reclamation process.

[0067] Figure 10 A schematic diagram of an AI cluster acceleration card according to an example embodiment is shown.

[0068] Figure 11 A block diagram of a cloud computing device according to an exemplary embodiment is shown. Detailed Implementation

[0069] Example embodiments now will be described more fully hereinafter with reference to the accompanying drawings. Example embodiments, however, can be implemented in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. Like reference numerals refer to like elements throughout the several views and, thus, description of the same elements will not be repeated.

[0070] Moreover, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the

[0071] The block diagrams in the drawings show only the functionality of the embodiments and do not imply that the functions must be implemented in a particular manner. For example, the functions of one or more of the illustrated components can be implemented in hardware, software, or a combination of both hardware and software. Moreover, the illustrated components can be implemented in one or more physical devices.

[0072] The flow diagrams depicted in the figures are merely exemplary and do not necessarily include all of the steps or operations, nor do they necessarily indicate the order in which the steps or operations can be performed. For example, some operations can be performed in parallel, some operations can be omitted, and some operations can be combined or partially combined with other operations. The order in which the operations are depicted is not necessarily the order in which the operations are performed.

[0073] It should be understood that although the terms first, second, third, etc. can be used herein to describe various elements, components, regions, layers and / or sections, these elements, components, regions, layers and / or sections should not be limited by these terms. These terms are only used to distinguish one element, component, region, layer or section from another element, component, region, layer or section. Thus, a first element, component, region, layer or section discussed below could be termed a second element, component, region, layer or section without departing from the teachings of the present disclosure. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0074] The user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0075] Those skilled in the art can understand that the drawings are only schematic views of the example embodiments, and the modules or flows in the drawings are not necessarily essential for implementing the present application, and therefore cannot be used to limit the protection scope of the present application.

[0076] Memory has always been a component with high system occupation, and the stability of memory resources will directly affect the operation of system business. Figure 1 When the system business abnormally consumes a lot of memory, or when the ECC error exists in the memory, causing the physical memory block to be isolated, or when the system memory usage approaches the low waterline threshold of the system, the memory recycling task (Linux OS Swap mechanism) will be triggered. The Linux OS Swap mechanism will back up the inactive anonymous memory data to the swap partition (Swap partition), and write the inactive file page back to the disk, thereby realizing system memory recycling. Generally, the Swap partition is the DPU local disk.

[0077] Referring to Figure 1 In a conventional Linux operating system, when the Linux system starts, the kernel loads and initializes the Swap subsystem. First, the system detects the size of the available physical memory and sets up the Swap partition or file according to the configuration file. When a process needs to apply for memory, the system will first check whether the current available physical memory is sufficient. If the memory is sufficient, the memory will be directly allocated to the process for use. However, when the physical memory is insufficient, the system will trigger the memory swapping mechanism to release part of the memory space for new processes to use. Once the memory is detected to be insufficient, the system will select appropriate pages from the currently occupied memory pages for swapping. The selection criteria are usually based on the access frequency and recent usage of the pages, and the pages that have not been accessed for a long time are preferentially selected. The selected pages will be written to the pre-configured Swap partition or file, thereby releasing the corresponding physical memory space. The swapped-out page data will be written to the Swap space on the disk, and the specific location is recorded by Swapslot and Slot entry, which is saved in Swap_map, so that the corresponding page data can be quickly found and restored when needed. In this way, the system can temporarily store infrequently used data on the disk when the memory is tight, thereby freeing up valuable physical memory resources.

[0078] Since the current DPU card is generally configured with SATA for cost considerations, the performance of the SATA protocol will not be too high, and the maximum theoretical bandwidth can reach 600 MB / s. The actual bandwidth is usually close to 500 MB / s to 550 MB / s. The bandwidth of DDR5 can reach 36 GB / s, which is much lower than the bandwidth of DDR. When the system is running at full load and triggers the low waterline, the performance of the system will be affected by the disk IO delay, causing performance jitter, and severe IO delay may also cause business interruption.

[0079] To this end, the application provides a method and device for optimizing DPU memory, a DPU chip, an AI cluster acceleration card, and a cloud computing host, which can realize dynamic optimization and expansion of DPU memory resources at low cost, reduce disk IO delay, and improve overall system performance. According to embodiments, the technical scheme of the application realizes cross-device memory collaborative management by using the HOST side memory as the remote exchange space of the DPU, significantly improving the memory use efficiency and performance of the DPU system. When the local memory of the DPU is tight, instead of relying on the slower SATA disk for data swap-out, the inactive memory pages are directly migrated to the DDR memory of the HOST through the high-speed PCIe link and the DMA engine. This method greatly reduces the IO delay of memory exchange, avoids business jitter caused by disk writing, and ensures the stable operation of critical tasks on the DPU. Secondly, the available memory capacity of the DPU is effectively expanded. With the larger memory resources of the HOST, the DPU can handle larger-scale data and more complex tasks, breaking through the limitation of its own physical memory. This elastic memory expansion mechanism is particularly suitable for DPU applications in high-throughput scenarios such as networks and storage, improving the overall carrying capacity of the system.

[0080] Before describing the embodiments of the application, some terms or concepts related to the embodiments of the application are explained and described.

[0081] Swap refers to moving temporarily inactive pages from physical memory to a pre-set area on the disk (called Swap partition or Swap file), thereby releasing physical memory for more needed processes. This process is transparent to user-mode application programs, i.e., the application program does not know that its data is swapped in or out of physical memory.

[0082] Swap partition: directly translated as "swap partition", it is a mechanism used by Linux operating system to extend physical memory (RAM). When the physical memory of the system is insufficient, inactive memory pages will be moved to the Swap partition, thereby releasing physical memory for more important processes.

[0083] Swap space (Swap Space): refers to a part of disk space in the system dedicated to virtual memory management. It can be a separate partition (called Swap partition) or one or more files located in the file system (called Swap file). When the physical memory of the system is insufficient, inactive memory pages will be moved to the Swap space to release physical memory for other processes.

[0084] Swap slot: This term is not a direct standard term in Linux documentation, but it can be considered as the minimum allocation unit in Swap space, i.e., the location occupied by each memory page swapped out to disk.

[0085] Slot entry: Similarly, "slot entry" is not a widely used standard term, but according to the context, it can refer to an information item describing a certain Swap slot, including the state, location, and other metadata of the slot.

[0086] The example embodiments of the present application are described below in conjunction with the accompanying drawings.

[0087] Figure 2 A system block diagram of a method for optimizing DPU-side memory according to an example embodiment is shown.

[0088] Referring to Figure 2 , a system block diagram of a method for optimizing DPU-side memory is shown, where the HOST side includes CPU, DDR controller, DDR memory, and PCIe Root Complex (RC), and the DDR memory serves as the main storage resource with a large capacity. The DPU side includes CPU, DDR memory, and DMA Engine. Generally, the DPU DDR memory is relatively small, but it can exchange data with the HOST DDR at high speed through the DMA Engine.

[0089] The PCIe link connects the HOST RC and the DPU DMA Engine, providing a high-speed data transmission channel. Compared with the traditional SATA interface, the PCIe link has higher bandwidth and lower latency, which can significantly improve data transfer efficiency.

[0090] In a traditional Linux OS, when the local DDR memory of the DPU is insufficient, the system writes inactive page data to the local SATA disk as Swap space, which causes high IO delay and performance degradation. In the design scheme of the present application, the DDR on the HOST side is used as the Swap device on the DPU side, and the HOST bus and size can usually support larger memory. When the DPU detects that the local DDR memory usage rate is close to the preset low water threshold, the memory expansion function is started. The DPU uses its DMAEngine to directly write inactive memory pages into the DDR memory on the HOST side through the PCIe link, instead of writing into the SATA disk. It seems that the DDR on the HOST side is used as a backup of the DDR on the DPU side, and the content of the DDR on the DPU side is copied to the HOST DDR through system DMA, so as to release the memory to the operating system. This process is controlled by the DPU CPU, ensuring data integrity and consistency. After the local DDR memory of the DPU is released, it can be reallocated for use by other tasks or processes, thereby avoiding business jitter caused by insufficient memory.

[0091] According to some embodiments, the HOST DDR and the DPU DDR data migration is performed using the system DMA module on the DPU, mainly through the PCIe link. The overall performance bandwidth is several times that of the read-write performance of the SATA, which can significantly improve the system IO read-write delay. The DPU DMA Engine is responsible for copying the selected memory pages from the DPU DDR to the HOST DDR. Due to the high bandwidth characteristics of the PCIe link, this process is very fast and almost does not have a significant impact on system performance. Similarly, when the DPU needs to access the data previously exchanged, it is also read from the HOST DDR and restored to the DPU DDR through the DMA Engine.

[0092] According to some embodiments, in order to further improve the utilization rate of the HOST DDR, a memory compression technology can be applied before data migration. For example, a fast compression algorithm such as LZ4, ZSTD, etc. is used to compress the data before writing it into the HOST DDR. The compressed data occupies less storage space, so that the HOST DDR can accommodate more memory pages on the DPU side, thereby realizing more efficient memory expansion.

[0093] According to some embodiments, the design scheme according to the application eliminates the high delay bottleneck of disk read and write by transferring the swap operation from the SATA disk to the HOST DDR, significantly improving the IO performance of the system. The DPU service is no longer subject to the fluctuation of disk IO during operation, and can maintain more stable and consistent performance. Compared with the SATA interface, the PCIe link provides several times or even dozens of times of bandwidth, so that the data exchange speed is greatly improved, further enhancing the throughput capacity of the system. By skillfully using the DMA module and the PCIe link on the DPU, the HOST DDR is used as a virtual swap device of the DPU, realizing an efficient and low-delay memory expansion mechanism. Combined with the memory compression technology, the utilization rate of the HOST DDR is further improved, providing a more stable and high-performance running environment for the DPU service.

[0094] Figure 3 A flow chart of a method for optimizing DPU-side memory according to an example embodiment is shown.

[0095] Figure 6 An implementation method diagram of a method for optimizing DPU-side memory according to an example embodiment is shown.

[0096] Referring to Figure 6 , the design scheme of the application adds two memory management modules on the DPU side and the HOST side, i.e. a DPU-side memory management module and a HOST-side memory management module. The HOST-side memory management module is mainly used for management of HOST reserved memory and is responsible for negotiation with the DPU-side memory management module.

[0097] The DPU-side memory management module is used for DPU-side swap partition management and negotiation with the HOST-side memory management module. As the manager of the memory swap function negotiation, it is responsible for starting and closing the optimization function.

[0098] Referring to Figure 3 , the method is used for a DPU, and the specific steps are described as follows.

[0099] In S301, a memory optimization request is sent to the host side through a control channel.

[0100] According to some embodiments, referring to Figure 6The DPU-side memory management module sends a memory optimization request to the host-side memory management module through a control channel, and the DPU-side memory management module obtains a user memory optimization option; according to the user memory optimization option, the DPU-side memory management module sends a memory optimization request to the host-side memory management module through the control channel, wherein the memory optimization request includes a requested reserved memory space size, and the control channel includes a PCIe link. The DPU-side memory management module can receive a user-set memory optimization option, such as a desired size of host memory to be used as a swap space. According to the user's setting, the DPU sends a memory optimization request to the host side through the control channel. The memory optimization request includes a requested reserved memory space size, which contains the required memory size (for example, 1 GB, 2 GB) and other optional parameters, such as quality of service (QoS), access rights, whether to enable compression, etc. The DPU-side memory management module sends a "memory optimization request" to the host-side memory management module through the control channel (such as a PCIe configuration space, a mailbox mechanism, or a dedicated control link), which is not only used for data transmission but also can transmit such control messages to realize communication and coordination between the two sides.

[0101] At S303, a response notification from the host is received, and the response notification includes an address range of successfully reserved host-side memory.

[0102] According to some embodiments, the DPU-side memory management module receives a reply message from the host (HOST) side. If the host agrees with the request, the reply contains a "success response" and a reserved memory address range, indicating that this part of the host memory has been allocated to the DPU for memory optimization. If the host refuses the request, the reply is "refused request", which may be due to insufficient memory or policy restrictions, etc. After receiving the refusal, the DPU cannot use this mechanism for memory expansion.

[0103] At S305, according to the response notification, a virtual memory swap device is constructed using the successfully reserved host-side memory, and the virtual memory swap device is used to realize writing of DPU-side memory data to the successfully reserved host-side memory through a DMA engine.

[0104] According to some embodiments, the DPU-side memory management module creates a virtual memory swap device according to the response notification returned by the host. If a success response is received, the module uses the memory with the address range of the host-side memory in the response to establish a usable virtual swap space.

[0105] According to some embodiments, the DPU-side memory management module constructs a virtual memory swap device using the successfully reserved host-side memory according to the response notification. The DPU-side memory management module constructs virtual memory swap device structure information, which includes a swap map and a plurality of swap slots, wherein the virtual memory swap device is split into the plurality of swap slots, and the swap map is used to manage the entries of the plurality of swap slots. The DPU-side memory management module registers the virtual memory swap device with a memory swap module (Swap module) of a DPU-side operating system kernel. The swap slot is a slot into which a virtual memory swap device is split. The swap map is used to manage slot entries in the swap slot. When a memory low watermark threshold is triggered, the virtual memory swap device finds a slot entry from the swap slot through the swap map to give the memory for saving memory data (in essence, the memory data is moved to the host-side memory through DMA).

[0106] According to some embodiments, after the device is created, the DPU side also constructs related management structures, including a swap map and a swap slot. The swap map is used to record the correspondence between DPU-side memory pages and host memory addresses, and the swap slot is used to manage the allocation and release of swap space. Finally, the DPU-side memory management module registers the virtual memory swap device with the kernel Swap module of the operating system, so that it is recognized as a legal swap device by the system, thereby participating in subsequent memory swap-out and swap-in operations.

[0107] In S307, when a memory low watermark threshold is triggered, the DPU-side memory is swapped out to the virtual memory swap device.

[0108] According to some embodiments, the DPU-side memory management module swaps out the DPU-side memory to the virtual memory swap device through the DMA engine according to the virtual memory swap device structure information, so that the DPU-side memory is written to the memory corresponding to the successfully reserved host-side memory address range. When the system kernel detects that the DPU-side memory usage is lower than the set low watermark threshold, it indicates that the memory pressure is large and part of the memory needs to be released. At this time, the DPU-side memory management module starts the memory recycling mechanism to swap out the temporarily unused memory pages through the virtual memory swap device. During the swapping-out process, the DPU-side memory management module determines the storage location of the data according to the previously established swap map and swap slot information, and writes the memory data to the previously successfully reserved host-side memory address through the DMA engine, thereby releasing the DPU local memory.

[0109] At S309, when it is needed to access the memory in the virtual memory swap device, the corresponding memory in the successfully reserved host-side memory address range is swapped in to the DPU-side memory.

[0110] According to some embodiments, the DPU-side memory management module swaps the corresponding memory in the successfully reserved host-side memory address range into the virtual memory swap device through the DMA engine according to the virtual memory swap device structure related information. When the DPU needs to access the data previously swapped out to the virtual memory swap device, the system triggers the memory swap-in operation. The DPU-side memory management module finds the corresponding address of the data in the host-side memory according to the swap mapping table and other structure information. Then, the data is read from the address through the DMA engine and transmitted back to the DPU, completing the memory swap-in process and making the data available again.

[0111] According to some embodiments, the method further comprises: the DPU-side memory management module obtaining a memory optimization shutdown request from a user; according to the memory optimization shutdown request, the DPU-side memory management module checks whether there is data that has not been written back in the current swap mapping table, and if not, the DPU-side memory management module sends a memory release notification to the host-side memory management module to release the memory corresponding to the previously successfully reserved host-side memory address range, and unregisters and removes the virtual memory swap device from the DPU-side operating system kernel swap module to complete the resource cleaning. Specifically, when the user initiates a memory optimization shutdown request, the DPU-side memory management module will start to stop the function and clean up related resources. First, the module will check the swap mapping table to confirm whether there is memory data being used or not yet recovered. If there is still data to be processed, the system will prevent the shutdown operation to ensure data safety. If there is no data to be processed, the DPU side will send a memory release notification to the host side, informing it that it can release the previously reserved host memory. After receiving the confirmation feedback from the host side, the DPU side will unregister and delete the virtual memory swap device from the local swap module, completing the release and cleaning of all resources.

[0112] Figure 4 A flowchart of a method for optimizing DPU-side memory according to another example embodiment is shown.

[0113] Referring to Figure 4 , a method for optimizing DPU-side memory is shown, which is used for a host and specifically comprises the following steps:

[0114] At S401, a memory optimization request from the DPU is received through a control channel.

[0115] According to some embodiments, the host-side memory management module receives a memory optimization request from the DPU side, which contains information such as the size of memory to be reserved.

[0116] At S403, according to the memory optimization request, the host-side memory of the corresponding size is reserved, and a response notification is sent to the DPU, which includes the address range of the successfully reserved host-side memory.

[0117] According to some embodiments, according to the memory optimization request, the host-side memory management module checks the request memory size, and if the request memory size is abnormal, a rejection request is sent as the response notification to the DPU-side memory management module; the host-side memory management module reserves the host-side memory of the corresponding size, and if the reservation fails, a rejection request is sent as the response notification to the DPU-side memory management module, and if the reservation succeeds, a success response and the address range of the successfully reserved host-side memory are sent as the response notification to the DPU-side memory management module. Specifically, when the host-side memory management module receives the memory optimization request from the DPU, it first checks the request memory size. If the size is beyond the reasonable range or the upper limit allowed by the system and is considered abnormal, the host will directly return a rejection request. If the request size is normal, the host side will try to reserve the memory of the corresponding size. If the reservation fails due to insufficient memory or other reasons, a rejection request will also be returned to the DPU. Only when the memory size is legal and the reservation is successful, the host will send a success response to the DPU, and attach the address range of the reserved host-side memory, indicating that the resources are ready for use by the DPU.

[0118] According to some embodiments, the host side attempts to reserve the local memory of the corresponding size according to the request content. If the reservation is successful, a success response and the allocated memory address range are returned to the DPU; if the reservation fails or the request is illegal, a rejection request is returned.

[0119] At S405, when the DPU swaps out the DPU-side memory to the virtual memory swapping device, the DPU swapped out DPU-side memory data is received and stored in the successfully reserved host-side memory, wherein the virtual memory swapping device is used to realize the operation of writing the DPU-side memory data to the successfully reserved host-side memory through the DMA engine.

[0120] According to some embodiments, when the DPU swaps out local memory data to the virtual memory swap device, the device writes data into the successfully reserved memory region on the HOST side by the DMA engine and stores the swapped out data in the reserved memory. The virtual memory swap device uses the reserved HOST memory as a backend storage space, and achieves efficient and low-latency data transmission by DMA, ensuring that the DPU side memory data is reliably saved to the host side, thereby releasing the DPU local memory resources.

[0121] At S407, when the DPU needs to access the memory in the virtual memory swap device, the DPU side memory data stored in the successfully reserved host side memory is sent to the DPU side memory.

[0122] According to some embodiments, when the DPU needs to access the memory data swapped out to the virtual memory swap device, the system reads the corresponding data stored in the successfully reserved HOST side memory by the DMA engine and transmits it back to the DPU local memory, completing the memory swap-in operation. This process locates the data position based on the pre-established mapping relationship (such as the slot entry in the swap map), ensuring accurate data recovery, so that the DPU can continue to normally access the required content.

[0123] According to some embodiments, the virtual memory swap device is constructed according to virtual memory swap device structure related information, which includes a swap map and a plurality of swap slots, wherein the virtual memory swap device is divided into the plurality of swap slots, the swap map is used to manage the entries of the plurality of swap slots, and the control channel includes a PCIe link.

[0124] According to some embodiments, the successfully reserved host side memory is fixed to avoid being recycled by a memory pinning mechanism. By the memory pinning mechanism, the physical memory corresponding to the address range of the successfully reserved host side memory is fixed, preventing the operating system kernel from releasing, recycling or swapping it out to the disk. This locking ensures that the memory mapping relationship remains persistent and effective when the DPU accesses the region by the DMA engine or the CPU accesses the region by the system, and prevents access interruption or data loss caused by page recycling mechanism. At the same time, during the entire memory swap process, the reserved region can be safely used for the virtual swap operation of the DPU side, improving system stability and performance, and the locking is released and returned to the system management after receiving the release notification.

[0125] According to some embodiments, the method further comprises the host-side memory management module receiving a memory release notification from the DPU-side memory management module, thereby releasing the successfully reserved host-side memory. The host-side can also receive a memory release notification from the DPU when the DPU no longer needs the function. Upon receiving the notification, the host-side releases the previously reserved physical memory and can send a confirmation feedback to the DPU, completing the resource recovery.

[0126] Figure 5 An optimized operating system memory recovery flow diagram is shown for a method for optimizing DPU-side memory according to an example embodiment.

[0127] Referring to Figure 5 , according to the memory optimization request, the request memory size is checked, and if the request memory size is insufficient, a rejection request is sent as the response notification to the DPU; a host-side memory of a corresponding size is reserved, and if the reservation fails, a rejection request is sent as the response notification to the DPU, and if the reservation is successful, a success response and the address range of the successfully reserved host-side memory are sent as the response notification to the DPU, wherein the memory optimization request includes a request reserved memory space size, and the control channel includes a PCIe link. Specifically, when the DPU needs additional memory resources, it first sends a memory optimization request to the host, which contains the required memory size. After receiving the request, the host-side checks whether it has enough available memory. If the memory is sufficient and the request is legal, the host will reserve a memory of a corresponding size and notify the DPU of the memory address range reserved successfully through a response notification; otherwise, the host will return a notification of a rejection request.

[0128] Once the host-side reserved memory information is successfully obtained, the DPU will construct a virtual memory exchange device, so that the DPU can efficiently exchange data with the host's DDR memory using the DMA engine. In this way, when the DPU encounters a memory shortage situation, it can swap out inactive data pages to the host-side reserved memory space instead of the traditional disk Swap partition, thereby greatly reducing the IO delay and improving the performance.

[0129] In addition, when this part of the additional memory is no longer needed or a user initiates a shutdown request, the DPU will first confirm whether all data has been safely written back, then notify the host to release the previously reserved memory, and unregister the virtual memory exchange device from the system, completing the resource cleaning work. This method not only expands the available memory of the DPU, but also effectively improves the stability and efficiency of the system.

[0130] Figure 7 An apparatus diagram is shown for optimizing DPU-side memory according to an example embodiment.

[0131] Referring toFigure 7 , the device for optimizing DPU side memory is shown, comprising: a request module, a response acceptance module, a virtual device construction module, a memory swap-out module, and a memory swap-in module.

[0132] According to some embodiments, the request module is configured to send a memory optimization request to the host through a control channel. The request module is responsible for sending a memory optimization request to the host, including the size of the memory that needs to be reserved, communicating with the host through a control channel such as PCIe, and starting the memory expansion process.

[0133] According to some embodiments, the response acceptance module is configured to receive a response notification from the host, the response notification including the address range of the successfully reserved host side memory. The response acceptance module receives the response notification returned by the host, judges whether the request is successful, and if so, obtains the address range of the host side reserved memory to provide a basis for subsequent operations.

[0134] According to some embodiments, the virtual device construction module is configured to construct a virtual memory swap device using the successfully reserved host side memory according to the response notification, so that the virtual memory swap device is used to implement the operation of writing DPU side memory data to the memory corresponding to the address range of the successfully reserved host side memory through a DMA engine. The virtual device construction module creates a virtual memory swap device on the DPU side according to the address information returned by the host, and associates it with the reserved host memory, so that the device can read and write data through the DMA engine.

[0135] According to some embodiments, the memory swap-out module is configured to swap out the DPU side memory to the virtual memory swap device when the kernel triggers the low watermark threshold. The memory swap-out module writes the inactive memory pages to the virtual memory swap device through DMA when the DPU local memory is tight and the kernel triggers the low watermark threshold, releasing the local memory resources.

[0136] According to some embodiments, the memory swap-in module is configured to swap in the corresponding memory of the successfully reserved host side memory address range to the virtual memory swap device DPU side memory when the memory in the virtual memory swap device needs to be accessed. The memory swap-in module reads data from the host side reserved memory through DMA and restores it to the DPU local memory when the program accesses the swapped out memory page, ensuring the normal operation of the program.

[0137] Figure 8 A schematic diagram of a device for optimizing DPU side memory according to another example embodiment is shown.

[0138] Referring to Figure 8, the device for optimizing DPU side memory is shown in the figure, comprising: request receiving module, memory reservation module, memory swap response module, memory swap response module.

[0139] According to some embodiments, the request receiving module is used to receive memory optimization requests from the DPU through the control channel. The request receiving module is used to receive memory optimization requests sent from the DPU side through the control channel (such as PCIe link), which contains configuration information such as the size of the required reserved host side memory, as the basis for triggering the host to allocate memory resources.

[0140] According to some embodiments, the memory reservation module is used to reserve the corresponding size of the host side memory according to the memory optimization request, and send a response notification to the DPU, which includes the address range of the successfully reserved host side memory. The memory reservation module is used to check the legality of the request and the available resources of the system according to the received memory optimization request, and reserve the specified size of physical memory on the host side; after successful reservation, a success response notification containing the start address and size of the memory area is generated and sent to the DPU side through the control channel, which is used to build a virtual memory exchange device; if the reservation fails, a response rejecting the request is returned to ensure the safety and controllability of memory allocation.

[0141] According to some embodiments, the memory swap response module is used to receive and store the DPU swapped DPU side memory data in the successfully reserved host side memory when the DPU swaps the DPU side memory to the virtual memory exchange device, wherein the virtual memory exchange device is used to realize the operation of writing the DPU side memory data to the successfully reserved host side memory through the DMA engine. The memory swap response module is used to cooperate with the work of the virtual memory exchange device when the DPU performs memory swap operation, receive the memory data transmitted from the DPU side through the DMA engine, and write it into the successfully reserved and locked physical memory area on the host side for storage. This module ensures that the reserved memory as a remote exchange space can reliably receive and save the data swapped by the DPU, and supports efficient and low-latency cross-device memory expansion.

[0142] According to some embodiments, the memory swap-in response module is configured to send the DPU-side memory data stored in the successfully reserved host-side memory to the DPU-side memory when the DPU needs to access the memory in the virtual memory swap device. The memory swap-in response module is configured to read the corresponding data from the successfully reserved host-side memory region through the DMA engine when the DPU needs to access the memory data swapped out to the virtual memory swap device, and send the data back to the DPU-side memory through the control channel (such as the PCIe link), thereby completing the swap-in operation of the data. The module ensures that the DPU can accurately and efficiently recover the access to the swapped-out memory content, thereby guaranteeing business continuity and system performance.

[0143] Figure 9 A schematic diagram of a method for optimizing DPU-side and host-side memory recycling according to an example embodiment is shown.

[0144] Referring to Figure 9 In the start-up phase, the host-side starts the host-side memory management module and begins to listen to messages from the DPU-side. The DPU-side starts the DPU-side memory management module, and the user initiates the memory optimization option.

[0145] The DPU-side sends a function start request to the host-side. After receiving the request, the host-side checks whether the memory size of the request is abnormal. If it is abnormal, the front-end request is rejected; otherwise, the host-side attempts to reserve memory of the corresponding size. If the host-side successfully reserves the memory, the DPU-side is notified of the success of the reservation, and the host-side memory address range reserved is informed; if the reservation fails, the front-end request is rejected. The DPU-side constructs a virtual memory swap device (Swapinfo) structure according to the response of the host-side, and registers the virtual memory swap device with the kernel Swap module.

[0146] When the memory low-water line threshold is triggered, the inactive data pages in the DPU-side memory are swapped out to the virtual memory swap device, and the data is written to the reserved host-side DDR memory through the DMA engine.

[0147] When the swapped-out memory needs to be accessed, the DPU-side triggers the memory swap-in operation to read the data from the host-side memory back to the DPU-side memory.

[0148] Finally, when the user initiates the optimization scheme shutdown request, the resources are cleaned up. The DPU-side receives the user-initiated optimization scheme shutdown request, and checks whether there is data that has not been written back in the Swapinfo. If there is data that has not been written back, the shutdown operation is prohibited; otherwise, the host-side is sent a memory release notification. After receiving the memory release notification, the host-side releases the previously reserved physical memory, and ends the entire process.

[0149] Through the above steps, the scheme realizes effective management and optimization of the DPU side memory, uses the memory space reserved by the host side as a virtual switching device, improves the memory utilization and system performance. At the same time, through the listening and checking mechanism, the safety and consistency of the data are ensured.

[0150] According to some embodiments, in the AI computing cluster scenario, the memory pressure of the DPU side is very large when the business is intensive, or in the complex business scenario and the full business case. The traditional scheme has a large IO delay after triggering the memory swap. In the design scheme of the present application, the memory swap process is optimized, the system DMA of the DPU side is used to swap the memory to the host memory through the system DMA, and the overall delay is several times higher than that of the traditional scheme, which greatly relieves the performance consumption of the large IO delay caused by the large memory pressure to the business.

[0151] According to some embodiments, the technical scheme of the present application can also be used in the design of the DPU chip, which includes a processor and a memory, and the memory stores a computer program, and the processor runs the computer program stored in the memory to realize the method as claimed in any one of the above, to reduce the IO delay of memory exchange, avoid business jitter caused by disk writing, and ensure the stable operation of the key task on the DPU.

[0152] Figure 10 An AI cluster acceleration card according to an example embodiment is shown.

[0153] According to some embodiments, referring to Figure 10 , the technical scheme of the present application can also be applied to the design of the AI cluster acceleration card, which includes a DPU chip 200 and a plurality of interfaces 310 and a GPU 320 according to the above-mentioned DPU chip, and the DPU chip 200 communicates externally through the plurality of interfaces 310.

[0154] According to some embodiments, the design scheme of the present application optimizes the DPU side OS memory swap process, uses the system DMA of the DPU side to swap the memory to the DDR of the front-end host through the system DMA, and the overall delay is several times higher than that of the traditional way of writing to the local disk, which greatly relieves the phenomenon of business jitter caused by large IO delay under large memory pressure.

[0155] According to some embodiments, the technical solution of the present application realizes cross-device memory cooperative management by taking the host (HOST) side memory as the remote exchange space of the DPU, significantly improving the memory use efficiency and performance of the DPU system. When the DPU local memory is tight, instead of relying on the slower SATA disk for data swap-out, the inactive memory page is directly migrated to the host's DDR memory through the high-speed PCIe link and the DMA engine. This way greatly reduces the IO delay of memory exchange, avoids the business jitter caused by disk writing, and guarantees the stable operation of the key tasks on the DPU.

[0156] Secondly, the technical solution of the present application effectively expands the available memory capacity of the DPU. With the host's usually larger memory resources, the DPU can handle larger-scale data and more complex tasks, breaking through the limitation of its own physical memory. This elastic memory expansion mechanism is particularly suitable for DPU applications in high-throughput scenarios such as networks and storage, improving the overall carrying capacity of the system. In addition, the technical solution of the present application has good compatibility and manageability. By building a virtual memory exchange device and registering it to the kernel Swap module, it is seamlessly integrated with the existing Linux memory management mechanism without modifying the upper-layer application or the operating system core logic. At the same time, it supports on-demand application, dynamic release, and safe shutdown mechanisms to ensure controllable resource use and reliable data security. Without increasing hardware costs, the technical solution of the present application makes full use of the existing system resources to realize a high-performance, low-latency, and scalable DPU-side memory optimization solution.

[0157] Figure 11 A block diagram of a cloud computing device according to an example embodiment is shown.

[0158] As shown in Figure 11 , the cloud computing device 30 includes a processor 12 and a memory 14. The cloud computing device 30 can also include a bus 22, a network interface 16, and an I / O interface 18. The processor 12, the memory 14, the network interface 16, and the I / O interface 18 can communicate with each other through the bus 22.

[0159] The processor 12 can include one or more general-purpose CPUs (Central Processing Units), microprocessors, or application-specific integrated circuits, etc., for executing relevant program instructions. According to some embodiments, the cloud computing device 30 can also include a high-performance display adapter (GPU) 20 for accelerating the processor 12.

[0160] Memory 14 may include machine-readable media in the form of volatile memory, such as random access memory (RAM), read-only memory (ROM), and / or cache memory. Memory 14 is used to store one or more programs containing instructions and data. Cloud computing device 30 may also communicate with one or more networks via a network interface (such as a DPU smart network card) 16. The DPU smart network card is used to process data or conduct external communication, and the central processing unit is used to process data scheduled by the DPU smart network card. The DPU smart network card includes a network chip. The network chip stores a computer program, which, when executed, implements the method according to embodiments of the present invention. Bus 22 may include an address bus, a data bus, a control bus, etc. Bus 22 provides a path for exchanging information between components.

[0161] The cloud computing device 30 can also communicate with one or more networks via network interface 16 (e.g., DPU network interface, AI cluster acceleration card).

[0162] Bus 22 can include address bus, data bus, control bus, etc. Bus 22 provides a path for exchanging information between components.

[0163] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), network storage devices, cloud storage devices, or any type of medium or device suitable for storing instructions and / or data.

[0164] This application also provides a computer program product including a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments.

[0165] Those skilled in the art will clearly understand that the technical solutions of this application can be implemented using software and / or hardware. In this specification, "unit" and "module" refer to software and / or hardware capable of independently performing or cooperating with other components to perform a specific function, where the hardware may be, for example, a field-programmable gate array (FPGA), integrated circuit, etc.

[0166] It should be noted that, for the foregoing method embodiments, the purposes of brief description, they are all described as a series of action combinations, but those skilled in the art should know that the present application is not limited to the order of the actions described, because according to the present application, certain steps can be performed in other orders or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0167] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0168] In several embodiments provided in the present application, it should be understood that the disclosed device can be implemented by other means. For example, the device embodiments described above are only schematic, and the division of units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some service interface, device or unit, which can be electrical or other forms.

[0169] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0170] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0171] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions to make a computer device (which can be a personal computer, a server or a network device, etc.) execute all or part of the steps of the embodiments of the present application.

[0172] In the above-described embodiments, the description of each embodiment focuses on different aspects, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0173] The exemplary embodiments of this application are specifically shown and described above. It is to be understood that this application is not limited to the detailed construction, arrangements or implementation methods described herein; on the contrary, this application is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A method for optimizing DPU-side memory, characterized in that, The method is used for DPU and includes: Send a memory optimization request to the host via the control channel; Receive a response notification from the host, the response notification including: the address range of host-side memory successfully reserved; According to the response notification, a virtual memory swapping device is constructed using the successfully reserved host-side memory. The virtual memory swapping device is used to write DPU-side memory data to the successfully reserved host-side memory through a DMA engine. When the memory low watermark threshold is triggered, the DPU-side memory is swapped out to the virtual memory swapping device; When it is necessary to access memory in the virtual memory swapping device, the memory corresponding to the address range of the successfully reserved host-side memory is swapped into the DPU-side memory. The method of constructing a virtual memory swapping device using the successfully reserved host-side memory, according to the response notification, includes: The virtual memory swapping device structure information is constructed, including a swap mapping table and multiple swap slots. The virtual memory swapping device is divided into these multiple swap slots, and the swap mapping table manages the entries of these multiple swap slots. Register the virtual memory swapping device with the memory swapping module of the operating system kernel on the DPU side.

2. The method according to claim 1, characterized in that, Send memory optimization requests to the host via the control channel, including: Get user memory optimization options; Based on the user memory optimization options, a memory optimization request is sent to the host via the control channel. The memory optimization request includes the amount of memory space to be reserved, and the control channel includes a PCIe link.

3. The method according to claim 1, characterized in that, Swapping DPU-side memory to the virtual memory swapping device includes: Based on the virtual memory swapping device structure information, the DMA engine swaps out the DPU-side memory to the virtual memory swapping device, thereby writing the DPU-side memory to the memory corresponding to the address range of the successfully reserved host-side memory; Swap the memory corresponding to the address range of the successfully reserved host-side memory into the DPU-side memory, including: Based on the virtual memory swapping device structure information, the DMA engine swaps the memory corresponding to the successfully reserved host-side memory address range into the virtual memory swapping device.

4. The method according to claim 1, characterized in that, The method further includes: Get the memory optimization disable request from the user; Based on the memory optimization shutdown request, check if there is any data in the current swap mapping table that has not yet been written back. If not, send a memory release notification to the host to release the memory corresponding to the address range of the successfully reserved host-side memory, and deregister and remove the virtual memory swap device from the operating system kernel swap module on the DPU side to complete resource cleanup.

5. A method for optimizing DPU-side memory, characterized in that, The method is used for a host and includes: Receive memory optimization requests from the DPU via the control channel; Based on the memory optimization request, reserve a corresponding amount of host-side memory and send a response notification to the DPU, the response notification including the address range of the successfully reserved host-side memory; When the DPU swaps out DPU-side memory to the virtual memory swapping device, the DPU-side memory data swapped out by the DPU is received and stored in the successfully reserved host-side memory. The virtual memory swapping device is used to implement the operation of writing the DPU-side memory data to the successfully reserved host-side memory through the DMA engine. When the DPU needs to access memory in the virtual memory swapping device, the DPU-side memory data stored in the successfully reserved host-side memory is sent to the DPU-side memory. The virtual memory swapping device is constructed based on virtual memory swapping device structure-related information, which includes a swapping mapping table and multiple swapping slots. The virtual memory swapping device is divided into the multiple swapping slots, and the swapping mapping table is used to manage the entries of the multiple swapping slots.

6. The method according to claim 5, characterized in that, The method further includes: Receive a memory release notification from the DPU, thereby releasing the successfully reserved host-side memory.

7. The method according to claim 5, characterized in that, Based on the memory optimization request, reserve a corresponding amount of host-side memory and send a response notification to the DPU, including: Based on the memory optimization request, check the requested memory size. If the requested memory size is insufficient, send a rejection request as the response notification to the DPU. Reserve a corresponding amount of host-side memory. If reservation fails, a rejection request is sent as a response notification to the DPU. If reservation is successful, a success response and the address range of the successfully reserved host-side memory are sent as a response notification to the DPU. The memory optimization request includes the amount of memory space to be reserved, and the control channel includes a PCIe link.

8. An apparatus for optimizing DPU-side memory, characterized in that, include: The request module is used to send memory optimization requests to the host via the control channel; A response receiving module is used to receive a response notification from the host, the response notification including: the address range of successfully reserved host-side memory; The virtual device construction module is used to construct a virtual memory swapping device using the successfully reserved host-side memory according to the response notification, so that the virtual memory swapping device can be used to implement the operation of writing DPU-side memory data to the memory corresponding to the address range in the successfully reserved host-side memory through the DMA engine; The memory swap-out module is used to swap out DPU-side memory to the virtual memory swapping device when the kernel triggers the memory low watermark threshold. The memory swapping module is used to swap the memory corresponding to the address range of the successfully reserved host-side memory into the DPU-side memory of the virtual memory swapping device when it is necessary to access the memory in the virtual memory swapping device. The virtual device construction module is configured as follows: The virtual memory swapping device structure information is constructed, including a swap mapping table and multiple swap slots. The virtual memory swapping device is divided into these multiple swap slots, and the swap mapping table manages the entries of these multiple swap slots. Register the virtual memory swapping device with the memory swapping module of the operating system kernel on the DPU side.

9. An apparatus for optimizing DPU-side memory, characterized in that, include: The request receiving module is used to receive memory optimization requests from the DPU through the control channel; The memory reservation module is used to reserve a corresponding amount of host-side memory according to the memory optimization request, and send a response notification to the DPU. The response notification includes the address range of the successfully reserved host-side memory. The memory swap-out response module is used to receive and store the swapped-out DPU-side memory data in the successfully reserved host-side memory when the DPU swaps out DPU-side memory to the virtual memory swapping device. The virtual memory swapping device is used to implement the operation of writing the DPU-side memory data to the successfully reserved host-side memory through the DMA engine. The memory swap-in response module is used to send the DPU-side memory data stored in the successfully reserved host-side memory to the DPU-side memory when the DPU needs to access memory in the virtual memory swapping device. The virtual memory swapping device is constructed based on virtual memory swapping device structure-related information, which includes a swapping mapping table and multiple swapping slots. The virtual memory swapping device is divided into the multiple swapping slots, and the swapping mapping table is used to manage the entries of the multiple swapping slots.

10. A DPU chip, comprising a processor and a memory, characterized in that, A computer program is stored in the memory, and when the processor runs the computer program stored in the memory, it implements the method as described in any one of claims 1 to 4.

11. An AI cluster acceleration card, characterized in that, The AI ​​cluster acceleration card includes: the DPU chip and multiple interfaces as described in claim 10, and a GPU, wherein the DPU chip communicates externally through the multiple interfaces.

12. A cloud computing host, characterized in that, include: The AI ​​cluster acceleration card, central processing unit, and memory as described in claim 11, wherein the AI ​​cluster acceleration card is used for data acceleration, network optimization, and / or storage management, the central processing unit is used to process data scheduled from the AI ​​cluster acceleration card, and the memory stores a computer program, and when the processor runs the computer program stored in the memory, it implements the method as described in any one of claims 5 to 7.

Images