Storage optimization method, device, equipment and storage medium

By modifying the memory manager configuration, the registered memory used by the remote direct memory access memory pool was changed to the storage performance development kit memory, and a judgment and copy-free process was added to the non-volatile memory interface device, which solved the performance problem caused by data copying in the SPDK NVMe driver and improved system performance.

CN119807092BActive Publication Date: 2026-04-10LANGCHAO ELECTRONIC INFORMATION IND CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
LANGCHAO ELECTRONIC INFORMATION IND CO LTD
Filing Date
2024-12-31
Publication Date
2026-04-10

AI Technical Summary

Technical Problem

When using the SPDK NVMe driver, data stored in ordinary memory needs to be copied from the NVMe Device to the SPDK memory before it is submitted to the SPDK NVMe driver for use, which wastes system resources and affects performance.

Method used

By obtaining the initial configuration of the memory manager, the registered memory used by the remote direct memory access memory pool is modified to the storage performance development kit memory, and a judgment and copy-free process is added to the non-volatile memory interface device to directly submit the data of the storage performance development kit memory to the SPDK NVMe driver.

Benefits of technology

It reduces data copying processes and the number of interrupts and context switches, thereby lowering CPU utilization, saving system resources, and improving performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119807092B_ABST
    Figure CN119807092B_ABST
Patent Text Reader

Abstract

Embodiments of the present application provide a storage optimization method, device, equipment and storage medium, comprising: obtaining an initial configuration of a memory manager, the memory manager being used for managing a remote direct memory access memory pool, modifying the initial configuration to modify registered memory used by the remote direct memory access memory pool into storage performance development kit memory, obtaining storage data of the remote direct memory access memory pool, transmitting the storage data to a plurality of preset buffers of a non-volatile memory interface device, obtaining memory data including the storage data in the plurality of preset buffers, and judging the source of the memory data in any preset buffer, if the memory data in the preset buffer is from the storage performance development kit memory, submitting the memory data in the preset buffer to a storage performance development kit non-volatile memory interface driver, and the embodiments of the present application improve system performance.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a storage optimization method and device, equipment and storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence, big data and cloud computing technologies, the demand for computing power and storage optimization is growing rapidly, and the importance of high-performance computing and large-scale storage optimization systems is increasingly prominent. Due to its inherent concurrency and latency problems, traditional disk-based storage systems have been difficult to meet the needs of modern applications. Therefore, new storage media such as NAND FLASH (Not AND Flash Memory) and storage interfaces such as NVMe (Non-Volatile Memory Express) have gradually become mainstream. As a high-performance interface designed for SSD (Solid State Drives), NVMe significantly improves the performance of storage systems by supporting multiple I / O queues and optimizing flash chip-level parallelism, and has become one of the core technologies of high-performance storage systems.

[0003] However, the storage system implemented based on the user-mode NVMe driver (such as the NVMe driver provided by SPDK (Storage Performance Development Kit)) avoids excessive context switching overhead and reduces resource contention during I / O through an event framework, but because the SPDK memory used by the user-mode driver is not unified with the normal memory of the local storage system, when using the SPDK NVMe driver, the data stored on the normal memory needs to be copied to the SPDK memory in the NVME Device first, and then the data is submitted to the SPDK NVMe driver for use, which wastes system resources and affects performance. SUMMARY

[0004] The purpose of the embodiment of the present application is to provide a storage optimization method, device, equipment and storage medium, which solves the problem that when using the SPDK NVMe driver, the data stored on the normal memory needs to be copied to the SPDK memory in the NVME Device first, and then the data is submitted to the SPDK NVMe driver for use, which wastes system resources and affects performance. The specific technical solutions are as follows:

[0005] In the first aspect of the present application, a storage optimization method is first provided, which comprises:

[0006] Obtaining an initial configuration of a memory manager, the memory manager being used to manage a remote direct memory access memory pool, the remote direct memory access memory pool using a registered memory;

[0007] Modifying the initial configuration to modify the registered memory used by the remote direct memory access memory pool to a storage performance development kit memory;

[0008] Obtaining storage data of the remote direct memory access memory pool;

[0009] Transferring the storage data to a plurality of preset buffers of a non-volatile memory interface device;

[0010] Obtaining memory data including the storage data in the plurality of preset buffers and determining a source of the memory data in any of the preset buffers;

[0011] If the memory data in the preset buffer is from the storage performance development kit memory, submitting the memory data in the preset buffer to a storage performance development kit non-volatile memory interface driver.

[0012] Optionally, the transferring the storage data to the plurality of preset buffers of the non-volatile memory interface device comprises:

[0013] Obtaining a data amount of the storage data;

[0014] If the data amount meets a preset condition, transferring the storage data to the plurality of preset buffers of the non-volatile memory interface device;

[0015] If the data amount does not meet the preset condition, obtaining target data from a general memory;

[0016] Transferring the target data and the storage data to the plurality of preset buffers of the non-volatile memory interface device.

[0017] Optionally, after the obtaining the memory data including the storage data in the plurality of preset buffers and determining the source of the memory data in any of the preset buffers, the method further comprises:

[0018] Based on the plurality of preset buffers in the non-volatile memory interface device, generating an initial buffer linked list of the non-volatile memory interface device;

[0019] If the memory data in any of the preset buffers is from the general memory, setting a memory block allocator in the storage performance development kit;

[0020] allocating a plurality of first target memory blocks of preset sizes from a memory of a storage performance development kit by the memory block allocator;

[0021] copying memory data in the preset buffer to the first target memory block;

[0022] if any of the memory data is from the memory of the storage performance development kit, determining a preset buffer corresponding to the memory data as a target buffer;

[0023] generating a new buffer linked list based on the target buffer and the first target memory block;

[0024] sending the new buffer linked list to the storage performance development kit, wherein an input / output interface function is set in the storage performance development kit;

[0025] submitting corresponding memory data in the new buffer linked list to a non-volatile memory express interface driver of the storage performance development kit by the input / output interface function.

[0026] Optionally, if the data amount does not meet the preset condition, obtaining target data from a general memory, comprising:

[0027] if the data amount is less than a preset value or is not an integer multiple of the preset value, taking a modulus of the data amount and the preset value to obtain a modulus data amount;

[0028] obtaining a target difference value by using the modulus data amount and the preset value;

[0029] obtaining target data from the general memory by using the target difference value.

[0030] Optionally, the modifying the registered memory used by the remote direct memory access memory pool to the memory of the storage performance development kit by modifying the initial configuration, comprising:

[0031] pre-dividing a second target memory block to be allocated from the memory of the storage performance development kit;

[0032] setting a memory allocation pointer in the memory manager;

[0033] obtaining a memory address of the second target memory block by using the memory allocation pointer;

[0034] controlling the remote direct memory access memory pool to use the second target memory block of the storage performance development kit by using the memory address.

[0035] Optionally, after the memory allocation pointer is set in the memory manager, further comprising:

[0036] setting a memory release pointer in the memory manager;

[0037] in a case where it is determined that the second target memory block ends use, acquiring a memory address of the second target memory block through the memory release pointer;

[0038] releasing the second target memory block through the memory address.

[0039] Optionally, after the initial configuration of the memory manager is acquired, further comprising:

[0040] acquiring a component of a buffer list, the component of the buffer list managing buffers in a remote direct memory access memory pool and a non-volatile memory interface device, the buffers including preset buffers of the non-volatile memory interface device;

[0041] adding a Boolean flag component in the component of the buffer list;

[0042] when data of the buffer is from a storage performance development kit memory, setting a Boolean flag in the buffer through the Boolean flag component, the Boolean flag being set as a first Boolean value;

[0043] when data of the buffer is from a common memory, setting a Boolean flag in the buffer through the Boolean flag component, the Boolean flag being set as a second Boolean value.

[0044] In a second aspect of the embodiment of the present application, a storage optimization device is further provided, comprising:

[0045] a first acquiring module, configured to acquire an initial configuration of a memory manager, the memory manager being used to manage a remote direct memory access memory pool, the remote direct memory access memory pool using a registered memory;

[0046] a configuration modifying module, configured to modify the registered memory used by the remote direct memory access memory pool to a storage performance development kit memory by modifying the initial configuration;

[0047] a second acquiring module, configured to acquire storage data of the remote direct memory access memory pool;

[0048] a transmitting module, configured to transmit the storage data to a plurality of preset buffers of a non-volatile memory interface device;

[0049] a third acquiring module, configured to acquire memory data including the storage data in the plurality of preset buffers, and to determine a source of the memory data in any of the preset buffers;

[0050] The data submission module is configured to submit the memory data in the preset buffer to a storage performance development kit non-volatile memory interface driver if the memory data in the preset buffer is from the storage performance development kit memory.

[0051] In a third aspect of the embodiments of the present application, a communication device is further provided, which comprises a transceiver, a memory, a processor and a program stored in the memory and executable on the processor.

[0052] The processor is configured to read the program in the memory to implement the storage optimization method.

[0053] In a fourth aspect of the embodiments of the present application, a computer readable storage medium is further provided, which stores instructions, and when the instructions are executed on a computer, the computer is caused to execute the storage optimization method.

[0054] The storage optimization method provided by the embodiments of the present application acquires an initial configuration of a memory manager, the memory manager is used to manage a remote direct memory access memory pool, the remote direct memory access memory pool uses registered memory, the registered memory used by the remote direct memory access memory pool is modified to storage performance development kit memory by modifying the initial configuration; data is stored on the storage performance development kit memory when passing through the remote direct memory access memory pool, storage data of the remote direct memory access memory pool is acquired, the storage data is transmitted to a plurality of preset buffers of a non-volatile memory interface device, memory data including the storage data in the plurality of preset buffers is acquired, and the source of the memory data in any preset buffer is judged; if the memory data in the preset buffer is from the storage performance development kit memory, the memory data in the preset buffer is submitted to a storage performance development kit non-volatile memory interface driver, the copy process is reduced, and then the number of interrupts, the number of context switches and the CPU usage are reduced. In the embodiments of the present application, the memory used by the remote direct memory access memory pool is modified, so that the remote direct memory access memory pool uses the memory from the storage performance development kit, and the source of the memory data of the non-volatile memory interface device is judged, so that the memory from the storage performance development kit can be directly submitted to the storage performance development kit non-volatile memory interface driver, the copy operation is reduced, the CPU usage is reduced, the system resources are saved, and the performance is improved. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the embodiments or the related art description will be briefly introduced.

[0056] Figure 1A use SPDK NVMe driver use schematic diagram of the related art;

[0057] Figure 2 Another use SPDK NVMe driver use schematic diagram of the related art;

[0058] Figure 3 A related flowchart in the remote direct memory access memory pool + number control separation scenario;

[0059] Figure 4 An improved flowchart in the remote direct memory access memory pool + number control separation scenario;

[0060] Figure 5 A step flowchart of a storage optimization method provided by the embodiment of the application;

[0061] Figure 6 It is Figure 5 A schematic diagram of a memory manager modified in a storage optimization method provided by the embodiment of the application is shown in the figure;

[0062] Figure 7 It is Figure 5 A flowchart of memory allocation by a memory allocation pointer in a storage optimization method provided by the embodiment of the application is shown in the figure;

[0063] Figure 8 It is Figure 5 A flowchart of memory release by a memory release pointer in a storage optimization method provided by the embodiment of the application is shown in the figure;

[0064] Figure 9 It is Figure 5 A component schematic diagram of a modified buffer chain table in a storage optimization method provided by the embodiment of the application is shown in the figure;

[0065] Figure 10 It is Figure 5 A memory source judgment flowchart of a constructor in a storage optimization method provided by the embodiment of the application is shown in the figure;

[0066] Figure 11 It is Figure 5 A schematic diagram of a modified non-volatile memory interface device in a storage optimization method provided by the embodiment of the application is shown in the figure;

[0067] Figure 12 It is Figure 5 A copy-free flowchart of a non-volatile memory interface device added in a storage optimization method provided by the embodiment of the application is shown in the figure;

[0068] Figure 13 It is a structural schematic diagram of a fault processing device provided by the embodiment of the application;

[0069] Figure 14 Figure 1 is a structural schematic diagram of a communication device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0070] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below with reference to the drawings. However, those skilled in the art can understand that, in the embodiments of the present application, many technical details are presented in order to make the readers better understand the present application. However, the technical solutions claimed by the present application can be implemented even without these technical details and based on various changes and modifications of the following embodiments. The division of the following embodiments is for the convenience of description, and should not constitute any limitation on the specific implementation of the present application, and the embodiments can be combined and referenced with each other without contradiction.

[0071] Currently, when using a storage performance development kit non-volatile memory interface (SPDK NVMe) driver, only storage performance development kit memory can be used as shown in Figure 1 , and data samples stored on ordinary memory cannot be directly used. For ordinary memory, the data thereon needs to be copied to the storage performance development kit memory in a Ceph storage system block device non-volatile memory interface device (NVMEDevice) first, so that the SPDK NVMe driver can submit the data to the NVMEDevice as shown in Figure 2 . Embodiments of the present application are applied to a remote direct memory access memory pool + number control separation scenario. The related process is shown in Figure 3 , the remote direct memory access memory pool stores data using registered memory, the RDMA registered memory allows a network adapter (RNIC) to directly access the memory without the CPU participating in data transmission, thereby releasing CPU resources for computing tasks. In the remote direct memory access memory pool + number control separation scenario, ordinary memory will also be used. In a non-critical path (such as log recording, metadata management, control information transmission), the performance overhead of ordinary memory can be accepted, and there is no need to use registered memory. The registered memory and the ordinary memory are managed by the OSD + BlueStore in the Ceph storage system, and then the data of the registered memory and the data of the ordinary memory are sent to the non-volatile memory interface device (NVMEDevice). In the NVMEDevice, the data using the registered memory and the ordinary memory needs to be copied to the storage performance development kit memory, and then the SPDK NVMe driver uses the data in the storage performance development kit memory. However, a large amount of data is usually stored in the registered memory, so that when copying, the CPU usage rate rises, affecting system performance. Therefore, the above process is improved in embodiments of the present application as shown inFigure 4 The application discloses a storage optimization method, and relates to the technical field of storage systems. Figure 5 The application discloses a storage optimization method, and relates to the technical field of storage systems.

[0072] In step 101, the initial configuration of a memory manager is obtained, the memory manager is used for managing a remote direct memory access memory pool, and the remote direct memory access memory pool uses registered memory.

[0073] The memory manager in the embodiment of the application refers to a Memory Manager, and the memory manager is used for managing a remote direct memory access memory pool. For example, the memory manager can be responsible for allocating a memory block required by a remote direct memory access memory pool RDMA (Remote Direct Memory Access); the memory manager can work cooperatively with a remote direct memory access memory pool driver to ensure that the memory meets registration requirements, and the memory allocated for the remote direct memory access memory pool is registered to RDMA hardware; and the memory manager can use a buddy allocator to manage large blocks of memory in the RDMA memory pool. In the initial configuration of the memory manager, a plurality of allocation algorithms (such as a Buddy Allocator buddy memory allocator, a slab allocator and the like) can be contained, and the RDMA memory pool is managed through the allocation algorithms.

[0074] The remote direct memory access memory pool is a network communication technology, and allows a computer to directly access remote memory without the participation of a CPU. Generally, the remote direct memory access memory pool is used in combination with registered memory to achieve high performance. However, this setting causes a large amount of copying to be performed when the SPDK NVMe driver is used subsequently, and thus influences system performance, so it is necessary to modify the memory used by the remote direct memory access memory pool. In order to achieve this purpose, the memory manager can be modified.

[0075] In step 102, the registered memory used by the remote direct memory access memory pool is modified to storage performance development kit memory through modification of the initial configuration.

[0076] The embodiment of the application modifies the initial configuration of the memory manager (Buddy Allocator), obtains a memory manager containing a new configuration, and an example is as shown in the figure Figure 6 In the new configuration of the memory manager, there is a memory pool BuddyPool implemented based on the Buddy Allocator, which contains the Buddy Allocator, an interface function set_cb() is set in the Buddy Allocator, which is used to provide the spdk_malloc() function address and the spdk_free() function address to the Buddy Allocator after the SPDK environment is initialized, and then a memory allocation pointer malloc_cb* and a memory release pointer free_cb* are added, the malloc_cb* can receive the spdk_malloc() function address, and the free_cb* can receive the spdk_free() function address. The original functions mem_alloc() and mem_free() in the Buddy Allocator are retained. However, the use of mem_alloc() and mem_free() is subject to conditional restrictions.

[0077] The set spdk_malloc() function address can obtain the memory address of the memory of the storage performance development kit, and the memory can be a second target memory block allocated in advance for the remote direct memory access memory pool. By obtaining the memory address, the remote direct memory access memory pool can find the available memory space. The specific steps include:

[0078] Pre-allocate a second target memory block in the memory of the storage performance development kit;

[0079] Set a memory allocation pointer in the memory manager;

[0080] Obtain the memory address of the second target memory block through the memory allocation pointer;

[0081] Control the remote direct memory access memory pool to use the second target memory block of the storage performance development kit through the memory address.

[0082] Through the above settings, the remote direct memory access memory pool can directly use the memory of the storage performance development kit, avoid subsequent copy operations, reduce CPU occupation, and improve system performance.

[0083] Further, the memory allocated for the remote direct memory access memory pool is dynamic, when the remote direct memory access memory pool does not need to use this part of memory, the memory allocation pointer can be set in the memory manager, and then the memory address is obtained again through the memory allocation pointer, the corresponding memory space is found, and the memory space is released, and the specific steps include:

[0084] Setting a memory release pointer in the memory manager;

[0085] In the case of determining that the use of the second target memory block ends, the memory address of the second target memory block is obtained through the memory release pointer;

[0086] The second target memory block is released through the memory address.

[0087] Through the above setting, the storage performance development kit memory can be flexibly allocated and released, and the memory is released after use to avoid memory leakage. Efficient memory management can reduce memory occupation and improve the overall performance and response speed of the system. In addition, through the memory release pointer and the explicit release steps, the logic of memory management is simplified, and the complexity of the code is reduced.

[0088] It should be noted that after the above modification, when the memory manager allocates memory, as shown in Figure 7 , it is first determined whether the memory allocation pointer is empty, if not, the memory is allocated through the memory allocation pointer, at this time the memory belongs to the second target memory block divided from the memory of the storage performance development kit, if it is empty, it indicates that the storage performance development kit has not allocated memory to the remote direct memory access memory pool at this time, and the Buddy Allocator of the memory manager allocates the registered memory through the original process. When the memory manager releases the memory, as shown in Figure 8 , it is first determined whether the memory release pointer is empty, if not, the memory is released through the memory release pointer, if it is empty, the Buddy Allocator of the memory manager releases the memory through the original process.

[0089] Step 103, obtaining the storage data of the remote direct memory access memory pool.

[0090] In the embodiment of the application, after the memory used by the remote direct memory access memory pool is modified, the storage data therein can be obtained. The storage data can be different in type according to different scenes and requirements. For example, in a distributed storage system (such as a distributed file system, an object storage, or a database), the storage data of the RDMA memory pool is usually a file block, object data, or a database record. In high-performance computing

[0091] In RDMA memory pool, the storage data is usually network transmitted data (such as messages, tasks, computing results, etc.) in HPC or distributed computing. In distributed file system or object storage, the storage data of RDMA memory pool is usually metadata such as file index, object metadata, directory structure, etc.

[0092] Step 104, transfer the storage data to several preset buffer zones of the non-volatile memory interface device.

[0093] Non-volatile memory technology can use PCIe to directly connect SSD storage to servers or central processing units (CPUs). With significant performance improvement, NVMe technology has become the preferred data storage / transmission option for gamers, video editors and other users who need higher performance than SAS or SATA HDD. Moreover, PCIe connections are wider and have more bandwidth than SAS or SATA ports. It also improves with each generation, with twice the bandwidth of the previous generation. The bandwidth connection of SAS and SATA is much lower and fixed, so they will not improve over time. Another feature that sets PCIe connections apart is that they are scalable in "lanes", so users can double the bandwidth even in the same generation of products by doubling the number of lanes. NVMe drives can provide higher speed than SAS or SATA drives because they can send and receive NVMe commands faster and provide better throughput. NVMe is widely regarded as a more compatible option than SAS / SATA, and is often updated as key technologies such as AI, ML and cloud computing develop rapidly. NVMe technology can work seamlessly with all modern operating systems, including mobile phones, laptops and game consoles.

[0094] Because the storage performance development kit non-volatile memory interface driver can only directly use the data of the storage performance development kit memory, data that is not in the storage performance development kit memory needs to be converted by the non-volatile memory interface device. The data of the ordinary memory or the registered memory is first stored in the storage performance development kit memory, and then used by the storage performance development kit non-volatile memory interface driver. Therefore, the storage data is transferred to the non-volatile memory interface device.

[0095] In the process of transferring the storage data to the non-volatile memory interface device, the OSD+BlueStore will manage the transferred data. In order to improve the performance of the non-volatile memory interface device (NVMe device), the data will be supplemented from the ordinary memory when the data does not meet the preset conditions. This can ensure the efficiency and alignment of data transmission. The specific steps include:

[0096] an amount of data of the stored data;

[0097] if the amount of data meets a preset condition, the stored data is transmitted to a plurality of preset buffers of a non-volatile memory interface device;

[0098] if the amount of data does not meet the preset condition, target data is obtained from a general memory;

[0099] the target data and the stored data are transmitted to the plurality of preset buffers of the non-volatile memory interface device.

[0100] The preset condition can be equal to a preset value or an integer multiple of the preset value. The preset value is usually aligned with the block size or the buffer size of the non-volatile memory interface device (such as an NVMe device). The general memory is used to supplement the insufficient amount of data to ensure that the size of the data block to be transmitted meets the preset condition. The buffer is an area used by the storage device to temporarily store data.

[0101] In the above manner, the performance of the storage device can be improved, the fragmentation and additional overhead can be reduced, frequent small data block transmission can be avoided, the efficiency of the storage device can be improved, data can be aligned and batch transmitted, the access frequency of the storage device can be reduced, and the performance can be improved.

[0102] After the preset condition is not met, data is obtained from the general memory for padding operation. The specific steps include:

[0103] if the amount of data is less than the preset value or not equal to an integer multiple of the preset value, the amount of data and the preset value are taken modulo to obtain a modulo amount of data;

[0104] a target difference value is obtained through the modulo amount of data and the preset value;

[0105] target data is obtained from the general memory through the target difference value.

[0106] For example, the preset value is 4KB, and the amount of data of the stored data is 10KB. Since 10KB is not an integer multiple of 4KB, it does not meet the preset condition. The modulo amount of data is calculated: 10KB%4KB=2KB, the target difference value is calculated: 4KB-2KB=2KB, 2KB of target data is read from the general memory, 10KB of stored data and 2KB of target data are combined to form a 12KB data block, and the 12KB data block is transmitted to the buffer of the non-volatile memory interface device.

[0107] It should be noted that in the scenario of RDMA memory pool + CNC separation, in addition to obtaining data from the RDMA memory pool or aligning data in the RDMA memory pool from the ordinary memory, in the case of extremely small data volume, only ordinary memory is used without using the RDMA memory pool. The data for this part also needs to be copied, and the specific steps include:

[0108] Obtain the to-be-transmitted data in the storage system and the data volume of the to-be-transmitted data.

[0109] If the data volume of the to-be-transmitted data is less than a preset value, store the to-be-transmitted data through the ordinary memory.

[0110] Copy the to-be-transmitted data to the SPDK memory in the NVM interface device, so as to store the NVM interface driver of the SPDK.

[0111] For extremely small data volume, using the RDMA memory pool may not be able to fully exert its performance advantage, but instead increase additional overhead, so the above setting can reduce the management overhead of the memory pool. At the same time, the above setting can adapt to the needs of different data volumes, both processing extremely small data and efficiently processing large data volume.

[0112] Step 105, obtain the memory data including storage data in a plurality of preset buffer areas, and determine the source of the memory data in any preset buffer area.

[0113] In the embodiment of the application, the concept of BufferList can be used to manage data in the remote direct memory access memory pool and the non-volatile memory interface device (such as NVMe device). BufferList is a data structure used to manage multiple discontinuous memory blocks (Buffers) in the Ceph storage system. It supports efficient data storage, operation and transmission, and is widely used in network transmission, storage systems and data processing scenarios. Through BufferList, Ceph can efficiently manage memory data and improve the performance and flexibility of the system. BufferList is a linked list or array composed of multiple Buffers. Therefore, a plurality of preset buffer areas are set in the non-volatile memory interface device to receive data from the remote direct memory access memory pool and data processed by OSD+BlueStore. The data stored in the preset buffer area is collectively referred to as memory data. The memory data includes memory data from the ordinary memory and memory data from the SPDK memory.

[0114] It should be noted that the data received by RDMA can be passed to the NVM EDevice (BlockDevice) entry through Bufferlist. In the remote direct memory access memory pool and the non-volatile memory interface device NVMEDevice, the memory data in the preset buffer can be judged through the Boolean flag preset in the Bufferlist. The specific steps include:

[0115] Obtaining a component of a buffer list, the component of the buffer list respectively manages the buffer in the remote direct memory access memory pool and the non-volatile memory interface device, and the buffer includes a preset buffer of the non-volatile memory interface device;

[0116] Adding a Boolean flag component in the component of the buffer list;

[0117] When the data of the buffer is from the storage performance development kit memory, setting the Boolean flag in the buffer to the first Boolean value through the Boolean flag component;

[0118] When the data of the buffer is from the ordinary memory, setting the Boolean flag in the buffer to the second Boolean value through the Boolean flag component.

[0119] The first Boolean value can be true, and the second Boolean value can be false. The embodiment of the application adds a judgment flag to the bufferlist to judge whether it is from RDMA, so as to distinguish the data from the storage performance development kit memory and the data from the ordinary memory. For example, Figure 9As shown, the BufferList includes a bufferlist module, a PtrNode module, and a raw module. The BufferList is a linked list or an array composed of multiple buffers, and each buffer internally relies on the buffer::raw to manage its underlying memory data. A Boolean flag from_spdk is added in the buffer::raw to determine whether the raw buffer is from the storage performance development kit memory, that is, to determine whether the memory data of the preset buffer is from the storage performance development kit memory. If yes, the Boolean value is set to true, and if not, the Boolean value is set to false. The buffer::ptr of the BufferList is a core class used to represent a logical data block in the BufferList. The buffer::ptr is an encapsulation of the buffer::raw, which provides an access interface to the actual memory data and supports reference counting and memory sharing. Therefore, the interface function get_raw() for obtaining the raw* is added in the buffer::ptr, which is used by the NVMEDevice module to obtain the instance of the raw to obtain the from_spdk flag.

[0120] Through the above setting, it is convenient to distinguish the high-performance memory (such as the SPDK memory) and the ordinary memory in the subsequent process, so as to optimize the data processing flow and help the system to quickly determine different processing methods for different data. For example, unnecessary operations (such as copying) on the high-performance memory can be avoided, thereby improving the performance.

[0121] In addition, the Boolean value of from_spdk is also set in the memory manager of the remote direct memory access memory pool management. Since the remote direct memory access memory pool uses the storage performance development kit memory, if the buffer data is from the buddy pool in the memory manager, the from_spdk is directly modified to true. For example, Figure 10 As shown, the constructor of the raw_rdma is set in the BuddyPool of the memory manager, and a judgment is added therein. If the chunk is not null, it indicates that the memory is from the buddy memory pool, and at this time, the Boolean value of the Boolean flag from_spdk is modified to true. If it is empty, the Boolean value of the from_spdk remains unchanged, which is the default false.

[0122] The embodiment of the present application also modifies the NVMEDevice, removes the original copy logic, and makes the copy-free buffer list directly submitted to the IO interface function of SPDK. First, the buffer list is used to manage a plurality of preset buffers in the non-volatile memory interface device, so that based on the plurality of preset buffers in the non-volatile memory interface device, an initial buffer linked list of the non-volatile memory interface device is generated, and the memory data in each preset buffer in the initial buffer linked list is judged by the from_spdk flag. If the flag is false, a raw_spdk object based on the original memory management object of SPDK is created and the data is copied to the raw_spdk, the raw_spdk is a plurality of preset size first target memory blocks allocated from the memory of the storage performance development kit by a memory block allocator, and then the raw_spdk (first target memory block) is placed in the new buffer linked list. If the flag is true, the data is directly placed in the new buffer linked list, and the new buffer linked list is submitted to the input / output interface function of the storage performance development kit, and then the corresponding memory data in the new buffer linked list is submitted to the non-volatile memory interface driver of the storage performance development kit through the input / output interface function. The specific steps include:

[0123] Based on a plurality of preset buffers in the non-volatile memory interface device, an initial buffer linked list of the non-volatile memory interface device is generated;

[0124] If the memory data in any preset buffer is from a normal memory, a memory block allocator is set in the storage performance development kit;

[0125] A plurality of preset size first target memory blocks are allocated from the memory of the storage performance development kit by a memory block allocator;

[0126] The memory data in the preset buffer is copied to the first target memory block;

[0127] If any memory data is from the memory of the storage performance development kit, the preset buffer corresponding to the memory data is determined as a target buffer;

[0128] Based on the target buffer and the first target memory block, a new buffer linked list is generated;

[0129] The new buffer linked list is sent to the storage performance development kit, and an input / output interface function is set in the storage performance development kit;

[0130] The corresponding memory data in the new buffer linked list is submitted to the non-volatile memory interface driver of the storage performance development kit through the input / output interface function.

[0131] Wherein, the modification of the non-volatile memory interface device (NVMEDevice) is as shown in the following table: Figure 11 As shown in the modification of the write / aio_write() in the NVMEDevice, the read / aio_read() function is modified, and the copy_read() / aio_copy_read() function, the get_append_read_buffer() function, the slab_allocator function, the raw_spdk function, and the get_aligned_bufferlist() function are added. Through the modification of the write / aio_write(), the original write flow is modified, and the segmentation and copying are removed, and instead, the from_spdk flag of the raw is judged to select the data in the buffer for copying or direct use. Through the modification of the read / aio_read() function, the original read flow is modified, and a plurality of first target memory blocks (raw_spdk) of a preset size are allocated from the memory of the storage performance development kit to read data from the SSD, and the raw_spdk is appended to the tail of the entry bufferlist to generate a new buffer list. The combination of the write / aio_write() modification and the read / aio_read() function is realized through the added copy_read() / aio_copy_read() function, as shown in the following table: Figure 12 As shown in the following table, the memory data in each preset buffer in the initial buffer list is judged by the from_spdk flag. If the flag is false, a plurality of raw memory management objects (raw_spdk objects) of 4K size are allocated and the data is copied to the raw_spdk, and the raw_spdk is a plurality of first target memory blocks of a preset size allocated from the memory of the storage performance development kit by the memory block allocator. Then, the raw_spdk (first target memory block) is placed at the tail of the new buffer list. If the flag is true, the data is directly placed in the new buffer list, and the new buffer list is submitted to the storage performance development kit. The new buffer list is polled for I / O completion processing by the input / output interface function of the storage performance development kit, and then the corresponding memory data in the new buffer list is submitted to the non-volatile memory interface driver of the storage performance development kit by the input / output interface function. After use, the new buffer list is also judged to determine whether it still exists. If it does not exist, the data needs to be copied back to the original buffer list.

[0132] The added raw_spdk function is used to use the memory from SPDK; for accepting the copy of metadata and the write copy and read operation of data not from RDMA. The added slab_allocator function is used to manage the SPDK memory required by raw_spdk, to accelerate the allocation of SPDK memory. The added get_append_read_buffer() function is used to obtain a buffer suitable for the read operation of the NVMe device, and append it to the new buffer list; the added get_aligned_bufferlist() function is used to obtain a memory-aligned buffer list (BufferList), which is usually used to prepare to write into NVMe, to meet the data transmission requirements of the NVMe device.

[0133] Through the copy-free operation of the memory data in the preset buffer from the storage performance development kit memory, the number of interrupts (SYSTEMIN) is reduced, the number of context switches is reduced, and the CPU usage and the total bandwidth of the PCM memory are obviously improved.

[0134] In addition, the new SGL positioning function is also used in NVMEDevice, and the bufferlist::buffers_t iterator is used to position the SGE; (Note: 1. SGL is a data structure used to describe data transmission operations in NVMe commands. It is used to specify the physical memory address and length of data, and how to gather multiple scattered memory segments together for data transmission; 2. SGE is a single entry in SGL; 3. In SPDK, the SGL positioning function is provided by the upper layer, and SPDK is responsible for checking and constructing the SGL in the NVMe specification).

[0135] In step 106, if the memory data in the preset buffer is from the storage performance development kit memory, the memory data in the preset buffer is submitted to the storage performance development kit non-volatile memory interface driver.

[0136] The storage performance development kit non-volatile memory interface driver in the embodiment of the application can only directly use the data of the storage performance development kit memory, so the memory data in the preset buffer from the storage performance development kit memory can be used, and the memory data in the preset buffer not from the storage performance development kit memory needs to be copied to the storage performance development kit memory first. Through the copy-free operation of the memory data in the preset buffer from the storage performance development kit memory, the number of interrupts (SYSTEMIN) is reduced, the number of context switches is reduced, and the CPU usage and the total bandwidth of the PCM memory are obviously improved.

[0137] The driver generally refers to a device driver, which is a special program enabling a computer and a device to communicate with each other. The device driver is an interface of hardware, and an operating system can control the operation of a hardware device only through the interface.

[0138] The storage optimization method provided by the embodiment of the application comprises the following steps: obtaining an initial configuration of a memory manager, the memory manager being used for managing a remote direct memory access memory pool, the remote direct memory access memory pool using registered memory; modifying the initial configuration to modify the registered memory used by the remote direct memory access memory pool into storage performance development kit memory; storing data in the storage performance development kit memory when the data passes through the remote direct memory access memory pool; obtaining storage data of the remote direct memory access memory pool; transmitting the storage data to a plurality of preset buffers of a non-volatile memory interface device; obtaining memory data including the storage data in the plurality of preset buffers; and judging the source of the memory data in any preset buffer. If the memory data in the preset buffer is from the storage performance development kit memory, the memory data in the preset buffer is submitted to a storage performance development kit non-volatile memory interface driver, thereby reducing the copy process and the number of interrupts, context switches and CPU usage. In the embodiment of the application, the memory used by the remote direct memory access memory pool is modified to be from the storage performance development kit, and the source of the memory data of the non-volatile memory interface device is judged, so that the memory from the storage performance development kit can be directly submitted to the storage performance development kit non-volatile memory interface driver, the copy operation is reduced, the CPU usage is reduced, system resources are saved, and performance is improved.

[0139] Reference Figure 13 , a structure schematic diagram of a storage optimization device provided by an embodiment of the application is shown, as Figure 13 shown, the device can comprise:

[0140] The first obtaining module 201 is configured to obtain an initial configuration of a memory manager, the memory manager being used for managing a remote direct memory access memory pool, the remote direct memory access memory pool using registered memory.

[0141] The configuration modification module 202 is configured to modify the initial configuration to modify the registered memory used by the remote direct memory access memory pool into storage performance development kit memory.

[0142] The second obtaining module 203 is configured to obtain storage data of the remote direct memory access memory pool.

[0143] The transmission module 204 is configured to transmit the storage data to a plurality of preset buffers of a non-volatile memory interface device.

[0144] The third obtaining module 205 is configured to obtain memory data in the plurality of preset buffers, and determine the source of the memory data in any preset buffer.

[0145] The data submitting module 206 is configured to submit the memory data in the preset buffer to a storage performance development kit non-volatile memory interface driver if the memory data in the preset buffer is from a storage performance development kit memory.

[0146] Optionally, the transmission module 204 specifically comprises:

[0147] The first obtaining submodule is configured to obtain the data amount of the storage data.

[0148] The first transmission submodule is configured to transmit the storage data to the plurality of preset buffers of the non-volatile memory interface device if the data amount meets the preset condition.

[0149] The second obtaining submodule is configured to obtain target data from the normal memory if the data amount does not meet the preset condition.

[0150] The second transmission submodule is configured to transmit the target data and the storage data to the plurality of preset buffers of the non-volatile memory interface device.

[0151] Optionally, the storage optimization apparatus further comprises:

[0152] The first generating module is configured to generate an initial buffer linked list of the non-volatile memory interface device based on the plurality of preset buffers in the non-volatile memory interface device.

[0153] The first setting module is configured to set a memory block allocator in the storage performance development kit if the memory data in any preset buffer is from the normal memory.

[0154] The memory allocating module is configured to allocate a plurality of first target memory blocks with preset sizes from the memory of the storage performance development kit through the memory block allocator.

[0155] The copying module is configured to copy the memory data in the preset buffer to the first target memory block.

[0156] The determining module is configured to determine the preset buffer corresponding to the memory data as a target buffer if the memory data is from the storage performance development kit memory.

[0157] The second generating module is configured to generate a new buffer linked list based on the target buffer and the first target memory block.

[0158] The sending module is configured to send the new buffer list to a storage performance development kit, and set an input / output interface function in the storage performance development kit.

[0159] The submitting module is configured to submit corresponding memory data in the new buffer list to a non-volatile memory interface driver of the storage performance development kit through the input / output interface function.

[0160] Optionally, the second obtaining sub-module specifically comprises:

[0161] The modulo unit is configured to take modulo of the data quantity and the preset value to obtain a modulo data quantity if the data quantity is less than the preset value or is not an integer multiple of the preset value.

[0162] The first obtaining unit is configured to obtain a target difference value through the modulo data quantity and the preset value.

[0163] The second obtaining unit is configured to obtain target data from the normal memory through the target difference value.

[0164] Optionally, the configuration modifying module 202 specifically comprises:

[0165] The dividing sub-module is configured to divide a second target memory block to be allocated in advance in a memory of a storage performance development kit.

[0166] The first setting sub-module is configured to set a memory allocation pointer in a memory manager.

[0167] The third obtaining sub-module is configured to obtain a memory address of the second target memory block through the memory allocation pointer.

[0168] The control sub-module is configured to control a remote direct memory access memory pool to use the second target memory block of the storage performance development kit through the memory address.

[0169] The second setting sub-module is configured to set a memory release pointer in the memory manager.

[0170] The fourth obtaining sub-module is configured to obtain the memory address of the second target memory block through the memory release pointer in a case where it is determined that the second target memory block ends to be used.

[0171] The memory releasing sub-module is configured to release the second target memory block through the memory address.

[0172] Optionally, the storage optimization apparatus further comprises:

[0173] The fourth obtaining module is configured to obtain a component about a buffer list, the component about the buffer list manages buffers in a remote direct memory access memory pool and a non-volatile memory interface device, and the buffers include preset buffers of the non-volatile memory interface device.

[0174] An adding component module is configured to add a Boolean flag component in the component of the buffer chain table.

[0175] A second setting module is configured to set a Boolean flag with a first Boolean value in the buffer through the Boolean flag component when the data of the buffer is from the storage performance development kit memory.

[0176] A third setting module is configured to set a Boolean flag with a second Boolean value in the buffer through the Boolean flag component when the data of the buffer is from the normal memory.

[0177] The storage optimization method provided by the embodiment of the application includes the following steps: obtaining an initial configuration of a memory manager, the memory manager being configured to manage a remote direct memory access memory pool, the remote direct memory access memory pool using registered memory; modifying the initial configuration to modify the registered memory used by the remote direct memory access memory pool into storage performance development kit memory; storing data in the storage performance development kit memory when the data passes through the remote direct memory access memory pool; obtaining storage data of the remote direct memory access memory pool; transmitting the storage data to a plurality of preset buffers of a non-volatile memory interface device; obtaining memory data including the storage data in the plurality of preset buffers; and judging the source of the memory data in any preset buffer. If the memory data in the preset buffer is from the storage performance development kit memory, the memory data in the preset buffer is submitted to a storage performance development kit non-volatile memory interface driver. The number of copying processes is reduced, and thus the number of interrupts, the number of context switches, and the CPU usage are reduced. In the embodiment of the application, the memory used by the remote direct memory access memory pool is modified, so that the remote direct memory access memory pool uses the memory from the storage performance development kit, and the source of the memory data of the non-volatile memory interface device is judged, so that the memory from the storage performance development kit can be directly submitted to the storage performance development kit non-volatile memory interface driver, the copying operation is reduced, the CPU usage is reduced, system resources are saved, and performance is improved.

[0178] The embodiment of the application further provides a communication device, as shown in the figure, comprising a processor 301, a communication interface 302, a memory 303 and a communication bus 304, wherein the processor 301, the communication interface 302 and the memory 303 complete mutual communication through the communication bus 304, Figure 14

[0179] The memory 303 is configured to store a computer program.

[0180] The processor 301 is configured to execute the program stored in the memory 303 to implement the following steps:

[0181] ​Obtaining an initial configuration of a memory manager, the memory manager being used to manage a remote direct memory access memory pool, the remote direct memory access memory pool using a registered memory;

[0182] Modifying the registered memory used by the remote direct memory access memory pool to a storage performance development kit memory by modifying the initial configuration;

[0183] Obtaining storage data of the remote direct memory access memory pool;

[0184] Transferring the storage data to a plurality of preset buffers of a non-volatile memory interface device;

[0185] Obtaining memory data including the storage data in the plurality of preset buffers and determining a source of the memory data in any of the preset buffers;

[0186] If the memory data in the preset buffer is from the storage performance development kit memory, submitting the memory data in the preset buffer to a storage performance development kit non-volatile memory interface driver.

[0187] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architectrue (EISA) bus, etc. The communication bus can be divided into a physical address bus, a data bus, a control bus, etc. For convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0188] The communication interface is used for communication between the terminal and other devices.

[0189] The memory can include a Random Access Memory (RAM) and can also include a non-volatile memory, such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0190] The processor described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0191] The application further provides a readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the storage optimization method of the foregoing embodiments.

[0192] For the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts are referred to the part of the description of the method embodiments.

[0193] The algorithms and displays provided herein are not inherently related to any particular computer, virtual apparatus, or other apparatus. Structural requirements of such apparatus required to practice the application are evident as based on the description provided herein. Furthermore, the present application is not inherently related to any particular programming language. It will be appreciated that a variety of programming languages can be used to implement the present application as described herein, and any references below to specific languages are provided for disclosure of enablement only.

[0194] In the specification provided herein, a large number of specific details are described. However, it can be understood that the embodiments of the application can be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure the understanding of the specification.

[0195] Similarly, it is to be understood that the embodiments of the application can be over-simplified for the sake of brevity. The individual features of the application are sometimes described in conjunction with common embodiments, drawings, or descriptions of the application. However, the disclosed method should not be interpreted as reflecting an intention that the claimed application requires more features than are explicitly recited in each claim. Rather, inventive aspects lie in less than all features of the previously disclosed single embodiments, as reflected in the following claims. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim is a separate embodiment of the application. The claims are not to be interpreted as a requirement that the application require more features than are explicitly recited in each claim.

[0196] Those skilled in the art will appreciate that the modules in the apparatuses in the embodiments can be adapted and placed in one or more apparatuses other than that of the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and furthermore can be split into multiple sub-modules or sub-units or sub-components. Any combination of all the features disclosed in the specification (including the accompanying claims, abstract and drawings), and any method or process or steps of an embodiment so disclosed, can be made unless it is explicitly stated otherwise. Each feature disclosed in the specification (including the accompanying claims, abstract and drawings), can be replaced by alternative features serving the same, equivalent or similar purpose unless expressly stated otherwise.

[0197] Embodiments of the various components of the application can be implemented in hardware, or as software modules running in one or more processors, or combinations thereof. Those skilled in the art will appreciate that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functionality of some or all of the components in the sequencing apparatus according to the application. The application can also be implemented as a program for executing one or more parts of the methods described herein on a device or apparatus. Such a program can be stored on a computer readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier medium, or in any other form.

[0198] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the system claims enumerating several means, several of these means can be embodied by one and the same item of hardware. The use of the word 'at least' followed by a list of one or more items means that any item in the list can be present or there can be more than one of an item in the list. The use of the term 'about' with respect to a numerical value specifies the closeness of the value to the stated value. The use of the term 'comprise' and variations thereof such as 'comprising', 'comprises' and 'comprised of' is not intended to exclude other additives, components, integers or steps.

[0199] It is clear for those skilled in the art that, for the convenience and brevity of description, the specific working processes of the apparatuses, devices and units described above can refer to the corresponding processes in the foregoing method embodiments, which will not be described here again.

[0200] The above merely describes preferred embodiments of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0201] The above merely describes preferred embodiments of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0202] It should be noted that the acquisition of various data related processes in the embodiments of the present application are performed under the premise of complying with the corresponding data protection regulations and policies of the country where the device is located, and with the authorization given by the corresponding device owner.

Claims

1. A storage optimization method, characterized by, The method comprises: obtaining an initial configuration of a memory manager, the memory manager being used for managing a remote direct memory access memory pool, the remote direct memory access memory pool using registered memory; modifying the registered memory used by the remote direct memory access memory pool to storage performance development kit memory by modifying the initial configuration; obtaining storage data of the remote direct memory access memory pool; transferring the storage data to a plurality of preset buffers of a non-volatile memory interface device; obtaining memory data including the storage data in the plurality of preset buffers and judging the source of the memory data in any of the preset buffers; if the memory data in the preset buffer is from the storage performance development kit memory, submitting the memory data in the preset buffer to a storage performance development kit non-volatile memory interface driver.

2. The method of claim 1, wherein, The transferring of the storage data to a plurality of preset buffers of a non-volatile memory interface device comprises: obtaining the data amount of the storage data; if the data amount meets a preset condition, transferring the storage data to a plurality of preset buffers of a non-volatile memory interface device; if the data amount does not meet the preset condition, obtaining target data from ordinary memory; transferring the target data and the storage data to a plurality of preset buffers of a non-volatile memory interface device.

3. The method of claim 2, wherein, After the obtaining of the memory data including the storage data in the plurality of preset buffers and the judging of the source of the memory data in any of the preset buffers, the method further comprises: generating an initial buffer linked list of the non-volatile memory interface device based on the plurality of preset buffers in the non-volatile memory interface device; if the memory data in any of the preset buffers is from the ordinary memory, setting a memory block allocator in the storage performance development kit; allocating a plurality of preset size first target memory blocks from the memory of the storage performance development kit through the memory block allocator; copying the memory data in the preset buffer to the first target memory block; if any of the memory data is from the storage performance development kit memory, determining the preset buffer corresponding to the memory data as a target buffer; generating a new buffer linked list based on the target buffer and the first target memory block; sending the new buffer linked list to the storage performance development kit, setting an input / output interface function in the storage performance development kit; submitting the corresponding memory data in the new buffer linked list to a storage performance development kit non-volatile memory interface driver through the input / output interface function.

4. The method of claim 2, wherein, The obtaining of target data from ordinary memory if the data amount does not meet the preset condition comprises: if the data amount is less than a preset value or is not an integer multiple of the preset value, taking the modulus of the data amount and the preset value to obtain a modulus data amount; obtaining a target difference value through the modulus data amount and the preset value; obtaining target data from ordinary memory through the target difference value.

5. The method of claim 1, wherein, The modifying the registered memory used by the remote direct memory access memory pool to the storage performance development kit memory comprises: pre-dividing a second target memory block to be allocated in the memory of the storage performance development kit; setting a memory allocation pointer in the memory manager; obtaining a memory address of the second target memory block through the memory allocation pointer; controlling the remote direct memory access memory pool to use the second target memory block of the storage performance development kit through the memory address.

6. The method of claim 5, wherein, After the setting the memory allocation pointer in the memory manager, the method further comprises: setting a memory release pointer in the memory manager; obtaining the memory address of the second target memory block through the memory release pointer in a case that the second target memory block is determined to end use; releasing the second target memory block through the memory address.

7. The method of claim 1, wherein, After the obtaining the initial configuration of the memory manager, the method further comprises: obtaining a component of a buffer list, the component of the buffer list respectively managing buffers in the remote direct memory access memory pool and the non-volatile memory interface device, the buffers comprising preset buffers of the non-volatile memory interface device; adding a Boolean flag component in the component of the buffer list; when the data of the buffer is from the storage performance development kit memory, setting a Boolean flag with a first Boolean value in the buffer through the Boolean flag component; when the data of the buffer is from a common memory, setting a Boolean flag with a second Boolean value in the buffer through the Boolean flag component.

8. A storage optimization apparatus, characterized by, The method comprises: a first obtaining module, configured to obtain an initial configuration of a memory manager, the memory manager being configured to manage a remote direct memory access memory pool, the remote direct memory access memory pool using registered memory; a configuration modifying module, configured to modify the registered memory used by the remote direct memory access memory pool to storage performance development kit memory by modifying the initial configuration; a second obtaining module, configured to obtain storage data of the remote direct memory access memory pool; a transmission module, configured to transmit the storage data to preset buffers of a non-volatile memory interface device; a third obtaining module, configured to obtain memory data comprising the storage data in the preset buffers, and determine a source of the memory data in any of the preset buffers; a data submitting module, configured to submit the memory data in the preset buffers to a storage performance development kit non-volatile memory interface driver if the memory data in the preset buffers is from the storage performance development kit memory.

9. A communication device, characterized by The method comprises: a transceiver, a memory, a processor, and a program stored in the memory and executable on the processor; the processor is configured to read the program in the memory to implement steps in the storage optimization method according to any one of claims 1-7.

10. A readable storage medium for storing a program, characterized by The storage program is executed by the processor to implement steps in the storage optimization method according to any one of claims 1-7.

Citation Information

Patent Citations

  • Data processing method and device based on remote direct memory access and medium

    CN115934623A

  • Method, device and equipment for realizing full copy storage of SPDK architecture and medium

    CN116521069A