Data processing method, device, equipment and readable storage medium

Abstract

The embodiments of the present application provide a data processing method, apparatus, device and readable storage medium, which relate to the field of data storage technology. The method comprises: a data processing unit DPU device side receives compression task metadata from a host side; compresses the compression task metadata by calling a hardware accelerator on the DPU device side to obtain compression result metadata; and sends the compression result metadata to the host side. In the embodiments of the present application, by offloading the compression (compaction) operation of the host side to the DPU device side for execution, data movement is eliminated, thereby reducing network traffic overhead. At the same time, the DPU device side can speed up the execution efficiency of the compaction operation by calling the hardware accelerator on the DPU device side to compress the compression task metadata, thereby reducing the occupancy of the host side CPU, releasing the computing resources of the host side, and improving energy efficiency.

Description

Technical Field

[0001] The present application relates to the field of data storage technology. Specifically, the present application relates to a data processing method, apparatus, device and readable storage medium. Background Art

[0002] The decoupled data center architecture based on the Data Processing Unit (DPU) decouples different heterogeneous hardware into corresponding resource pools and connects them via high-speed networks, allowing different types of resources to be independently scaled on demand, resulting in better resource utilization and elasticity. Furthermore, the DPU hardware integrates computing, networking, and storage functions, assisting in processing data traffic from the network, thereby achieving higher energy efficiency.

[0003] However, existing key-value storage systems based on log-structured merge trees are not well adapted to this new DPU separation structure because the background operations of the log-structured merge tree, especially the compaction operation, will cause data write amplification, resulting in a large amount of data movement, while consuming computing and network resources and seriously degrading system performance. Summary of the Invention

[0004] The present application provides a data processing method, apparatus, device and readable storage medium for solving technical problems in the prior art.

[0005] In a first aspect, a data processing method is provided, which is applied to a data processing unit (DPU) device. The method includes:

[0006] Receive compression task metadata from the host;

[0007] Perform compression on the compression task metadata by calling a hardware accelerator on the DPU device to obtain compression result metadata;

[0008] Send the compression result metadata to the host end.

[0009] In one possible implementation, the compression task metadata is compressed by calling a hardware accelerator on the DPU device to obtain compression result metadata, including:

[0010] Based on the compression task metadata, obtain the corresponding sort string table SSTable file;

[0011] The new SSTable file generated after decompressing, merging, and sorting the SSTable file is compressed by calling the hardware accelerator on the DPU device to obtain the compression result metadata.

[0012] In another possible implementation, obtaining the corresponding SSTable file includes:

[0013] Read the corresponding SSTable file from the storage device through the lightweight network file system;

[0014] The method further comprises:

[0015] The compression result metadata is written to the storage device through the lightweight network file system.

[0016] In yet another possible implementation, the method further includes:

[0017] The slave instance reads the write-before log file of the master instance through the lightweight network file system and synchronizes the write-before log file with the memory table of the master instance according to the write-before log file, wherein the slave instance runs on the DPU device side and the master instance runs on the host side;

[0018] The process of obtaining the compression result metadata includes:

[0019] The old SSTable file is read from the instance, and the number of the old SSTable file is added to the prefix of the instance thread number as the number of the generated new SSTable file to obtain the updated metadata;

[0020] The old SSTable file is an SSTable file read from the storage device through a lightweight network file system;

[0021] Sending the compression result metadata to the host side includes:

[0022] The slave instance returns the updated metadata to the master instance and retains the old SSTable file.

[0023] In another possible implementation, during network transmission through the lightweight network file system, the IO operations of the data plane are directly passed to the storage device side via the high-speed network-based non-volatile memory standard object (NVMe-oF Target) offloaded on the DPU device side, and the metadata operations of the control plane are processed by the general computing unit on the DPU device side.

[0024] In yet another possible implementation, the compressed task metadata transmitted by the host side through a Remote Direct Data Access (RDMA) process is received, and the compressed result metadata is sent to the host side through the RDMA process.

[0025] In a second aspect, a data processing method is provided, which is applied to a host side, and the method includes:

[0026] Sending compression task metadata to the data processing unit (DPU) device;

[0027] Receive compression result metadata sent by the DPU device, wherein the compression result metadata is obtained by the DPU device by calling a hardware accelerator on the DPU device to perform compression on the compression task metadata.

[0028] In one possible implementation, the method further includes:

[0029] The metadata information is updated according to the compression result metadata.

[0030] In another possible implementation, updating metadata information according to the compression result metadata includes:

[0031] The master instance running on the host side reads the number of the new SSTable file from the updated metadata, and obtains the SSTable file number synchronized with the master instance;

[0032] Modify the number of the new SSTable file to the SSTable file number synchronized with the main instance, and delete the old SSTable file.

[0033] In a third aspect, a data processing device is provided, which is applied to a data processing unit (DPU) device, including:

[0034] A first receiving module is configured to receive compression task metadata from a host side;

[0035] a processing module configured to compress the compression task metadata by calling a hardware accelerator on the DPU device to obtain compression result metadata;

[0036] The first sending module is configured to send the compression result metadata to the host end.

[0037] In a fourth aspect, a data processing device is provided, which is applied to a host side and includes:

[0038] A second sending module is configured to send compression task metadata to a data processing unit DPU device end;

[0039] The second receiving module is configured to receive the compression result metadata sent by the DPU device, wherein the compression result metadata is obtained by the DPU device by calling the hardware accelerator on the DPU device end to compress the compression task metadata.

[0040] According to a fifth aspect, an electronic device is provided, comprising:

[0041] one or more processors;

[0042] Memory;

[0043] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to: execute the data processing method shown in the first aspect above.

[0044] For example, in a fifth aspect of the present application, a computing device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus;

[0045] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the data processing method shown in the first aspect or the second aspect of the present application.

[0046] The electronic device of the fifth aspect of the present application may be a DPU device end or a host end.

[0047] In a sixth aspect, a computer-readable storage medium is provided, wherein the processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the data processing method shown in the first aspect or the second aspect above.

[0048] The beneficial effects of the technical solution provided by this application are:

[0049] By offloading the host-side compression (compaction) operation to the DPU device, data movement is eliminated, thereby reducing network traffic overhead. At the same time, the DPU device calls the hardware accelerator on the DPU device to compress the compression task metadata, which can speed up the execution efficiency of the compaction operation, thereby reducing the host-side CPU usage, freeing up the host-side computing resources, and improving energy efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments of the present application.

[0051] Figure 1 A flowchart of a data processing method provided in an embodiment of the present application;

[0052] Figure 2 A flowchart of a data processing method provided in another embodiment of the present application;

[0053] Figure 3A schematic diagram of a compaction offloading system structure for a key-value storage system with a DPU separation structure provided in an embodiment of the present application;

[0054] Figure 4 An architectural diagram illustrating an implementation of a compaction offloading method for a key-value storage system with a DPU separation structure provided in an embodiment of the present application;

[0055] Figure 5 A structural diagram of the data layout of the lightweight network file system provided in an embodiment of the present application;

[0056] Figure 6 A schematic diagram of a data plane and control plane separation mechanism for a lightweight network file system provided in an embodiment of the present application;

[0057] Figure 7 A schematic diagram of a metadata path IO queue of a lightweight network file system provided in an embodiment of the present application;

[0058] Figure 8 A schematic diagram of the RDMA communication process of the RDMA remote procedure call framework provided in an embodiment of the present application;

[0059] Figure 9 Schematic diagram of metadata synchronization mechanism between master and slave instances provided in an embodiment of the present application;

[0060] Figure 10 Schematic diagram of the hardware-accelerated compaction mechanism on the DPU device side provided in an embodiment of the present application.

[0061] Figure 11 A flowchart of a data processing device provided in another embodiment of the present application;

[0062] Figure 12 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0063] Figure 13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present invention.

[0065] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0066] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0067] With the continuous improvement of modern computer hardware and software performance, the rapid development of the internet, and the widespread adoption of devices like smartphones, people are generating data at an ever-increasing rate, resulting in an ever-increasing amount of data. This massive amount of information and data has undoubtedly increased storage requirements. Key-value storage database systems have also been evolving in recent years. Key-value storage systems based on log-structured merge-tree (LSM-tree) are a popular approach for building data center backend storage engines, especially for write-intensive scenarios. Log-structured merge-tree systems absorb write operations in a memory buffer and sequentially flush them to disk when the buffer is full. This results in superior write throughput compared to traditional relational databases. Key-value storage systems based on log-structured merge-trees are commonly used as standalone storage engines in various data center applications, deployed on hybrid storage architectures of memory and NVMe SSDs.

[0068] The recently emerged decoupled data center architecture based on Data Processing Units (DPUs) has attracted increasing attention. It decouples different heterogeneous hardware into corresponding resource pools and connects them through high-speed networks, allowing different types of resources to be independently scaled on demand, resulting in better resource utilization and elasticity. In addition, DPU hardware integrates computing, networking, and storage-related functions, and can assist in processing data traffic from the network, thereby achieving higher energy efficiency. However, existing key-value storage systems based on log-structured merge trees are not well adapted to this new DPU separation structure because the background operations of the log-structured merge tree, especially the compaction operation, can cause data write amplification, resulting in a large amount of data movement, while consuming computing and network resources and seriously degrading system performance.

[0069] Existing key-value storage systems based on a DPU-decoupled storage architecture have not fully leveraged the advantages of this architecture. Due to the separation of storage and compute resources in this new architecture, the front-end and back-end operations of the log-structured merge tree result in a large amount of data movement. In particular, compaction operations frequently require reading data from the storage pool, merging it into the compute pool, and then writing it back, resulting in enormous network traffic. Although existing research on optimizations to reduce write amplification can somewhat reduce the network traffic caused by compaction, it cannot completely eliminate it. In this decoupled storage and compute architecture, network resources are extremely precious. Therefore, further reducing data movement within the network is necessary to improve energy efficiency.

[0070] On the other hand, existing designs don't fully utilize the hardware resources on the dedicated data processing unit (DPU). The DPU includes various computing units, such as low-power general-purpose ARM processors, various hardware accelerators, and Remote Direct Memory Access (RDMA) network cards that support NVMe-oF target offload. Utilizing these resources effectively is crucial for reducing data movement and improving system energy efficiency.

[0071] To address the above issues, this application proposes a data processing method, specifically a compaction offloading method for a key-value storage system with a DPU separation structure.

[0072] First, several terms involved in this application are introduced and explained:

[0073] Nonvolatile Memory Express over Fabrics (NVMe-oF) based on high-speed networks: NVMe is a new, performance-based storage protocol that simplifies protocol complexity and fully leverages the low latency and parallelism of hardware PCIe channels, significantly improving the read and write performance of high-performance storage devices. NVMe-oF extends the NVMe specification's implementation on the PCIe bus, implementing NVMe functionality through the use of various common transport layer protocols, mapping NVMe to high-speed network transmission channels, and enabling shared network access to high-performance storage devices.

[0074] Storage Performance Development Kit (SPDK): A data plane development toolkit provided by Intel that provides library functions and driver support for user space under the Intel Architecture (IA) processor architecture, enabling efficient packet processing. Unlike the driver code of traditional storage devices, SPDK code runs in user mode, avoiding the frequent kernel context switching overhead and being more adapted to the characteristics of high-speed storage devices. In addition, the polling mode driver allows the application to continue performing other tasks after submitting a read or write request, and then use periodic polling to check whether the read or write request has been completed. This avoids the huge interrupt overhead caused by the traditional process of the application submitting a read or write request and then sleeping and waiting to be awakened by an interrupt after the request is completed.

[0075] Data Plane Development Kit (DPDK): A collection of function libraries and drivers developed by Intel and other companies for fast packet processing. DPDK uses polling rather than interrupts to process packets. When a packet is received, the driver of the data processing unit overloaded by DPDK does not notify the CPU through an interrupt, but directly stores the packet into memory and delivers it to the application layer software for direct processing through the interface provided by DPDK, thus saving a lot of CPU interrupt time and memory copy time. In addition, the driver provided by DPDK for the data processing unit includes the underlying interface for calling its hardware accelerator. Users can write corresponding functional modules based on these interfaces and use the hardware accelerator of the data processing unit by calling these functional modules in their own programs.

[0076] RocksDB Secondary Instance (RSI): RocksDB is a widely used LSM-tree-based key-value storage system. In addition to the RocksDB instance (primary instance) itself, one or more RocksDB slave instances can be opened. Unlike RocksDB in read-only mode, slave instances can read the latest data written by the primary instance. RSI dynamically tracks the primary instance's write-ahead log (WAL) file and manifest file by replaying the write-ahead log, thereby achieving data synchronization with the primary instance. However, replaying the write-ahead log is not done in real time and there is a certain time delay.

[0077] SSTable: (Sorted String Table), a log-structured merge tree (LSM-tree) uses the SSTable format to persist LSM-trees to disk. An SSTable is a format for storing key-value pairs, sorted by key. It is an ordered structure and can also be called a sorted string table. An SSTable consists of multiple sorted files called segments, which are immutable once written to disk.

[0078] Compression (Compaction) operation: When there are too many SSTable files, use the Compaction operation to remove the redundant data between different SSTables, merge and sort them, and generate new SSTable files.

[0079] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0080] The present invention provides a data processing method in an embodiment, which can be executed by a data processing unit DPU device. Figure 1 As shown, the method includes:

[0081] S101, receiving compression task metadata from a host;

[0082] S102, compressing the compression task metadata by calling a hardware accelerator on the DPU device to obtain compression result metadata;

[0083] S103: Send the compression result metadata to the host.

[0084] In this embodiment, the host-side compaction operation is offloaded to the DPU device to eliminate data movement and reduce network traffic overhead. At the same time, the DPU device uses its own hardware accelerator to compress the compression task metadata, which can speed up the execution of the compaction operation. This reduces host CPU usage, frees up host computing resources, and improves energy efficiency.

[0085] In this embodiment, S101 may specifically include: receiving compressed task metadata transmitted by the host end through a remote direct data access (RDMA) process;

[0086] S103 may specifically be: sending compression result metadata to the host end through the RDMA process.

[0087] In the solution of the embodiment of the present application, the high-bandwidth and low-latency network resources brought by the RDMA network card in the DPU device side are fully utilized. In the process of metadata transmission and communication through the remote direct data access RDMA process, the latency can be effectively reduced and the consumption of computing resources can be reduced.

[0088] This embodiment of the present application provides a possible implementation method, and S102 may specifically include:

[0089] Based on the compression task metadata, obtain the corresponding sort string table SSTable file;

[0090] The new SSTable file generated after decompressing, merging, and sorting the SSTable file is compressed by calling the hardware accelerator on the DPU device to obtain the compression result metadata.

[0091] In this embodiment, the compression task metadata includes information about the SSTable file to be compressed, such as the number and storage location of the SSTable file.

[0092] Specifically, in this embodiment, the SSTable file corresponding to the compression task metadata can be read from the storage device side through a lightweight network file system, and the new SSTable file generated after decompressing, merging, and sorting the read SSTable file can be compressed by calling the hardware accelerator on the DPU device side to obtain the compression result metadata, thereby enabling the compaction task on the host side to be executed on the DPU device side, releasing the computing resources on the host side and improving energy efficiency.

[0093] In a possible implementation, the above method may further include:

[0094] The slave instance reads the write-before log file of the master instance through the lightweight network file system and synchronizes the write-before log file with the memory table of the master instance according to the write-before log file, wherein the slave instance runs on the DPU device side and the master instance runs on the host side;

[0095] The process of obtaining the compression result metadata includes:

[0096] The slave instance adopts the original RocksDB logic corresponding to the master instance to read the old SSTable file and the contents of the merged SSTable, and adds the number of the old SSTable file to the prefix of the slave instance thread number as the number of the generated new SSTable file to obtain the updated metadata;

[0097] The old SSTable file is an SSTable file read from the storage device through a lightweight network file system;

[0098] Sending the compression result metadata to the host side includes:

[0099] The slave instance returns the updated metadata to the master instance and retains the old SSTable file.

[0100] In the solution of the above embodiment, based on the characteristics of RocksDB master-slave instances, a master-slave instance metadata synchronization mechanism is implemented, enabling read-only slave instances to handle compaction tasks and avoiding conflicts caused by inconsistencies between the master and slave instances.

[0101] A possible implementation is provided in an embodiment of the present application. The above method may further include:

[0102] The compression result metadata is written to the storage device through the lightweight network file system.

[0103] That is to say, the compression result metadata is written to the storage device side (for example: NVMe SSD) through the lightweight network file system to achieve data persistence.

[0104] In the above embodiment, during network transmission through the lightweight network file system, the IO operations of the data plane are directly passed to the storage device side via the high-speed network-based non-volatile memory standard object (NVMe-oFTarget) unloaded on the DPU device side, and the metadata operations of the control plane are processed by the general computing unit on the DPU device side.

[0105] Since the data plane and control plane separation mechanism is adopted during network transmission of the lightweight network file system, it can improve data reading and writing performance while reducing the consumption of general-purpose computing ARM architecture processors on the DPU device side.

[0106] The present application also provides a data processing method, which can be executed by the host side, such as Figure 2 As shown, the method includes:

[0107] S201, sending compression task metadata to the data processing unit DPU device end;

[0108] S202. Receive compression result metadata sent by the DPU device, wherein the compression result metadata is obtained by the DPU device by calling a hardware accelerator on the DPU device to compress the compression task metadata.

[0109] In this embodiment, the compression operation on the host side is offloaded to the DPU device side for execution, thereby eliminating data movement and reducing network traffic overhead.

[0110] In this embodiment, S201 may specifically include: sending the compressed task metadata to the data processing unit DPU device end through a remote direct data access RDMA process;

[0111] S202 may specifically be: receiving compression result metadata sent by the DPU device through the RDMA process.

[0112] In this embodiment, metadata is transmitted through a remote direct data access (RDMA) process, which effectively reduces the delay in the transmission process and reduces the consumption of computing resources.

[0113] A possible implementation is provided in an embodiment of the present application. The above method may further include:

[0114] S203 (not shown in the figure): update metadata information according to the compression result metadata.

[0115] In a possible implementation, S203 may specifically include:

[0116] The master instance running on the host side reads the number of the new SSTable file from the updated metadata, and obtains the SSTable file number synchronized with the master instance;

[0117] Modify the number of the new SSTable file to the SSTable file number synchronized with the main instance, and delete the old SSTable file.

[0118] In the scheme of the above embodiment, after the master instance running on the host side receives the updated metadata (i.e., the compressed result metadata) sent by the slave instance running on the DPU device, it reads the number of the new SSTable file from it, and based on this, updates the SSTable file number synchronized with the master instance, and deletes the old SSTable file, thereby avoiding conflicts caused by inconsistencies between the master and slave instances.

[0119] The above describes the technical solution of a data processing method provided by this application from the perspectives of the DPU device side and the host side. Figure 3-10 , a detailed description of the specific implementation process of the solution provided in this application is given.

[0120] like Figure 3As shown, the present application provides a compaction offload system for a key-value storage system with a DPU separation structure, and the above solution is implemented based on this compaction offload system. The compaction offload system includes: a host side, a DPU device side, and an NVMe SSD storage device side. The host side runs a lightweight network file system client, and the DPU device side runs a lightweight network file system server. The lightweight network file system provides a synchronized and unified file view for the host side and the DPU device side.

[0121] The host side is fully adapted to the RocksDB API. User programs on the host side can operate on data in the key-value storage system by calling the data read and write interfaces provided by the RocksDB API. The host side runs the master instance of the key-value storage system, forwards background compaction task metadata to the DPU device side, and receives compaction result metadata from the DPU device side.

[0122] The DPU device runs a slave instance of the key-value storage system, receives compaction task metadata from the host, executes the compaction task, and sends compaction result metadata to the host. The DPU device uses a hardware accelerator to accelerate the compaction task execution process.

[0123] The NVMe SSD storage device stores sorted string table (SSTable) files, write-ahead log (WAL) files, LOG files and Manifest files.

[0124] In the above embodiment, the architecture diagram of the compaction unloading method is as follows: Figure 4 As shown in the figure, the specific operation process of the uninstall system is as follows:

[0125] When a host-side user program calls the RocksDB API to perform data operations on the key-value storage system, a background compaction operation is triggered. The host-side background thread transfers the metadata of the compaction task to the DPU device through the Remote Direct Data Access (RDMA) process.

[0126] After the DPU device-side thread receives the metadata of the compaction task, it opens the slave instance of the key-value storage system and parses the compaction task information on the slave instance; it reads the sorted string table SSTable file involved in the compaction task from the NVMe SSD storage device through the lightweight network file system, calls the hardware accelerator on the DPU device side to decompress the SSTable file, merges and sorts the decompressed SSTable file content, and also calls the hardware accelerator to compress the newly generated SSTable file, and writes the new SSTable file to the NVMe SSD storage device side through the lightweight network file system to achieve data persistence. Finally, the metadata of the compaction result (new SSTable file) is returned to the host side through the remote direct data access RDMA process call.

[0127] The host deletes the old SSTable file on the primary instance based on the compaction result metadata and updates the metadata information.

[0128] The solution of this application uses a customized lightweight network file system, an RDMA remote procedure call framework, a metadata synchronization mechanism between master and slave instances, and a DPU-side hardware acceleration mechanism to achieve efficient execution of host-side compaction tasks on the DPU device side, achieving the effect of compaction task offloading in a DPU separation structure.

[0129] It should be noted that in order to maximize the use of the limited computing power of the data processing unit in compaction tasks, the network file system used on the host and DPU should be designed to be as lightweight as possible to avoid additional performance overhead.

[0130] The following describes in detail the customized lightweight network file system, RDMA procedure call framework, metadata synchronization mechanism between master and slave instances, and DPU-side hardware acceleration mechanism.

[0131] 1. Customized lightweight network file system:

[0132] Because the lightweight network file system is customized for an LSM-tree-based key-value storage system, the implemented interface can only meet the requirements of the key-value storage system. Furthermore, the disk layout should take into account the application characteristics of the key-value storage system, where write operations are sequential writes of large amounts of data, while read operations are essentially random reads of small amounts of data. Therefore, the lightweight network file system in the above embodiment can have the following performance:

[0133] 1. Implement the interfaces required for opening, reading, and writing files in a key-value storage system, ignoring the complex characteristics of traditional kernel file systems;

[0134] 2. Distribute write requests evenly to multiple storage devices, such as NVMe SSDs (Solid State Disks or Solid State Drives), based on the block granularity of the key-value storage system to maximize the parallelism of write operations.

[0135] 3. Use a streamlined disk layout to reduce metadata management overhead. The data layout structure of the lightweight network file system is as follows: Figure 5 As shown, the logical block address (LBA) of the storage device is divided into multiple clusters (Cluster) for management according to the size of the sorted string table (SSTable), which is generally 64MB. Each cluster contains multiple chunks (Chunk), and the size of the chunk is consistent with the write granularity of the key-value storage system, which is generally 1MB. Among them, the first cluster is also called the metadata cluster, which stores the file system's super block (Super Block), metadata item bitmap (Meta Bitmap), data block bitmap (DataBitmap), file metadata (Meta Region) and other information, and the remaining clusters store data of files such as SSTable, WAL, and Manifest;

[0136] 4. Use a bitmap to record the allocation and free blocks of files. When the file system starts, it scans the metadata pages in the metadata cluster and creates a hash mapping table from file names to blocks in memory. When searching, the file location is located through a hash calculation.

[0137] In the lightweight network file system, the data plane and the control plane are separated during the network transmission process. The separation mechanism is as follows: Figure 6 As shown in the figure, on the one hand, the goal of the data plane is to reduce the copying, context switching and locking overhead of the data path, give full play to the parallelism of NVMe devices, and reduce or eliminate the consumption of the data plane on the DPU general computing unit. On the other hand, the goal of the control plane is to reduce the latency of metadata operation requests and reduce the consumption of metadata operations on the DPU general computing unit.

[0138] The specific operations are described as follows:

[0139] 1. Data plane IO operations are directly transferred to NVMe SSD storage devices via the NVMe-oF target offloaded on the DPU without interrupting the DPU's general computing unit;

[0140] 1) Uninstall the NVMe-oF Target and transfer the read and write operations of the block device (storage device with block addressing) on ​​the data plane directly to the NVMe SSD storage device through the NVMe-oF Target uninstalled on the DPU, without interrupting the general computing unit of the DPU and reducing data copying;

[0141] 2) Host-side active write: When the key-value storage system executes a flush, the host side actively calls the NVMe-oF interface on the data plane to directly write to the NVMe SSD storage device without sending the request to the DPU for processing, reducing the computing pressure on the DPU and shortening the request processing path;

[0142] 3) Poll the IO queue and evenly distribute the IO operations of the chunks on the same NVMe SSD storage device in the cluster involved in the IO request to different IO queues of the NVMe SSD storage device. For example, Figure 5 Cluster 0 in the figure has multiple chunks distributed on SSD 0 (vertically). After the IO requests are segmented into chunks, they can be submitted in parallel to different chunks on the same SSD, leveraging the parallelism of NVMe SSD storage devices.

[0143] 4) User-mode caching uses RocksDB's block cache. However, unlike RocksDB, it uses the Intel SPDK framework's SPDK_DMA_ALLOC() function to allocate memory, bypassing the operating system's page cache to implement user-mode caching.

[0144] 2. Hand over the metadata operations of the control plane to the general computing unit of the DPU for processing.

[0145] 1) Use RDMA procedure calls to implement metadata transmission, thereby reducing the interruption of the DPU's general computing unit;

[0146] 2) Use lock-free and asynchronous persistence design to reduce metadata path latency. The metadata path IO queue diagram is as follows: Figure 7As shown, a set of metadata I / O queues is implemented using the Intel SPDK framework. The metadata I / O queues of the lightweight network file system include a send queue and a completion queue. After a user thread submits a metadata I / O request to the send queue, it immediately returns. The file system metadata I / O thread then proceeds to complete the request. After completing the request, the file system metadata I / O thread places the result of the request in the completion queue. The user thread periodically polls the completion queue, retrieves the corresponding result of the request submitted to the send queue, and continues processing subsequent processes. During the periodic query of the completion queue, the user thread does not switch threads and continues to submit or process other I / O requests.

[0147] 2. RDMA (Remote Direct Memory Access) procedure call framework:

[0148] In the above embodiment, the RDMA communication flow diagram of the RDMA process call framework is as follows: Figure 8 As shown. Unlike traditional network communication protocols, RDMA-based communication requires the operation of issuing tasks in advance. This is to avoid the situation where the target end does not have a corresponding RECV task when the SEND task is delivered to the target end, triggering a Receive Not Ready (RNR) error. In addition, before delivering the actual data, unlike traditional network communication, metadata must be delivered first. The metadata contains the memory address of the remote operation and the key that authorizes access to the remote memory. The specific process is described as follows:

[0149] 1. The server starts listening and waits for connection request events;

[0150] 2. After receiving the client connection request event, the server opens a new thread, which pre-sends a RECV task to the network card to receive the SEND task sent by the client;

[0151] 3. The client sends a SEND task with RDMA metadata, which is received by the RECV task previously sent by the server. The server then obtains the client's RDMA metadata.

[0152] 4. The server issues an RDMA READ task to read the client's request and process it. At the same time, the client pre-sends a RECV task to the network card to receive the server's response.

[0153] 5. After the server processes the request, it issues an RDMA WRITE task to write the response to the request directly into the client's memory buffer, which is perceived by the RECV task issued in advance by the client.

[0154] It should be noted that RDMA remote procedure calls (RDMA RPCs) are responsible for transmitting metadata about compaction offload tasks between master and slave instances throughout the system, as well as between the network file system server and client. In the transmission of metadata about compaction offload tasks between master and slave instances, the RDMA RPC server and client correspond to the slave instance (deployed on the DPU device) that receives the compaction offload task and the master instance (deployed on the host) that sends the compaction offload task. In the transmission of metadata between the network file system server and client, the RDMA RPC server and client correspond to the network file system server (deployed on the DPU) and client (deployed on the host), respectively.

[0155] 3. Metadata synchronization mechanism between master and slave instances:

[0156] The schematic diagram of metadata synchronization mechanism between master and slave instances is as follows Figure 9 As shown, the native RocksDB slave instance is designed to be read-only and is not strictly synchronized with the master instance. Since the compaction process requires reading and writing the sorted string table file and ensuring metadata synchronization between the master and slave instances, this embodiment transforms the native RocksDB slave instance to enable it to perform compaction tasks. At the same time, when writing the new sorted string table file during the compaction process, it ensures that there will be no conflict with the master instance. The specific operation is described as follows:

[0157] 1. Add the compaction feature to slave instances, enabling read-only slave instances to perform compaction tasks just like the master instance.

[0158] 2. After receiving the compaction task from the master instance, the slave instance reads the master instance's write-ahead log file through a lightweight network file system, synchronizes the log file with the master instance's memtable, and builds the slave instance's manifest file.

[0159] 3. During the compaction task execution on the slave instance, the logic for reading the old sort string table file and merging the sort string table contents is consistent with the original RocksDB logic. When writing the new sort string table file, the newly generated sort string table file is numbered with the old sort string table file number plus the slave instance thread number prefix to distinguish it from the master instance's sort string table file.

[0160] 4. After the slave instance completes the compaction task, it returns the updated metadata to the master instance. Unlike the original RocksDB, the old sort string table file is not deleted to avoid conflicts with the master instance.

[0161] 5. The master instance reads the new sort string table file number from the updated metadata, and reads the manifest file to obtain the sort string table file number synchronized with the master instance, modifies the new sort string table file number written by the slave instance to the sort string table file number synchronized with the master instance, and finally deletes the old sort string table file.

[0162] 4. DPU-side hardware-accelerated compaction mechanism:

[0163] 1. Use the Intel DPDK library to call the hardware accelerator on the DPU device to implement compression and decompression functions based on the DEFLATE algorithm and name it DPU_Compress. Add DPU_Compress to the various compression and decompression algorithms in RocksDB. Finally, specify the compression type as DPU_Compress during compaction.

[0164] 2. Use the Intel DPDK library to call the hardware accelerator on the DPU device to implement the cyclic redundancy check (CRC) algorithm and replace the original RocksDB CRC check algorithm with the hardware-accelerated CRC algorithm.

[0165] The schematic diagram of the DPU hardware acceleration compaction mechanism is as follows Figure 10 As shown in the figure, the compaction process of the LSM-tree based key-value storage system taking RocksDB as an example includes seven steps: reading the old sorted string table (SSTable) file, decompressing the old SSTable file, CRC checking the old SSTable file, merging the SSTable file content, CRC checking the new SSTable file, compressing the new SSTable file, and writing the new SSTable file.

[0166] In this embodiment, through the Intel DPDK library, the work of decompressing old SSTable files and compressing new SSTable files can be offloaded to the compression and decompression accelerator in the DPU to complete. At the same time, the work of CRC checking old SSTable files and calculating the CRC check value of new SSTable files can be offloaded to the CRC check accelerator in the DPU to complete. In this way, the general computing unit of the DPU only needs to complete the file reading, writing and merging operations, which greatly reduces the computing pressure of the general computing unit of the DPU. It should be noted that the dedicated hardware accelerator and the general computing unit carried by the DPU can execute tasks in parallel and will not restrict each other.

[0167] The embodiment of the present application provides a data processing device, which is applied to a DPU device, such as Figure 11 As shown, the data processing device 30 may include: a first receiving module 301, a processing module 302 and a first sending module 303, wherein:

[0168] The first receiving module 301 is configured to receive compression task metadata from the host side;

[0169] The processing module 302 is configured to compress the compression task metadata by calling the hardware accelerator on the DPU device to obtain compression result metadata;

[0170] The first sending module 303 is configured to send the compression result metadata to the host end.

[0171] In a possible implementation, the processing module 302 is configured to specifically perform the following operations:

[0172] Based on the compression task metadata, obtain the corresponding sort string table SSTable file;

[0173] The new SSTable file generated after decompressing, merging, and sorting the SSTable file is compressed by calling the hardware accelerator on the DPU device to obtain the compression result metadata.

[0174] In one embodiment, the processing module 302 is configured to read the corresponding SSTable file from the storage device through a lightweight network file system.

[0175] In another possible implementation, the processing module 302 is further configured to: write the compression result metadata to the storage device through the lightweight network file system.

[0176] In yet another possible implementation, the processing module 302 is further configured to:

[0177] The slave instance reads the write-before log file of the master instance through the lightweight network file system and synchronizes the write-before log file with the memory table of the master instance according to the write-before log file, wherein the slave instance runs on the DPU device side and the master instance runs on the host side;

[0178] The old SSTable file is read from the instance, and the number of the old SSTable file is added to the prefix of the instance thread number as the number of the generated new SSTable file to obtain the updated metadata;

[0179] The old SSTable file is an SSTable file read from the storage device through a lightweight network file system;

[0180] The above-mentioned first sending module 303 is configured as follows: the slave instance returns the updated metadata to the master instance and retains the old SSTable file.

[0181] In another possible implementation, during network transmission through the lightweight network file system, the IO operations of the data plane are directly passed to the storage device side via the high-speed network-based non-volatile memory standard object (NVMe-oF Target) offloaded on the DPU device side, and the metadata operations of the control plane are processed by the general computing unit on the DPU device side.

[0182] In another possible implementation, the first receiving module 301 is configured to: receive the compressed task metadata transmitted by the host through a remote direct data access (RDMA) process;

[0183] The first sending module 303 is configured to send the compression result metadata to the host through the RDMA process.

[0184] The data processing device 30 of this embodiment can execute the data processing method shown in the above embodiments of this application. The implementation principle is similar and will not be repeated here.

[0185] Figure 12 A structural diagram of a data processing device provided in another embodiment of the present application is shown in FIG. Figure 12 The device 40 shown is applied to the host side and may include: a second sending module 401 and a second receiving module 402, wherein:

[0186] The second sending module 401 is configured to send compression task metadata to the data processing unit DPU device end;

[0187] The second receiving module 402 is configured to receive the compression result metadata sent by the DPU device, wherein the compression result metadata is obtained by the DPU device by calling the hardware accelerator on the DPU device end to compress the compression task metadata.

[0188] In a possible implementation, the apparatus 40 may further include: a processing module configured to update metadata information according to the compression result metadata.

[0189] In another possible implementation, the processing module is configured to specifically perform the following operations:

[0190] The master instance running on the host side reads the number of the new SSTable file from the updated metadata, and obtains the SSTable file number synchronized with the master instance;

[0191] Modify the number of the new SSTable file to the SSTable file number synchronized with the main instance, and delete the old SSTable file.

[0192] The data processing device 40 of this embodiment can execute the data processing method shown in the previous embodiments of this application. The implementation principles are similar and will not be repeated here.

[0193] An embodiment of the present application provides an electronic device comprising: a memory and a processor; and at least one program stored in the memory for execution by the processor. Compared to the prior art, the electronic device can achieve the following: offloading compression (compaction) operations from the host side to the DPU device side for execution, thereby eliminating data movement and reducing network traffic overhead. At the same time, the DPU device side can accelerate the execution efficiency of the compaction operation by calling a hardware accelerator on the DPU device side to compress the compression task metadata, thereby reducing the host side CPU usage, freeing up the host side's computing resources, and improving energy efficiency.

[0194] It should be noted that the electronic device in the embodiment of the present application can be the DPU device end or the host end mentioned above.

[0195] In an alternative embodiment, an electronic device is provided, such as Figure 13 As shown, Figure 13The electronic device 50 shown includes: a processor 501 and a memory 503. The processor 501 and the memory 503 are connected, for example, via a bus 502. Optionally, the electronic device 50 may further include a transceiver 504, which may be used for data exchange between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the number of transceivers 504 is not limited to one, and the structure of the electronic device 50 does not constitute a limitation on the embodiments of the present application.

[0196] The processor 501 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 501 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0197] The bus 502 may include a path for transmitting information between the above components. The bus 502 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 502 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 13 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0198] The memory 503 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0199] The memory 503 is used to store application code for executing the solution of the present application, and the execution is controlled by the processor 501. The processor 501 is used to execute the application code stored in the memory 503 to implement the content shown in the above method embodiment.

[0200] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed on a computer, enables the computer to execute the corresponding contents of the aforementioned method embodiment. Compared with the prior art, by offloading the compression (compaction) operation on the host side to the DPU device side for execution, data movement is eliminated, thereby reducing network traffic overhead. At the same time, the DPU device side can accelerate the execution efficiency of the compaction operation by calling the hardware accelerator on the DPU device side to compress the compression task metadata, thereby reducing the host side CPU usage, freeing up the host side's computing resources, and improving energy efficiency.

[0201] The terms "first," "second," "third," "fifth," "1," "2," and the like (if any) in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequential sequence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the application described herein can be implemented in an order other than that shown or described in the drawings.

[0202] It should be noted that the division of units in the embodiments of the present application is schematic and is merely a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0203] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0204] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) that contain computer-usable program code.

[0205] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer-executable instructions. These computer-executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0206] These processor-executable instructions may also be stored in a processor-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor-readable memory produce an article of manufacture comprising an instruction device that implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0207] These processor-executable instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0208] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0209] The above descriptions are only partial embodiments of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and modifications without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A data processing method, characterized in that: Applied to the data processing unit (DPU) device side, including: Receive compression task metadata from the host; Perform compression on the compression task metadata by calling a hardware accelerator on the DPU device to obtain compression result metadata; Sending the compression result metadata to the host end; The step of compressing the compression task metadata by calling the hardware accelerator on the DPU device to obtain the compression result metadata includes: Based on the compression task metadata, read the corresponding sort string table SSTable file from the storage device through a lightweight network file system; The hardware accelerator on the DPU device is called to decompress, merge, and sort the SSTable files, thereby compressing the generated new SSTable files to obtain the compression result metadata; In the process of transmission through the lightweight network file system, the IO operations of the data plane are directly passed to the storage device side via the high-speed network-based non-volatile memory standard object unloaded on the DPU device side, and the metadata operations of the control plane are processed by the general computing unit of the DPU device side.

2. The method according to claim 1, characterized in that The method further comprises: The compression result metadata is written to the storage device through the lightweight network file system.

3. The method according to claim 2, characterized in that The method further comprises: The slave instance reads the write-before log file of the master instance through the lightweight network file system and synchronizes the write-before log file with the memory table of the master instance according to the write-before log file, wherein the slave instance runs on the DPU device side and the master instance runs on the host side; The process of obtaining the compression result metadata includes: The old SSTable file is read from the instance, and the number of the old SSTable file is added to the prefix of the instance thread number as the number of the generated new SSTable file to obtain the updated metadata; The old SSTable file is an SSTable file read from the storage device through a lightweight network file system; Sending the compression result metadata to the host side includes: The slave instance returns the updated metadata to the master instance and retains the old SSTable file.

4. The method according to claim 1, wherein The compressed task metadata transmitted by the host side through a remote direct data access (RDMA) process is received, and the compressed result metadata is sent to the host side through the RDMA process.

5. A data processing method, characterized in that: Applied to the host side, including: Sending compression task metadata to the data processing unit (DPU) device; Receive compression result metadata sent by the DPU device, wherein the compression result metadata is obtained by the DPU device by calling a hardware accelerator on the DPU device to decompress, merge, and sort the sort string table SSTable file to generate a new SSTable file, and the SSTable file is based on the compression task metadata and is read from the storage device through a lightweight network file system; In the process of transmission through the lightweight network file system, the IO operations of the data plane are directly passed to the storage device side via the high-speed network-based non-volatile memory standard object unloaded on the DPU device side, and the metadata operations of the control plane are processed by the general computing unit of the DPU device side.

6. The method according to claim 5, characterized in that The method further comprises: The metadata information is updated according to the compression result metadata.

7. The method according to claim 6, characterized in that The updating of metadata information according to the compression result metadata includes: The master instance running on the host side reads the number of the new SSTable file from the updated metadata, and obtains the SSTable file number synchronized with the master instance; Modify the number of the new SSTable file to the SSTable file number synchronized with the main instance, and delete the old SSTable file.

8. A data processing device, characterized in that: Applied to the data processing unit (DPU) device side, including: A first receiving module is configured to receive compression task metadata from a host side; A processing module is configured to read a corresponding sorted string table SSTable file from a storage device through a lightweight network file system based on the compression task metadata, and compress a new SSTable file generated by decompressing, merging, and sorting the SSTable file by calling a hardware accelerator on the DPU device to obtain compression result metadata; A first sending module is configured to send the compression result metadata to the host end; In the process of transmission through the lightweight network file system, the IO operations of the data plane are directly passed to the storage device side via the high-speed network-based non-volatile memory standard object unloaded on the DPU device side, and the metadata operations of the control plane are processed by the general computing unit of the DPU device side.

9. A data processing device, characterized in that: Applied to the host side, including: A second sending module is configured to send compression task metadata to a data processing unit DPU device end; A second receiving module is configured to receive compression result metadata sent by the DPU device, wherein the compression result metadata is obtained by the DPU device by calling the hardware accelerator on the DPU device to decompress, merge, and sort the sort string table SSTable file to generate a new SSTable file, and the SSTable file is based on the compression task metadata and is read from the storage device through a lightweight network file system; In the process of transmission through the lightweight network file system, the IO operations of the data plane are directly passed to the storage device side via the high-speed network-based non-volatile memory standard object unloaded on the DPU device side, and the metadata operations of the control plane are processed by the general computing unit of the DPU device side.

10. An electronic device, characterized in that: The electronic device comprises: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to: execute the data processing method according to any one of claims 1 to 4, or any one of claims 5 to 7.

11. A processor-readable storage medium, characterized in that: The processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute any one of claims 1 to 4, or any one of claims 5 to 7, the data processing method.

Images