Data processing method and device and computing equipment

Abstract

The invention provides a data processing method, the method is applied to a storage system, the storage system comprises a data processing unit DPU, a computing node and a storage node, the storage node and the computing node are arranged in at least one data center providing cloud service, and the computing node is arranged in the data center. The method comprises the following steps: the DPU obtains a data block to be written from the computing node, and stores the data block to be written in a memory of the DPU; the DPU processes the to-be-written data block to generate a target sorting character string table (SST), and the target SSTable comprises a data block obtained after the to-be-written data block is compressed and verified by the DPU; and the DPU sends the target SSTable to the storage node. The method can improve the resource utilization rate of the server.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of cloud computing, and more particularly, to a data processing method and device and a computing device. BACKGROUND

[0002] Flush operation generally refers to the operation of flushing buffered data from cache or temporary storage to permanent storage or external device. This operation ensures the persistence of data and updates the data in the storage device.

[0003] In a related technical solution, the central processing unit (CPU) on the server (also referred to as host) is responsible for performing the Flush operation described above. For example, a general write operation usually stores data in the CPU cache first, and the Flush operation of the CPU cache is to flush the data in the cache to the main memory or external device to ensure data consistency and reliability.

[0004] The above-mentioned related technical solution needs to occupy the resources of the server, so that the server cannot free up more resources for user use, reducing the resource utilization of the server.

[0005] Therefore, how to improve the resource utilization of the server has become a technical problem to be solved. SUMMARY

[0006] The present application provides a data processing method, device and computing device, which can improve the resource utilization of the server.

[0007] In a first aspect, a data processing method is provided, which is applied to a storage system including a data processing unit (DPU), a computing node and a storage node, wherein the storage node and the computing node are arranged in at least one data center providing cloud services, and the method comprises: the DPU obtains a data block to be written from the computing node and stores the data block to be written in the memory of the DPU; the DPU processes the data block to be written to generate a target sorted string table (SSTable), wherein the target SSTable includes the data block obtained after the DPU performs compression and verification processing on the data block to be written; and the DPU sends the target SSTable to the storage node.

[0008] In the above technical solution, the Flush operation can be offloaded from the server to the data processing unit (DPU), and the DPU flushes the data in the CPU cache to the main memory or external device, so that the server frees up more resources for user use, thereby improving the resource utilization of the server.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, the DPU includes at least one data path accelerator (DPA), the compute node stores a first table, the first table includes the data block to be written, the at least one DPA in the DPU retrieves the data block to be written from the first table, and stores the data block to be written in the memory of the DPU.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, the computing node also stores a second table, which further includes the data block to be written. The at least one DPA stores the data block to be written obtained from the first table in the memory of the DPU, and continues to store the data block to be written obtained from the second table in the memory of the DPU.

[0011] In the above technical solution, the physical resources in the DPU can be fully utilized through a pipeline approach, thereby improving the throughput of the Flush operation.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, the DPU also includes an advanced Reduced Instruction Set Machine Core (ARM Core), which processes the data block to be written to generate the target SSTable.

[0013] In conjunction with the first aspect, in some implementations of the first aspect, the DPU also includes a network interface card (NIC) that sends the target SSTable to the storage node.

[0014] In conjunction with the first aspect, in some implementations of the first aspect, the network interface card (NIC) stores the target SSTable in the memory of the DPU; if the number of SSTables currently being processed by the storage node is less than a preset threshold, the NIC sends the target SSTable to the storage node.

[0015] In the above technical solution, the network interface card (NIC) first stores the target SSTable in the memory of the DPU. If the number of SSTables currently being processed by the storage node is less than a preset threshold, then the target SSTable is sent to the storage node. This reduces the merging overhead of remote storage nodes and avoids write pauses at the front end, improving write performance for periodic high write loads.

[0016] In conjunction with the first aspect, in some implementations of the first aspect, the DPU is inserted on the computing node.

[0017] In conjunction with the first aspect, in some implementations of the first aspect, the DPU is inserted on a DPU bearer node, which is connected to the computing node.

[0018] Secondly, a storage system is provided, comprising a data processing unit (DPU), a compute node, and a storage node, wherein the storage node and the compute node are located in at least one data center providing cloud services. The DPU is used to retrieve a data block to be written from the compute node and store the data block in its memory. The DPU is also used to process the data block to be written to generate a target sorted string table (SSTable), wherein the target SSTable includes data blocks obtained by the DPU after compressing and verifying the data block to be written. The DPU is also used to send the target SSTable to the storage node.

[0019] In conjunction with the second aspect, in some implementations of the second aspect, the DPU includes at least one data path accelerator (DPA), the compute node stores a first table including the data block to be written, the at least one DPA is used to retrieve the data block to be written from the first table and store the data block to be written in the memory of the DPU.

[0020] In conjunction with the second aspect, in some implementations of the second aspect, the computing node also stores a second table, which further includes the data block to be written. The at least one DPA is specifically used to: store the data block to be written obtained from the first table in the memory of the DPU, and continue to store the data block to be written obtained from the second table in the memory of the DPU.

[0021] In conjunction with the second aspect, in some implementations of the second aspect, the DPU also includes an advanced RISC machine core (ARM Core) for processing the data block to be written and generating the target SSTable.

[0022] In conjunction with the second aspect, in some implementations of the second aspect, the DPU also includes a network interface card (NIC) for sending the target SSTable to the storage node.

[0023] In conjunction with the second aspect, in some implementations of the second aspect, the network card is specifically used to: store the target SSTable in the memory of the DPU; and send the target SSTable to the storage node when the number of SSTables currently being processed by the storage node is less than a preset threshold.

[0024] In conjunction with the second aspect, in some implementations of the second aspect, the DPU is inserted on the computing node.

[0025] In conjunction with the second aspect, in some implementations of the second aspect, the DPU is inserted on a DPU bearer node, which is connected to the computing node.

[0026] It should be understood that for the beneficial effects of the second aspect and its various implementations, please refer to the first aspect and its various implementations; they will not be repeated here.

[0027] Thirdly, a computing device is provided, including a processor and a memory, and optionally, an input / output interface. The processor controls the input / output interface to send and receive information, the memory stores a computer program, and the processor retrieves and runs the computer program from the memory, causing the methods of the second aspect or any possible implementation thereof to be executed.

[0028] Optionally, the processor can be a general-purpose processor, which can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, integrated circuit, etc.; when implemented in software, the processor can be a general-purpose processor that reads software code stored in memory. This memory can be integrated into the processor or located outside the processor and exist independently.

[0029] Fourthly, a computing device cluster is provided, including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the computing device cluster to perform the method of the second aspect or any possible implementation thereof.

[0030] Fifthly, a chip is provided that acquires and executes instructions to implement the method in the second aspect or any possible implementation of the second aspect.

[0031] Alternatively, as one implementation, the chip includes a processor and a data interface, through which the processor reads instructions stored in memory and executes the method in the second aspect or any possible implementation of the second aspect.

[0032] Alternatively, as one implementation, the chip may further include a memory storing instructions, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to perform the method in the second aspect or any possible implementation of the second aspect.

[0033] In a sixth aspect, a computer program product containing instructions is provided, which, when executed by a computing device, cause the computing device to perform the method of the second aspect or any possible implementation thereof.

[0034] In a seventh aspect, a computer program product containing instructions is provided, which, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the methods described in the second aspect or any possible implementation thereof.

[0035] Eighthly, a computer-readable storage medium is provided, including computer program instructions that, when executed by a computing device, perform the method as described in the second aspect or any possible implementation thereof.

[0036] As examples, these computer-readable storage devices include, but are not limited to, one or more of the following: read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), flash memory, electrically EPROM (EEPROM), and hard drive.

[0037] Alternatively, as one implementation method, the aforementioned storage medium can specifically be a non-volatile storage medium.

[0038] A ninth aspect provides a computer-readable storage medium including computer program instructions that, when executed by a cluster of computing devices, perform the method as described in the second aspect or any possible implementation thereof.

[0039] As examples, these computer-readable storage devices include, but are not limited to, one or more of the following: read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), flash memory, electrically EPROM (EEPROM), and hard drive.

[0040] Alternatively, as one implementation method, the aforementioned storage medium can specifically be a non-volatile storage medium. Attached Figure Description

[0041] Figure 1 This is a schematic block diagram of a cloud scenario applicable to embodiments of this application.

[0042] Figure 2 This is a schematic diagram of the architecture of a storage system provided in an embodiment of this application.

[0043] Figure 3 This is a schematic diagram of another storage system architecture provided in an embodiment of this application.

[0044] Figure 4 This is a schematic flowchart illustrating a data processing method provided in an embodiment of this application.

[0045] Figure 5 This is a schematic block diagram of the architecture of a DPU provided in an embodiment of this application.

[0046] Figure 6 This is a schematic diagram of the DPU performing a flush operation according to an embodiment of this application.

[0047] Figure 7 This is a schematic diagram of another DPU performing a flush operation provided in an embodiment of this application.

[0048] Figure 8 This is a schematic flowchart illustrating a flood control strategy for a Flush operation provided in an embodiment of this application.

[0049] Figure 9 This is a schematic block diagram illustrating the overall process of a DPU performing a Flush operation, as provided in an embodiment of this application.

[0050] Figure 10 This is a schematic diagram of the architecture of a computing device 1500 provided in an embodiment of this application.

[0051] Figure 11 This is a schematic diagram of the architecture of a computing device cluster provided in an embodiment of this application.

[0052] Figure 12 This is a schematic diagram showing the connection between computing devices 1500A and 1500B via a network, as provided in the embodiments of this application. Detailed Implementation

[0053] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0054] This application will present various aspects, embodiments, or features relating to systems comprising multiple devices, components, modules, etc. It should be understood and appreciated that individual systems may include additional devices, components, modules, etc., and / or may not include all devices, components, modules, etc. discussed in conjunction with the accompanying drawings. Furthermore, combinations of these approaches are also possible.

[0055] Furthermore, in the embodiments of this application, the words "exemplary," "for example," etc., are used to indicate that they are examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the term "exemplary" is intended to present the concept in a concrete manner.

[0056] In the embodiments of this application, "corresponding" and "corresponding" can sometimes be used interchangeably. It should be noted that when the distinction is not emphasized, their intended meanings are consistent.

[0057] The business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0058] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0059] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0060] For ease of description, the relevant concepts involved in the embodiments of this application will be explained below.

[0061] 1. Log-structured merge tree (LSM-Tree)

[0062] Log Structure Merge Tree (LSM-Tree), also known as Log Structure Merge Tree, is a data structure designed to optimize write performance, commonly used to address the efficiency issues of random disk read / write. Its core feature lies in leveraging the performance advantages of sequential disk writes by delaying disk writes and batch merging data to achieve efficient write operations while maintaining read operation efficiency as much as possible. It helps reduce perceived latency and maximizes the aggregate I / O bandwidth of the storage device by batching write operations in memory and then sequentially flushing them to the storage device.

[0063] An LSM-Tree typically consists of two parts: a memory component and a disk component. Write requests are first written to the memory component. When the memory component is full, it is flushed to disk to form a new disk component. Multiple components on disk are merged in the background to form a tree-like hierarchical structure. With the popularity of split architectures in data centers, more and more LSM-trees are being deployed on split structures. In this structure, the memory component remains on the compute node, while the disk component is stored on remote storage media, such as remote file storage, block storage, object storage, or split memory.

[0064] In memory, an LSM-Tree typically has a MemTable and an Immutable MemTable, which provide interfaces for writing, reading, and deleting key-value pairs. When the MemTable reaches a certain data volume, it is converted into an Immutable MemTable, which undergoes calculations, compression, and other operations to form an SSTable before being flushed to disk. On disk, the storage unit of an LSM-Tree is a hierarchical SSTable, divided into layers L0 to LN. Starting from layer L0, whenever the data volume of a layer reaches a certain threshold, an LSM-Tree merging operation is triggered to form an L1 layer SSTable, and so on.

[0065] 2. Data processing unit (DPU)

[0066] The Data Processing Unit (DPU) is a new generation of processors designed for data-centric computing, playing an increasingly important role in modern computing. It is currently widely used for offloading tasks related to storage, networking, and security.

[0067] 3. Flush operation

[0068] A flush operation typically refers to the process of flushing buffered data from a cache or temporary storage to permanent storage or an external device. This operation ensures data persistence and updates the data on the storage device.

[0069] In one related technical solution, the central processing unit (CPU) on the server is responsible for performing the aforementioned flush operation. For example, a typical write operation usually first stores the data in the CPU cache, and the CPU cache flush operation is the CPU flushing the data in the cache to main memory or external devices to ensure data consistency and reliability.

[0070] The aforementioned technical solutions require server resources, preventing the server from freeing up more resources for users and reducing server resource utilization.

[0071] In view of this, embodiments of this application provide a data processing method that can offload the above-mentioned flush operation from the server to the data processing unit (DPU), and have the DPU flush the data in the CPU cache to the main memory or external devices, so that the server can free up more resources for users, thereby improving the resource utilization of the server.

[0072] In one possible implementation, the method provided in this application embodiment can be applied to cloud service scenarios. For ease of description, the following will first combine... Figure 1 It provides a detailed description of cloud service scenarios.

[0073] Figure 1 This is a schematic block diagram illustrating a cloud scenario applicable to embodiments of this application. For example... Figure 1 As shown, the cloud scenario may include: cloud management platform 110, Internet 120, and client 130.

[0074] like Figure 1 As shown, the cloud management platform 110 is used to manage the infrastructure that provides multiple cloud services. The infrastructure includes multiple cloud data centers, each cloud data center includes multiple servers, and each server includes cloud service resources to provide corresponding cloud services to tenants.

[0075] The cloud management platform 110 can be located in a cloud data center and provides access interfaces (such as user interfaces or application program interfaces, APIs). Tenants can use client 130 to remotely access the cloud management platform 110, register a cloud account and password, and log in. After successful authentication of the cloud account and password, the tenant can further select and purchase virtual machines of specific specifications (processor, memory, disk) on the cloud management platform 110. After successful purchase, the cloud management platform 110 provides the remote login account and password for the purchased virtual machine, allowing client 130 to remotely log in and install and run the tenant's applications. Therefore, tenants can create, manage, log in to, and operate virtual machines in the cloud data center through the cloud management platform 110. Virtual machines can also be referred to as Elastic Compute Service (ECS) or Elastic Instances (different cloud service providers may use different names).

[0076] It should be understood that cloud service tenants can be individuals, businesses, schools, hospitals, government agencies, etc.

[0077] The cloud management platform 110 includes, but is not limited to, a user console, compute management services, network management services, storage management services, authentication services, and image management services. The user console provides an interface or API for interaction with tenants. The compute management services manage servers running virtual machines and containers, as well as bare metal servers. The network management services manage network services (such as gateways and firewalls). The storage management services manage storage services (such as data bucket services). The authentication services manage tenant account passwords. The image management services manage virtual machine images. Tenants can log in to the cloud management platform 110 via client 130 and the internet 120 to manage their rented cloud services.

[0078] Let's combine the following... Figures 2-3 The architecture of the storage system applicable to the embodiments of this application will be described in detail. It should be understood that... Figures 2-3 The examples are merely to help those skilled in the art understand the embodiments of this application, and are not intended to limit the embodiments of the application to... Figures 2-3 The specific numerical values ​​or specific scenarios illustrated. Those skilled in the art will understand based on... Figures 2-3 The examples given below can obviously be modified or varied in various ways, and such modifications and variations also fall within the scope of the embodiments of this application.

[0079] For example, Figure 2 This is a schematic diagram of the architecture of a storage system provided in an embodiment of this application. For example... Figure 2As shown, the storage system includes compute nodes, DPUs inserted on the compute nodes, and a remote storage pool, wherein the storage pool includes storage nodes.

[0080] The aforementioned compute nodes and storage nodes can be located in at least one data center providing cloud services. For example, a compute node can be... Figure 1 The servers shown are in a cloud data center. Similarly, storage nodes could be, for example, servers in a cloud data center. Figure 1 The disks in the servers of the cloud data center shown.

[0081] The aforementioned compute nodes (also known as hosts) run virtual instances or applications that are available for remote use by tenants. These virtual instances may include, but are not limited to, virtual machines (VMs) and containers.

[0082] As an example, there is a fixed correspondence between the DPU and the computing node. The DPU can be inserted into the computing node, and the DPU connects to the computing node through the connection channel between the DPU and the computing node.

[0083] It should be understood that the connection channel between the DPU and the computing node can be implemented through bus technology, high-speed interconnection network, or other communication methods. This application embodiment does not specifically limit this.

[0084] As an example, the bus technology may include, but is not limited to: Extended Peripheral Component Interconnect Express (PCIE) bus, Compute Express Link (CXL), Unified Bus (UB), Remote Direct Memory Access (RDMA), etc.

[0085] For example, Figure 3 This is a schematic diagram of another storage system architecture provided in an embodiment of this application. For example... Figure 3 As shown, the storage system includes compute nodes, DPU bearer nodes, and remote storage nodes. Each DPU bearer node contains at least two DPUs.

[0086] It should be understood that Figure 3The architecture shown is a pooled architecture, in which there is no fixed correspondence between DPUs and compute nodes, and multiple compute nodes can share at least two DPUs included in the resource pool. For example, the at least two DPUs are inserted into one or more physical nodes (which can also be called DPU-bearing nodes), and the at least two DPUs constitute a resource pool.

[0087] As an example, Figure 3 The example below uses two DPUs (the first DPU and the second DPU) for illustration. Figure 3 As shown, the first DPU and the second DPU are inserted into the DPU bearer node.

[0088] As an example, the DPU bearer node can be located in at least one data center that provides cloud services; for example, it could be... Figure 1 The servers in the cloud data center shown.

[0089] The aforementioned DPU bearer node and computing node can be connected via a high-speed interconnection network, or they can be connected via other communication methods. This application embodiment does not specifically limit this.

[0090] The first DPU and the second DPU in the aforementioned DPU bearer node can be connected via bus technology, high-speed interconnection network, or other communication methods. This application embodiment does not specifically limit this.

[0091] It should be understood that Figure 3 The computing nodes in Figure 2 The computation nodes are the same. Figure 3 Storage nodes in Figure 2 The storage nodes are the same; please refer to [link / reference] for details. Figure 2 The description in the text will not be repeated here.

[0092] The following is combined Figure 4 This application provides a detailed description of a data processing method based on an embodiment. It should be understood that... Figure 4 The examples are merely to help those skilled in the art understand the embodiments of this application, and are not intended to limit the embodiments of the application to... Figure 4 The specific numerical values ​​or specific scenarios illustrated. Those skilled in the art will understand based on... Figure 4 The examples given below can obviously be modified or varied in various ways, and such modifications and variations also fall within the scope of the embodiments of this application.

[0093] For example, Figure 4 This is a schematic flowchart illustrating a data processing method provided in an embodiment of this application. Figure 4As shown, the method may include steps 410-430, which will be described in detail below.

[0094] It should be understood that Figure 4 The method can be applied to any of the above-mentioned storage systems, and the embodiments of this application do not specifically limit it.

[0095] Step 410: The DPU retrieves the immutable memory table Imm MemTable from the compute node.

[0096] In this embodiment of the application, the DPU can obtain the immutable memory table stored in the memory of the computing node, which stores key-value (KV) pairs.

[0097] As an example, the Immutable MemTable mentioned above can also be simply referred to as Imm MemTable.

[0098] For example, the Imm MemTable mentioned above corresponds to the first and second tables mentioned above.

[0099] For example, such as Figure 5 As shown, when data is written to the LSM tree, it is first stored in a memory table (MemTable) in the compute node's memory. The MemTable is an in-memory data structure used to store the most recently updated data. When the amount of data in the MemTable reaches a certain threshold, the MemTable is converted to an Imm MemTable.

[0100] One possible implementation is, such as Figure 5 As shown, the DPU may include a data path accelerator (DPA). In this embodiment, the DPA in the DPU can obtain the aforementioned ImmMemTable from the compute node.

[0101] It should be understood that a DPA is a highly programmable embedded processor residing within a DPU, specifically built for network-intensive, low-computational-load tasks. It allows users to programmatically offload and accelerate network-centric data processing tasks, such as device emulation, congestion control, and custom protocols. Through DPA, the DPU can handle network traffic more efficiently, reducing its reliance on general-purpose multi-core processors, thereby improving overall performance and efficiency.

[0102] For example, to offload the data plane operation of Flush to the DPU, the DPU needs to have access to the compute node's memory. This requires registering the compute node's memory index with the DPA in the DPU during the Init phase. When a Flush operation is triggered, the DPA's hardware thread can sequentially traverse each key-value pair in the ImmMemTable based on the starting address of the ImmMemTable in the compute node's memory, generating corresponding temporary data blocks, and copying these temporary data blocks into the DPU's memory.

[0103] It should be understood that in computer and software systems, the Init phase typically refers to an initialization phase during the system or program startup process. During this phase, the system or program performs a series of necessary settings and preparatory tasks to ensure proper subsequent operation. These preparatory tasks may include loading necessary modules, configuring environment variables, checking system resources, and starting critical services.

[0104] Step 420: The DPU constructs a sorted string table SSTable based on the acquired immutable memory table Imm MemTable.

[0105] In this embodiment of the application, each KV pair in the ImmMemTable is traversed sequentially according to the starting address of the ImmMemTable in the memory of the computing node, and corresponding temporary data blocks are generated. These temporary data blocks are then stored in the temporary table (Tmp Table) in the memory of the DPU.

[0106] It should be understood that Tmp Table is a commonly used temporary table in databases.

[0107] In this embodiment of the application, the DPU can compress the data in the Tmp Table, perform cyclic redundancy check (CRC) calculation to obtain a compressed data block, generate an index block based on the offset of the compressed data block, and finally append the filter block to the end of the compressed data block and the index block to generate a sorted string table (SSTable).

[0108] It should be understood that SSTable is an immutable, ordered, and persistent data structure for storing large numbers of ordered key-value pairs. SSTable stores key-value pairs in ascending order of the keys, so that each block in the SSTable file contains multiple ordered key-value pairs. Adjacent keys may have long common prefixes; to save space, SSTable employs optimization techniques such as prefix compression. That is, at the end of the SSTable, there is a block index for quickly locating each block within the SSTable.

[0109] One possible implementation is, such as Figure 5 As shown, the DPU may include an advanced reduced instruction set computing machine core (ARM Core). In this embodiment, the ARM Core in the DPU can perform compression and CRC calculation on the data in the Tmp Table to obtain compressed data blocks. An index block is generated based on the offset of the compressed data block, and then the filter block is finally appended to the end of the compressed data block and the index block to generate the aforementioned SSTable. Since the DPA has a lower clock frequency and fewer computing units, the compression and verification operations are offloaded to the Arm Core in the DPU. Compared to the DPA, the Arm Core has a higher clock frequency, enabling decompression and verification throughput comparable to that of a computing node, and can also call the hardware accelerator on the DPU to accelerate these data-intensive calculations, thereby achieving better performance.

[0110] For example, the ARM Core in the DPU compresses and verifies the temporary data blocks in the Tmp Table generated by DPA to generate an SSTable. For instance, the ARM Core can compress the temporary data blocks generated by DPA to generate compressed data blocks, generate an index block based on the offset of the compressed data block, and finally append the filter block to the end of the compressed data block and the index block to generate the SSTable. That is, the SSTable includes the compressed data block, the index block, and the filter block in sequence.

[0111] Optionally, in some embodiments, the processes of data compression and CRC calculation in the Tmp Table are delegated to the ARM Core in the DPU, introducing additional interaction overhead between the DPA and the ARM Core. To reduce this interaction overhead, a batch approach can be used, where the DPA generates a batch of data blocks and then submits them to the Arm Core for processing in batches to optimize overall throughput.

[0112] Step 430: The DPU writes the generated SSTable to the remote storage node.

[0113] In this embodiment, the DPU can write the generated SSTable to a remote storage node after it is generated. For example, the SSTable can be sent to a remote memory or storage device.

[0114] One possible implementation is, such as Figure 5 As shown, the DPU may include a remote direct memory access network interface controller (RDMA NIC). In this embodiment, the DPU can write the generated SSTable to a remote storage node via the RDMA NIC.

[0115] It should be understood that an RDMA NIC is a network interface card that supports RDMA technology, allowing applications to bypass the operating system kernel and directly access the memory of a remote computer in a zero-copy manner. This technology can significantly reduce network latency, improve bandwidth utilization, and reduce CPU load. In the architecture of a DPU, RDMA NICs are used to accelerate data transmission within the data center, improving overall performance and efficiency.

[0116] It should also be understood that an RDMA NIC can also be called an RDMA network adapter or an RDMA network interface card.

[0117] For example, the RDMA NIC in the DPU can directly call the relevant file system or block storage interface to write data in the SSTable to the remote storage node.

[0118] For example, if the remote storage node is a remote memory server, Arm Core can establish a connection with the remote memory server and register a memory region (MR) during the Init phase. The DPU can store the SSTable generated during the flush execution in the corresponding MR region in the DPU's memory. Then, the DPU writes the data blocks in the SSTable to the remote storage node through a one-sided RDMA operation.

[0119] In the above technical solution, the DPU is used to perform the flush operation of the computing node, thereby releasing the CPU resources of the computing node, so that the computing node can free up more CPU resources for users, thereby improving the utilization rate of the computing node's CPU resources.

[0120] Optionally, embodiments of this application also propose a two-dimensional parallel processing procedure for the Flush operation, thereby improving the performance of the DPU in performing the Flush operation.

[0121] For example, Figure 6 (a) in the diagram represents a flush process in which data blocks within an Imm MemTable are scheduled in an ordered manner. For a single data block within an Imm MemTable, there are data dependencies between the block's generation, compression, verification, and flushing processes. For example, ... Figure 6 In step (a), the data block contained in ImmMemTable i in the compute node is first copied to the Tmp Table in the DPU via the DPA in the DPU. Then, the ARM Core in the DPU compresses the data in the Tmp Table and performs CRC calculation to obtain the SSTable. Finally, the RDMA NIC in the DPU flushes the SSTable to the remote storage node. This process is repeated for the data block contained in ImmMemTable i+1. Due to the ordered scheduling of data blocks, only one type of hardware resource in the DPU is used in each stage, resulting in low resource utilization.

[0122] For example, Figure 6 (b) in this embodiment illustrates a pipelined parallel processing procedure. In reality, a data block may have data dependencies, but different data blocks do not have overlapping key value ranges, and therefore there are no data dependencies between them. Therefore, it is unnecessary to schedule data blocks in an ordered manner. For example, as... Figure 6In (b), the DPA in the DPU copies the data blocks contained in Imm MemTable i to the DPU's Tmp Table i. Then, the DPA can continue to process the data blocks contained in Imm MemTable i+1, for example, copying the data blocks contained in Imm MemTable i+1 to the DPU's Tmp Table i+1, and so on. Similarly, the ARMCore in the DPU compresses the data in Tmp Table i and performs CRC calculations to obtain SSTable i. After the DPA in the DPU copies the data blocks contained in Imm MemTable i+1 to the DPU's Tmp Table i+1, the ARMCore can continue to compress the data in Tmp Table i and perform CRC calculations to obtain SSTable i+1, and so on. Likewise, when the ARMCore obtains SSTable i, the RDMA NIC in the DPU flushes SSTable i to a remote storage node. Then, when the ARMCore obtains SSTable i+1, the RDMA NIC can continue to flush SSTable i+1 to a remote storage node, and so on.

[0123] In the aforementioned pipelined parallel processing, the DPU divides the processing steps of a data block in an Imm MemTable into three parts, which are then executed by three different physical components within the DPU: the DPA, the ARM Core, and the RDMA NIC. These three physical components operate independently, without resource sharing. Furthermore, because there are no data dependencies between data blocks in different Imm MemTables, the pipelined approach allows for full utilization of the DPU's physical resources, thereby improving the throughput of the flush operation.

[0124] In some embodiments, due to the different computing capabilities of different heterogeneous units, the execution times of their respective steps are not necessarily perfectly matched. If any step in the pipeline takes too long to execute, it will cause a large pipeline cavitation, resulting in the idleness of certain hardware resources. For example, the throughput of data blocks generated by a single DPA thread in the DPU is difficult to keep up with the compression throughput of a single ARM Core in the DPU. Therefore, this application provides a data parallel processing procedure for DPA, which can utilize multiple DPA threads in the DPU to generate data blocks in parallel to match the compression speed of the ARM Core, thereby avoiding large pipeline cavitation, avoiding the idleness of certain hardware resources in the DPU, and further improving the utilization rate of hardware resources in the DPU.

[0125] For example, such asFigure 7 As shown, before initiating a flush task, the CPU first divides the key-value pairs contained in the Imm MemTable into finer-grained key-value ranges. Then, it starts multiple DPA threads in the DPU to process the different key-value ranges in the Imm MemTable in parallel, in order to achieve line-speed matching with the ARM Core. For example, as Figure 7 As shown, two DPA threads in the DPU copy the data blocks contained in Imm MemTable i to Tmp Table i in parallel. Then, the two DPA threads in the DPU can continue to copy the data blocks contained in Imm MemTable i+1 to Tmp Table i+1 in parallel, and so on. Similarly, the ARM Core in the DPU compresses the data in Tmp Table i and performs CRC calculation to obtain SSTable i. Then, while the two DPA threads in the DPU are copying the data blocks contained in Imm MemTable i+1 to Tmp Table i+1 in parallel, the ARM Core can continue to compress the data in Tmp Table i and perform CRC calculation to obtain SSTable i+1, and so on. Likewise, when the ARM Core obtains SSTable i, the RDMA NIC in the DPU flushes SSTable i to the remote storage node. Then, when the ARM Core obtains SSTable i+1, the RDMA NIC can continue to flush SSTable i+1 to the remote storage node, and so on.

[0126] It should be understood that since the throughput of the RDMA NIC is higher than the maximum compression throughput of 16 ARM Cores and the memory access throughput of DPA, the RDMA NIC will not become a bottleneck restricting the flush operation.

[0127] In some embodiments, under normal circumstances, after the DPU generates an SSTable, it will directly flush it to the remote storage node. If the flush operation is too fast, the remote storage node will have too many SSTables in the L0 layer. This phenomenon is called L0 Stall. When the compute node receives a message from the remote storage node indicating that L0 Stall has occurred, it limits the write traffic to the front end, thereby causing a write pause in the front end.

[0128] In view of this, embodiments of this application provide a flood prevention strategy for flush operations. When a Stall occurs at the L0 layer of a remote storage node, this flood prevention strategy can reduce the merging overhead of the remote storage node and avoid write pauses at the front end, thereby improving write performance for periodic high write loads.

[0129] For example, such as Figure 8 As shown, if the compute node does not detect a Stall at the L0 layer of the remote storage node, it can follow the normal flush operation process. After the DPU generates an SSTable, it will directly flush it to the remote storage node. If the compute node detects a Stall at the L0 layer of the remote storage node, it sends a flush message with additional information to the DPU. After the DPA thread and ARM Core execution thread in the DPU perform the normal flush process (e.g., process the ImmMemTable and generate an SSTable), they do not pass or flush the generated SSTable to the remote storage node. Instead, they use the DPU's memory to cache the generated SSTable until the Stall situation at the L0 layer of the remote storage node is alleviated, at which point the SSTable stored in the DPU's memory is flushed to the remote storage node.

[0130] Optional, such as Figure 8 As shown, during the process of the DPU caching the generated SSTable into its memory, it can also use its own computing power to perform compression from L0 to L1 layers until the L0 stall condition of the remote storage node is alleviated, at which point the normal flush process resumes. This effectively reduces the harm of L0 stall and makes fuller use of the DPU's computing power.

[0131] Optionally, in some embodiments, for read operations, since the SSTable is partially cached in the DPU's memory, read performance can be improved to some extent. However, since this increases the number of unordered SSTables in the DPU's memory, this embodiment constructs a global index to accelerate lookup performance.

[0132] It should be understood that the above process uses the DPU's memory as a read / write cache for the SSTable, while the SSTable's metadata remains stored on the compute node (also known as the host). For read requests, the system first looks up the memory index on the compute node, then accesses the SSTable's metadata to determine if the data to be read is in the DPU. If the data is not in the DPU, it can be retrieved directly from the remote storage node. If the data is in the DPU, it can be retrieved directly from the DPU.

[0133] The following is combined Figure 9 This application provides a detailed description of the overall process of the DPU performing the Flush operation in the embodiments of this application. It should be understood that... Figure 9 The examples are merely to help those skilled in the art understand the embodiments of this application, and are not intended to limit the embodiments of the application to... Figure 9 The specific numerical values ​​or specific scenarios illustrated. Those skilled in the art will understand based on... Figure 9 The examples given below can obviously be modified or varied in various ways, and such modifications and variations also fall within the scope of the embodiments of this application.

[0134] For example, Figure 9 This is a schematic block diagram illustrating the overall process of a DPU performing a flush operation, as provided in an embodiment of this application. Figure 9 As shown, the Flush Worker in the compute node first connects to the ARM Core main thread on the DPU via the network. When the compute node receives a Flush task, it passes the task information to the ARM Core main thread. After the task is completed on the DPU side, the Flush Worker in the compute node terminates. For the ARM Core main thread on the DPU side, after connecting to the compute node, it waits for the Flush Worker on the compute node to send the Flush task information, and starts the ARM Core processing thread upon receiving the task information. For the ARM Core processing thread on the DPU side, it starts the DPA processing thread in the DPU to copy the data blocks contained in the Imm MemTable in the compute node to the Tmp Table in the DPU. This ARM Core processing thread can perform compression, CRC calculation, etc., on the data blocks in the Tmp Table to generate an SSTable. This ARM Core processing thread can also pass the generated SSTable to the RDMANIC in the DPU, and then transmit the SSTable to the remote storage node through the RDMA NIC.

[0135] The method provided in this application can be executed by a computing device, which can also be referred to as a computer system. It includes a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as processing units, memory, and memory control units; the functions and structure of this hardware will be described in detail later. The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux, Unix, Android, iOS, or Windows. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software. Optionally, the computer system can be a handheld device such as a smartphone, or a terminal device such as a personal computer; this application does not particularly limit this, as long as the method provided in this application can be used. The executing entity of the method provided in this application can be a computing device, or a functional module within the computing device capable of calling and executing programs.

[0136] The following is combined Figure 10 This application provides a detailed description of a computing device provided in an embodiment.

[0137] Figure 10 This is a schematic diagram of the architecture of a computing device 1500 provided in an embodiment of this application. The computing device 1500 can be a server, a computer, or other device with computing capabilities. Figure 10 The computing device 1500 shown includes at least one processor 1510 and a memory 1520.

[0138] It should be understood that this application does not limit the number of processors and memories in the computing device 1500.

[0139] The processor 1510 executes instructions in the memory 1520, causing the computing device 1500 to implement the method provided in this application. Alternatively, the processor 1510 executes instructions in the memory 1520, causing the computing device 1500 to implement the various functional modules provided in this application, thereby implementing the method provided in this application.

[0140] Optionally, the computing device 1500 also includes a communication interface 1530. The communication interface 1530 uses a transceiver module, such as, but not limited to, a network interface card or a transceiver, to enable communication between the computing device 1500 and other devices or communication networks.

[0141] Optionally, the computing device 1500 further includes a system bus 1540, wherein the processor 1510, memory 1520, and communication interface 1530 are respectively connected to the system bus 1540. The processor 1510 can access the memory 1520 through the system bus 1540; for example, the processor 1510 can perform data read / write or code execution in the memory 1520 through the system bus 1540. The system bus 1540 is a peripheral component interconnect express (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus 1540 is divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 10 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0142] In one possible implementation, the processor 1510 primarily functions to interpret the instructions (or code) of a computer program and process data within the computer software. The instructions of the computer program and the data within the computer software can be stored in memory 1520 or cache 1516.

[0143] Optionally, processor 1510 may be an integrated circuit chip with signal processing capabilities. By way of example and not limitation, processor 1510 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor, etc. For example, processor 1510 may be a central processing unit (CPU).

[0144] Optionally, each processor 1510 includes at least one processing unit 1512 and a memory control unit 1514.

[0145] Optionally, the processing unit 1512, also known as the core, is the most important component of the processor. The processing unit 1512 is manufactured from single-crystal silicon using a specific production process. All calculations, command reception, command storage, and data processing are performed by the core. Each processing unit independently executes program instructions, utilizing parallel computing capabilities to accelerate program execution. Various processing units have fixed logical structures; for example, a processing unit includes logical units such as a Level 1 cache, a Level 2 cache, an execution unit, an instruction-level unit, and a bus interface.

[0146] In one implementation example, the memory control unit 1514 controls the data interaction between the memory 1520 and the processing unit 1512. Specifically, the memory control unit 1514 receives memory access requests from the processing unit 1512 and controls access to memory based on the memory access requests. By way of example and not limitation, the memory control unit is a device such as a memory management unit (MMU).

[0147] In one implementation example, each memory control unit 1514 addresses the memory 1520 via the system bus. An arbitrator is configured in the system bus. Figure 10 (Not shown in the image), the arbitrator is responsible for handling and coordinating competing accesses of multiple processing units 1512.

[0148] In one implementation example, the processing unit 1512 and the memory control unit 1514 are connected via internal chip connection lines, such as address lines, thereby enabling communication between the processing unit 1512 and the memory control unit 1514.

[0149] Optionally, each processor 1510 also includes a cache 1516, which is a buffer for data exchange (called a cache). When the processing unit 1512 needs to read data, it first looks for the required data in the cache. If the data is found, it is executed directly; otherwise, it looks for the data in memory. Since the cache operates much faster than memory, its purpose is to help the processing unit 1512 run faster.

[0150] The memory 1520 provides runtime space for processes in the computing device 1500. For example, the memory 1520 stores the computer program (specifically, the program code) used to generate the process. After the computer program is run by the processor to generate a process, the processor allocates corresponding storage space for the process in the memory 1520. Furthermore, the aforementioned storage space further includes text segments, initialized data segments, bit initialized data segments, stack segments, heap segments, etc. The memory 1520 stores data generated during the process's execution, such as intermediate data or process data, in the aforementioned process-specific storage space.

[0151] Optionally, the memory, also known as RAM, is used to temporarily store the data processed by the processor 1510, as well as data exchanged with external storage devices such as hard disks. As long as the computer is running, the processor 1510 will load the data that needs to be processed into RAM for processing, and after the processing is completed, the processing unit 1512 will send the result out.

[0152] By way of example and not limitation, memory 1520 is volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory is read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory is random access memory (RAM) used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DRRAM). It should be noted that the memory 1520 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0153] The above-described structure of the computing device 1500 is merely illustrative and is not intended to limit the application. The computing device 1500 in this application includes various hardware components found in existing computer systems. For example, the computing device 1500 may also include other memories besides the memory 1520, such as disk storage. Those skilled in the art should understand that the computing device 1500 may also include other devices necessary for normal operation. Furthermore, depending on specific needs, those skilled in the art should understand that the computing device 1500 may also include hardware devices for implementing other additional functions. Moreover, those skilled in the art should understand that the computing device 1500 may only include the devices necessary for implementing the embodiments of this application, and may not necessarily include... Figure 10 All the devices shown.

[0154] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device may be a server. In some embodiments, the computing device may also be a desktop computer, a laptop computer, or a smartphone, or other terminal device.

[0155] like Figure 11 As shown, the computing device cluster includes at least one computing device 1500. The memory 1520 of one or more computing devices 1500 in the computing device cluster may store the same instructions for performing the methods described above.

[0156] In some possible implementations, the memory 1520 of one or more computing devices 1500 in the computing device cluster may also each store a portion of the instructions for executing the above-described methods. In other words, a combination of one or more computing devices 1500 can jointly execute the instructions of the above-described methods.

[0157] It should be noted that the memory 1520 in different computing devices 1500 within the computing device cluster can store different instructions, each used to execute a portion of the functions of the aforementioned device. That is, the instructions stored in the memory 1520 of different computing devices 1500 can implement the functions of one or more modules within the aforementioned device.

[0158] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 12 One possible implementation is shown. For example... Figure 12 As shown, the two computing devices 1500A and 1500B are connected via a network. Specifically, they are connected to the network through the communication interfaces in each computing device.

[0159] It should be understood that Figure 12The functions of computing device 1500A shown can also be performed by multiple computing devices 1500. Similarly, the functions of computing device 1500B can also be performed by multiple computing devices 1500.

[0160] In this embodiment, a computer program product containing instructions is also provided. The computer program product may be a software or program product containing instructions capable of running on a computing device or stored on any usable medium. When run on a computing device, it causes the computing device to perform the methods provided above, or causes the computing device to perform the functions of the apparatus provided above.

[0161] In this embodiment, a computer-readable storage medium is also provided. This computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that, when executed on a computing device, cause the computing device to perform the method described above.

[0162] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0163] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0164] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0165] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0166] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0167] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0168] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0169] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, The method is applied to a storage system, which includes a data processing unit (DPU), compute nodes, and storage nodes, wherein the storage nodes and the compute nodes are located in at least one data center providing cloud services, and the method includes: The DPU obtains the data block to be written from the computing node and stores the data block to be written in the memory of the DPU; The DPU processes the data block to be written and generates a target sorted string table SSTable, wherein the target SSTable includes the data block obtained by the DPU after compressing and verifying the data block to be written; The DPU sends the target SSTable to the storage node.

2. The method according to claim 1, characterized in that, The DPU includes at least one Data Path Accelerator (DPA), and the compute node stores a first table containing the data blocks to be written. The DPU obtains the data block to be written from the computing node and stores the data block to be written in the DPU's memory, including: At least one DPA in the DPU retrieves the data block to be written from the first table and stores the data block to be written in the memory of the DPU.

3. The method according to claim 2, characterized in that, The computing node also stores a second table, which further includes the data block to be written. The at least one DPA in the DPU retrieves the data block to be written from the first table and stores the data block to be written in the memory of the DPU, including: The at least one DPA stores the data blocks to be written obtained from the first table in the memory of the DPU, and continues to store the data blocks to be written obtained from the second table in the memory of the DPU.

4. The method according to any one of claims 1 to 3, characterized in that, The DPU also includes an advanced ARM Core (Reduced Instruction Set Computing Machine). The DPU processes the data block to be written and generates a target sorted string table SSTable, including: The ARM Core in the DPU processes the data block to be written and generates the target SSTable.

5. The method according to any one of claims 1 to 4, characterized in that, The DPU also includes a network interface card (NIC). The DPU sends the target SSTable to the storage node, including: The network interface card in the DPU sends the target SSTable to the storage node.

6. The method according to claim 5, characterized in that, The network interface card in the DPU sends the target SSTable to the storage node, including: The network interface card stores the target SSTable in the memory of the DPU; If the number of SSTables currently being processed by the storage node is less than a preset threshold, the network card will send the target SSTable to the storage node.

7. The method according to any one of claims 1 to 6, characterized in that, The DPU is inserted into the computing node.

8. The method according to any one of claims 1 to 6, characterized in that, The DPU is inserted into the DPU bearer node, and the DPU bearer node is connected to the computing node.

9. A storage system, characterized in that, The storage system includes a data processing unit (DPU), compute nodes, and storage nodes, wherein the storage nodes and the compute nodes are located in at least one data center providing cloud services. The DPU is used to obtain data blocks to be written from the computing node and store the data blocks to be written in the memory of the DPU; The DPU is also used to process the data block to be written and generate a target sorted string table SSTable, wherein the target SSTable includes the data block obtained by the DPU after compressing and verifying the data block to be written; The DPU is also used to send the target SSTable to the storage node.

10. The storage system according to claim 9, characterized in that, The DPU includes at least one Data Path Accelerator (DPA), and the compute node stores a first table containing the data blocks to be written. The at least one DPA is used to retrieve the data block to be written from the first table and store the data block to be written in the memory of the DPU.

11. The storage system according to claim 10, characterized in that, The computing node also stores a second table, which further includes the data block to be written. The at least one DPA is specifically used for: The data blocks to be written obtained from the first table are stored in the memory of the DPU, and the data blocks to be written obtained from the second table are also stored in the memory of the DPU.

12. The storage system according to any one of claims 9 to 11, characterized in that, The DPU also includes an advanced ARM Core (Reduced Instruction Set Computing Machine). The ARM Core is used to process the data block to be written and generate the target SSTable.

13. The storage system according to any one of claims 9 to 12, characterized in that, The DPU also includes a network interface card (NIC). The network interface card is used to send the target SSTable to the storage node.

14. The storage system according to claim 13, characterized in that, The network interface card is specifically used for: The target SSTable is stored in the memory of the DPU; If the number of SSTables currently being processed by the storage node is less than a preset threshold, the target SSTable will be sent to the storage node.

15. The storage system according to any one of claims 9 to 14, characterized in that, The DPU is inserted into the computing node.

16. The storage system according to any one of claims 9 to 14, characterized in that, The DPU is inserted into the DPU bearer node, and the DPU bearer node is connected to the computing node.

17. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1 to 8.

18. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster causes the computing device cluster to perform the method as described in any one of claims 1 to 8.

19. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by a cluster of computing devices, perform the method as described in any one of claims 1 to 8.

Images