Data processing method, apparatus, and computing device

Abstract

Provided in the present application is a data processing method. The method is applied to a storage system, and the storage system comprises a data processing unit (DPU), a computing node, and a storage node, the storage node and the computing node being arranged in at least one data center that provides cloud services. The method comprises: the DPU acquires from the computing node a data block to be written, and stores said data block in a memory of the DPU; the DPU processes said data block to generate a target sorted string table (SSTable), wherein the target SSTable comprises a data block obtained after the DPU compresses and verifies the data block to be written; and the DPU sends the target SSTable to the storage node. The method can improve the resource utilization rate of servers.

Description

Method, apparatus, and computing device for data processing

[0001] The present application claims priority to the Chinese patent application No. 202411357212.5, filed on September 27, 2024, entitled "Method, apparatus, and computing device for data processing", the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of cloud computing, and more particularly, to a method, apparatus, and computing device for data processing. BACKGROUND

[0003] Flush operation generally refers to the operation of flushing buffered data from cache or temporary storage to permanent storage or external device. This operation ensures the persistence of data and updates the data in the storage device.

[0004] In a related technical solution, the central processing unit (CPU) on the server (also referred to as host) is responsible for performing the Flush operation. For example, a general write operation usually stores data in the CPU cache first, and the Flush operation of the CPU cache is to flush the data in the cache to the main memory or external device to ensure data consistency and reliability.

[0005] The above-mentioned related technical solution needs to occupy the resources of the server, so that the server cannot free up more resources for user use, reducing the resource utilization rate of the server.

[0006] Therefore, how to improve the resource utilization rate of the server has become a technical problem to be solved urgently. SUMMARY

[0007] The present application provides a method, apparatus, and computing device for data processing, which can improve the resource utilization rate of the server.

[0008] In a first aspect, a data processing method is provided, which is applied to a storage system including a data processing unit (DPU), a computing node, and a storage node, wherein the storage node and the computing node are arranged in at least one data center providing cloud services, and the method comprises: obtaining, by the DPU, a data block to be written from the computing node and storing the data block to be written in a memory of the DPU; processing, by the DPU, the data block to be written to generate a target sorted string table (SSTable), wherein the target SSTable includes a data block obtained by compressing and checking the data block to be written by the DPU; and sending, by the DPU, the target SSTable to the storage node.

[0009] In the above technical solution, the flush operation can be offloaded from the server to the DPU, and the DPU flushes the data in the CPU cache to the main memory or an external device, so that the server releases more resources for user use, thereby improving the resource utilization of the server.

[0010] In combination with the first aspect, in some implementations of the first aspect, the DPU includes at least one data path accelerator (DPA), and the computing node stores a first table including the data block to be written, and the at least one DPA in the DPU obtains the data block to be written from the first table and stores the data block to be written in the memory of the DPU.

[0011] In combination with the first aspect, in some implementations of the first aspect, the computing node further stores a second table including the data block to be written, and the at least one DPA stores the data block to be written obtained from the first table in the memory of the DPU and continues to store the data block to be written obtained from the second table in the memory of the DPU.

[0012] In the above technical solution, the physical resources in the DPU can be fully utilized in a pipelining manner, thereby improving the throughput of the Flush operation.

[0013] In combination with the first aspect, in some implementations of the first aspect, the DPU further includes an advanced reduced instruction set machine core (ARM Core), and the ARM Core in the DPU processes the data block to be written to generate the target SSTable.

[0014] In combination with the first aspect, in some implementations of the first aspect, the DPU further includes a network card, and the network card in the DPU sends the target SSTable to the storage node.

[0015] With reference to the first aspect, in some implementations of the first aspect, the network card stores the target SSTable in the memory of the DPU; and the network card sends the target SSTable to the storage node when the number of SSTables currently processed by the storage node is less than a preset threshold.

[0016] In the above technical solution, the network card first stores the target SSTable in the memory of the DPU, and then sends the target SSTable to the storage node when the number of SSTables currently processed by the storage node is less than a preset threshold. In this way, on the one hand, the merging overhead of the remote storage node can be reduced, and on the other hand, the write pause of the front end can be avoided, and the write performance can be improved for periodic high write load.

[0017] With reference to the first aspect, in some implementations of the first aspect, the DPU is interposed on the computing node.

[0018] With reference to the first aspect, in some implementations of the first aspect, the DPU is interposed on the computing node.

[0019] In a second aspect, a storage system is provided, which includes a data processing unit (DPU), a computing node, and a storage node. The storage node and the computing node are arranged in at least one data center providing cloud services. The DPU is configured to obtain a data block to be written from the computing node and store the data block to be written in the memory of the DPU. The DPU is further configured to process the data block to be written to generate a target SSTable, wherein the target SSTable includes a data block obtained by the DPU after compression and verification processing of the data block to be written. The DPU is further configured to send the target SSTable to the storage node.

[0020] With reference to the second aspect, in some implementations of the second aspect, the DPU includes at least one data path accelerator (DPA), and the computing node stores a first table including the data block to be written. The at least one DPA is configured to obtain the data block to be written from the first table and store the data block to be written in the memory of the DPU.

[0021] With reference to the second aspect, in some implementations of the second aspect, the computing node further stores a second table including the data block to be written. The at least one DPA is specifically configured to store the data block to be written obtained from the first table in the memory of the DPU, and continue to store the data block to be written obtained from the second table in the memory of the DPU.

[0022] With reference to the second aspect, in some implementations of the second aspect, the DPU further includes an advanced reduced instruction set machine core (ARM Core) configured to process the data block to be written to generate the target SSTable.

[0023] With reference to the second aspect, in some implementations of the second aspect, the DPU further includes a network card configured to send the target SSTable to the storage node.

[0024] With reference to the second aspect, in some implementations of the second aspect, the network card is specifically configured to: store the target SSTable in a memory of the DPU; and send the target SSTable to the storage node when a number of SSTables currently processed by the storage node is less than a preset threshold.

[0025] With reference to the second aspect, in some implementations of the second aspect, the DPU is disposed on the computing node.

[0026] With reference to the second aspect, in some implementations of the second aspect, the DPU is disposed on a DPU hosting node connected to the computing node.

[0027] It should be understood that the beneficial effects of the second aspect and the various implementations of the second aspect are the same as those of the first aspect and the various implementations of the first aspect, which will not be repeated here.

[0028] In a third aspect, a computing device is provided, which includes a processor and a memory, and optionally, an input / output interface. The processor is configured to control the input / output interface to receive and send information, and the memory is configured to store a computer program. The processor is configured to call and run the computer program from the memory, so as to execute the method in the second aspect or any possible implementation of the second aspect.

[0029] Optionally, the processor can be a general-purpose processor, which can be implemented by hardware or software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented by software, the processor can be a general-purpose processor, which is configured to read software codes stored in the memory to implement the processor. The memory can be integrated in the processor or exist independently outside the processor.

[0030] In a fourth aspect, a computing device cluster is provided, which includes at least one computing device, and each computing device includes a processor and a memory. The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so as to enable the computing device cluster to execute the method in the second aspect or any possible implementation of the second aspect.

[0031] In a fifth aspect, a chip is provided, which obtains instructions and executes the instructions to implement the method in the second aspect or any possible implementation manner of the second aspect.

[0032] Optionally, as an implementation manner, the chip comprises a processor and a data interface, the processor reads the instructions stored on the memory through the data interface, and executes the method in the second aspect or any possible implementation manner of the second aspect.

[0033] Optionally, as an implementation manner, the chip can further comprise a memory, the memory stores the instructions, and the processor is configured to execute the instructions stored on the memory, and when the instructions are executed, the processor is configured to execute the method in the second aspect or any possible implementation manner of the second aspect.

[0034] In a sixth aspect, a computer program product comprising instructions which, when executed by a computing device, cause the computing device to perform the method in the second aspect or any possible implementation manner of the second aspect.

[0035] In a seventh aspect, a computer program product comprising instructions which, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method in the second aspect or any possible implementation manner of the second aspect.

[0036] In an eighth aspect, a computer-readable storage medium is provided, which comprises computer program instructions, when the computer program instructions are executed by a computing device, the computing device performs the method in the second aspect or any possible implementation manner of the second aspect.

[0037] As an example, the computer-readable storage includes, but is not limited to, one or more of the following: read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), Flash memory, electrically EPROM (EEPROM), and hard drive.

[0038] Optionally, as an implementation manner, the storage medium can be a non-volatile storage medium.

[0039] In a ninth aspect, a computer-readable storage medium is provided, which comprises computer program instructions, when the computer program instructions are executed by a cluster of computing devices, the cluster of computing devices performs the method in the second aspect or any possible implementation manner of the second aspect.

[0040] By way of example, such computer-readable storage can include one or more of read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), Flash memory, electrically EPROM (EEPROM), and hard drives.

[0041] Optionally, as an implementation manner, the storage medium can be a nonvolatile storage medium. BRIEF DESCRIPTION OF DRAWINGS

[0042] FIG. 1 is a schematic block diagram of a cloud scenario suitable for embodiments of the present application.

[0043] FIG. 2 is a schematic diagram of an architecture of a storage system according to embodiments of the present application.

[0044] FIG. 3 is a schematic diagram of an architecture of another storage system according to embodiments of the present application.

[0045] FIG. 4 is a schematic flowchart of a method of data processing according to embodiments of the present application.

[0046] FIG. 5 is a schematic block diagram of an architecture of a DPU according to embodiments of the present application.

[0047] FIG. 6 is a schematic diagram of a DPU performing a Flush operation according to embodiments of the present application.

[0048] FIG. 7 is a schematic diagram of another DPU performing a Flush operation according to embodiments of the present application.

[0049] FIG. 8 is a schematic flowchart of a flood control strategy for a Flush operation according to embodiments of the present application.

[0050] FIG. 9 is a schematic block diagram of an overall flow of a DPU performing a Flush operation according to embodiments of the present application.

[0051] FIG. 10 is a schematic diagram of an architecture of a computing device 1500 according to embodiments of the present application.

[0052] FIG. 11 is a schematic diagram of an architecture of a computing device cluster according to embodiments of the present application.

[0053] FIG. 12 is a schematic diagram of a connection between computing devices 1500A and 1500B via a network according to embodiments of the present application. DETAILED DESCRIPTION

[0054] The technical solutions in the present application will be described below with reference to the drawings.

[0055] The present application will present various aspects, embodiments or features around systems including a plurality of devices, components, modules, etc. It should be understood and appreciated that each system can include additional devices, components, modules, etc., and / or can not include all of the devices, components, modules, etc. discussed in connection with the figures. Furthermore, a combination of these aspects can also be used.

[0056] In addition, in the embodiments of the present application, the words "example" and "for example" are used to mean serving as an example, instance, or illustration. Any implementation described herein as "example" is not necessarily to be construed as preferred or advantageous over other implementations. Rather, use of the word example is intended to present concepts in a concrete manner.

[0057] In the embodiments of the present application, "corresponding" and "relevant" can be used interchangeably, and it should be pointed out that when the distinction is not emphasized, the meanings expressed are consistent.

[0058] The business scenarios described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, as the network architecture evolves and new business scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0059] In the present specification, the reference to "one embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in other embodiments", "in additional embodiments", etc. appearing in various places in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically stated. The terms "comprise", "include", "have" and their conjugates mean "including but not limited to", unless otherwise specifically stated.

[0060] In this application, "at least one" means one or more, "multiple" means two or more. The "and / or" describes the association relationship of the associated objects, which means that there can be three kinds of relationships, for example, A and / or B can represent: including the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent: a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0061] For ease of description, the related concepts involved in the embodiments of the present application will be explained first.

[0062] 1. Log-structured merge tree (LSM-Tree)

[0063] The log-structured merge tree (LSM-Tree) is also known as the log-structured merge tree, which is a data structure designed to optimize write performance and is commonly used to solve the efficiency problem of random read and write of a disk. Its core feature is to take advantage of the performance of sequential writing of a disk, to realize efficient write operation through delayed writing of a disk and batch merging of data, while trying to maintain the efficiency of read operation. It helps to reduce the user-perceived delay and play the aggregate IO bandwidth of the storage device by batching write operations in memory and then sequentially flushing them to the storage device.

[0064] An LSM-Tree is usually composed of two parts: an in-memory component and a disk component. Write requests are first written to the in-memory component, and when the in-memory component is full, they are flushed to the disk to form a new disk component. Multiple components on the disk form a tree hierarchy through background merging operations. With the popularity of the split architecture in data centers, more and more LSM-trees are deployed on the split structure. Under this structure, the in-memory component is retained on the computing node, while the disk component is stored on the remote storage medium, such as remote file storage, block storage, object storage, or split memory.

[0065] In memory, LSM-Tree generally has MemTable and Immutable MemTable, which provide write, read and delete interfaces for key-value pairs. When the MemTable reaches a certain amount of data, it will be converted into an Immutable MemTable, which will perform operations such as computing compression to form an SSTable and then flush to disk. In disk, the storage unit of LSM-Tree is layered SSTable, divided into L0-LN layers. Starting from L0 layer, whenever the data volume of a layer reaches a certain threshold, the merge operation of LSM-Tree will be triggered to form the SSTable of L1 layer, and so on.

[0066] 2、data processing unit (DPU)

[0067] The data processing unit (DPU) is a new generation of processors for data-centric computing, which plays an increasingly important role in modern computing. It has been widely used in tasks such as offloading storage, network, security, etc.

[0068] 3、Flush operation

[0069] Flush operation generally refers to the operation of flushing buffered data from cache or temporary memory to permanent memory or external device. This operation ensures the persistence of data and updates the data in the storage device.

[0070] In a related technical solution, the central processing unit (CPU) on the server is responsible for performing the above Flush operation. For example, a general write operation usually stores data in the CPU cache first, and the flush operation of the CPU cache is to flush the data in the cache to the main memory or external device by the CPU, to ensure the consistency and reliability of the data.

[0071] The above related technical solution needs to occupy the resources of the server, so that the server cannot free up more resources for user use, reducing the resource utilization rate of the server.

[0072] Therefore, the embodiments of the present application provide a data processing method, which can offload the above flush operation from the server to the data processing unit (DPU), and flush the data in the CPU cache to the main memory or external device by the DPU, so that the server can free up more resources for user use, thereby improving the resource utilization rate of the server.

[0073] In a possible implementation, the method provided by the embodiments of the present application can be applied to a cloud service scenario. For ease of description, the cloud service scenario is described in detail below in combination with FIG. 1.

[0074] FIG. 1 is a schematic block diagram of a cloud scenario applicable to the embodiments of the present application. As shown in FIG. 1, the cloud scenario can include a cloud management platform 110, the Internet 120, and a client 130.

[0075] As shown in FIG. 1, the cloud management platform 110 is configured to manage an infrastructure that provides a plurality of cloud services. The infrastructure includes a plurality of cloud data centers, each of which includes a plurality of servers, and each of the servers includes cloud service resources, which provide corresponding cloud services for tenants.

[0076] The cloud management platform 110 can be located in a cloud data center, which can provide an access interface (such as an interface or an application program interface (API)). A tenant can remotely access the access interface to register a cloud account and a password at the cloud management platform 110 by using the client 130, and log in to the cloud management platform 110. After the cloud management platform 110 authenticates the cloud account and the password successfully, the tenant can further select and purchase a virtual machine of a specific specification (processor, memory, disk) at the cloud management platform 110 by paying a fee. After the payment and purchase are successful, the cloud management platform 110 provides a remote login account and password of the purchased virtual machine, and the client 130 can remotely log in to the virtual machine, and install and run an application of the tenant in the virtual machine. Therefore, the tenant can create, manage, log in to, and operate a virtual machine in the cloud data center through the cloud management platform 110. The virtual machine can also be referred to as an elastic compute service (ECS) or an elastic instance (different cloud service providers have different names).

[0077] It should be understood that the tenant of the cloud service can be an individual, an enterprise, a school, a hospital, an administrative organ, or the like.

[0078] The functions of the cloud management platform 110 include, but are not limited to, a user console, a computing management service, a network management service, a storage management service, an authentication service, and an image management service. The user console provides an interface or an API to interact with the tenant, the computing management service is configured to manage servers running virtual machines and containers and bare metal servers, the network management service is configured to manage network services (such as gateways and firewalls), the storage management service is configured to manage storage services (such as data bucket services), the authentication service is configured to manage the account and password of the tenant, and the image management service is configured to manage virtual machine images. The tenant uses the client 130 to log in to the cloud management platform 110 through the Internet 120, and manage the rented cloud services.

[0079] First, the architecture of the storage system to which the embodiments of the present application are applicable will be described in detail in combination with FIGS. 2-3. It should be understood that the examples of FIGS. 2-3 are only to help those skilled in the art to understand the embodiments of the present application, and are not intended to limit the embodiments of the present application to the specific values or specific scenarios shown in the examples of FIGS. 2-3. Those skilled in the art can obviously make various equivalent modifications or changes according to the examples given below in FIGS. 2-3, and such modifications and changes also fall within the scope of the embodiments of the present application.

[0080] For example, FIG. 2 is a schematic diagram of the architecture of a storage system provided by the embodiments of the present application. As shown in FIG. 2, the storage system includes a computing node, a DPU interposed on the computing node, and a remote storage pool, wherein the storage pool includes storage nodes.

[0081] The above computing node and storage node can be provided in at least one data center providing cloud services, for example, the computing node can be a server of the cloud data center shown in FIG. 1. For another example, the storage node may, for example, be a disk in the server of the cloud data center shown in FIG. 1.

[0082] The computing node (which can also be referred to as a host) runs a virtual instance or an application for remote use by a tenant. The virtual instance can include, but is not limited to, a virtual machine (VM), a container, etc.

[0083] For example, there is a fixed correspondence between the DPU and the computing node, the DPU can be interposed on the computing node, and the DPU realizes connection with the computing node through a connection channel between the DPU and the computing node.

[0084] It should be understood that the connection channel between the DPU and the computing node can be realized by bus technology, or by high-speed interconnection network, or by other communication means, and the embodiments of the present application do not make specific limitations thereto.

[0085] For example, the bus technology can include, but is not limited to, an extended peripheral component interconnect (peripheral component interconnect express, PCIE) bus, a compute express link (compute express link, CXL), a unified bus (Unified BUS, UB), a remote direct memory access (remote direct memory access, RDMA), etc.

[0086] For example, FIG. 3 is a schematic diagram of another architecture of a storage system according to an embodiment of the present application. As shown in FIG. 3, the storage system includes a computing node, a DPU hosting node, and a remote storage node. At least two DPUs are inserted in the DPU hosting node.

[0087] It should be understood that the architecture shown in FIG. 3 is a pooling architecture, in which there is no fixed correspondence between the DPUs and the computing nodes, and multiple computing nodes can share the at least two DPUs included in a resource pool. For example, the at least two DPUs are inserted in one or more physical nodes (which can also be referred to as a DPU hosting node), and the at least two DPUs constitute a resource pool.

[0088] For example, FIG. 3 illustrates two DPUs (a first DPU and a second DPU). As shown in FIG. 3, the first DPU and the second DPU are inserted in the DPU hosting node.

[0089] For example, the DPU hosting node can be provided in at least one data center that provides cloud services, for example, a server of the cloud data center shown in FIG. 1.

[0090] The DPU hosting node and the computing node can be connected through a high-speed interconnection network, or can be connected through other communication means, which are not limited in the embodiments of the present application.

[0091] The first DPU and the second DPU in the DPU hosting node can be connected through a bus technology, or can be connected through a high-speed interconnection network, or can be connected through other communication means, which are not limited in the embodiments of the present application.

[0092] It should be understood that the computing node in FIG. 3 is the same as the computing node in FIG. 2, and the storage node in FIG. 3 is the same as the storage node in FIG. 2. For details, please refer to the description of FIG. 2, which will not be repeated here.

[0093] Next, a data processing method according to an embodiment of the present application will be described in detail with reference to FIG. 4. It should be understood that the example of FIG. 4 is only to help those skilled in the art to understand the embodiments of the present application, and is not intended to limit the embodiments of the present application to the specific values or specific scenarios shown in FIG. 4. Those skilled in the art can obviously make various equivalent modifications or changes based on the examples given in FIG. 4, and such modifications and changes also fall within the scope of the embodiments of the present application.

[0094] For example, FIG. 4 is a schematic flowchart of a data processing method according to an embodiment of the present application. As shown in FIG. 4, the method can include steps 410-430, which will be described in detail below.

[0095] It should be understood that the method of FIG. 4 can be applicable to any of the storage systems described above, and the embodiments of the present application do not make specific limitations thereto.

[0096] Step 410: The DPU obtains an immutable memory table Imm MemTable from the compute node.

[0097] In the embodiments of the present application, the DPU can obtain an immutable memory table (Immutable MemTable) stored in the memory of the compute node, and the Immutable MemTable stores key value (KV) pairs.

[0098] As an example, the above-mentioned Immutable MemTable can also be referred to as Imm MemTable.

[0099] For example, the above-mentioned Imm MemTable corresponds to the first table and the second table mentioned above.

[0100] For example, as shown in FIG. 5, when data is written into the LSM tree, it is first stored in the memory table (MemTable) of the memory of the compute node. The MemTable is a data structure in memory that holds the most recently updated data. When the amount of data in the MemTable reaches a certain threshold, the MemTable is converted into the Imm MemTable.

[0101] In one possible implementation, as shown in FIG. 5, the DPU can include a data path accelerator (DPA). In the embodiments of the present application, the DPA in the DPU can obtain the above-mentioned Imm MemTable from the compute node.

[0102] It should be understood that the DPA is a highly programmable embedded processor that exists in the DPU and is specifically designed for network-intensive, low-computing tasks. It allows users to offload and accelerate network-centric data processing tasks such as device emulation, congestion control, custom protocols, etc. through programming. Through the DPA, the DPU can more effectively process network traffic and reduce dependence on general-purpose multi-core processors, thereby improving overall performance and efficiency.

[0103] For example, in order to offload the data plane operation of Flush to the DPU, it is necessary to enable the DPU to access the memory of the computing node. For example, it is necessary to register the memory index of the computing node to the DPA in the DPU in the Init phase. When a Flush operation is triggered, the hardware thread of the DPA can traverse each KV pair in the Imm MemTable in the memory of the computing node according to the first address of the Imm MemTable in the memory of the computing node, generate corresponding temporary data blocks, and copy these temporary data blocks to the memory of the DPU.

[0104] It should be understood that in computer and software systems, the Init phase generally refers to an initialization phase in the startup process of a system or program. In this phase, the system or program performs a series of necessary settings and preparations to ensure normal operation later. These preparations may include loading necessary modules, configuring environment variables, checking system resources, starting key services, etc.

[0105] Step 420: The DPU constructs a sorted string table SSTable based on the acquired immutable memory table Imm MemTable.

[0106] In the embodiments of the present application, each KV pair in the Imm MemTable in the memory of the computing node is traversed according to the first address of the Imm MemTable in the memory of the computing node, corresponding temporary data blocks are generated, and these temporary data blocks are stored in the Temporary Table (Tmp Table) in the memory of the DPU.

[0107] It should be understood that the Tmp Table is a temporary table commonly used in databases.

[0108] In the embodiments of the present application, the DPU can compress the data in the Tmp Table, perform cyclic redundancy check (CRC) calculation to obtain compressed data blocks, generate index blocks based on the offsets of the compressed data blocks, and finally append the filter blocks to the tail of the compressed data blocks and the index blocks to generate a sorted string table (SSTable).

[0109] It should be understood that the SSTable is an immutable, sorted, and persistent data structure for storing a large number of ordered key value pairs. The SSTable stores the key value pairs in order from small to large according to the Key, so that each Block included in the SSTable file contains a plurality of ordered key value pairs. There can be a long common prefix between adjacent Keys, and in order to save space, the SSTable adopts optimization techniques such as prefix compression. That is, at the end of the SSTable, a Block index is included for quickly locating each Block in the SSTable.

[0110] In a possible implementation, as shown in FIG. 5, an advanced reduced instruction set computing machine core (ARM Core) can be included in the DPU. In the embodiment of the present application, the data in the Tmp Table can be compressed and CRC calculated by the ARM Core in the DPU to obtain a compressed data block, an index block is generated based on the offset of the compressed data block, and finally the filter block is appended to the tail of the compressed data block and the index block to generate the SSTable described above. Since the DPA has a low clock frequency and a small number of computing units, the compression and verification operations and the like are offloaded to the Arm Core in the DPU for execution. Compared with the DPA, the Arm Core has a higher clock frequency, can achieve a decompression and verification throughput comparable to that of the computing node, and can also call the hardware accelerator on the DPU to accelerate these data-intensive computations, thereby achieving better performance.

[0111] For example, the ARM Core in the DPU can compress and verify the temporary data block in the Tmp Table generated by the DPA to generate the SSTable. For example, the ARM Core can compress the temporary data block generated by the DPA to generate a compressed data block, generate an index block based on the offset of the compressed data block, and finally append the filter block to the tail of the compressed data block and the index block to generate the SSTable. That is, the SSTable sequentially includes the compressed data block, the index block, and the filter block.

[0112] Optionally, in some embodiments, the compression, CRC calculation, and the like of the data in the Tmp Table are performed by the ARM Core in the DPU, and additional interaction overhead is introduced between the DPA and the ARM Core. In order to reduce this interaction overhead, a batch mode can be used, and the DPA generates a batch of data blocks and then submits them to the Arm Core for processing to optimize the overall throughput.

[0113] Step 430: The DPU writes the generated SSTable to a storage node in the remote.

[0114] In the embodiments of the present application, the DPU can write the generated SSTable to a storage node in the remote after generating the SSTable. For example, the SSTable is sent to a remote memory or a storage device.

[0115] In a possible implementation, as shown in FIG. 5, the DPU can include a remote direct memory access network interface controller (RDMA NIC). In the embodiments of the present application, the DPU can write the generated SSTable to a storage node in the remote through the RDMA NIC.

[0116] It should be understood that the RDMA NIC is a network interface card that supports the RDMA technology, which allows an application program to bypass the operating system kernel to directly access the memory of a remote computer in a zero-copy manner. This technology can significantly reduce network latency, improve bandwidth utilization, and reduce CPU load. In the architecture of the DPU, the RDMA NIC is used to accelerate data transmission within the data center and improve overall performance and efficiency.

[0117] It should also be understood that the RDMA NIC can also be referred to as an RDMA network adapter or an RDMA network interface card.

[0118] For example, the RDMA NIC in the DPU can directly call a related file system or block storage interface to write data in the SSTable to a storage node in the remote.

[0119] For example, for a remote memory server as a storage node in the remote, the Arm Core can establish a connection with the remote memory server and register a memory region (MR) in the Init phase, the DPU can store the SSTable generated in the flush execution process in the MR area in the memory of the DPU, and then the DPU writes the data blocks in the SSTable to the storage node in the remote through a one-sided RDMA operation.

[0120] In the above technical solution, the DPU performs the Flush operation of the computing node, thereby releasing the CPU resources of the computing node, so that the computing node can free up more CPU resources for user use, thereby improving the utilization rate of the CPU resources of the computing node.

[0121] Optionally, the embodiments of the present application further propose a two-dimensional parallel processing procedure of Flush operation, so as to improve the performance of DPU in performing Flush operation.

[0122] For example, (a) in FIG. 6 is a flush procedure in which the data blocks contained in an Imm MemTable are scheduled in an ordered manner. For a single data block in an Imm MemTable, there is a data dependency between the generation, compression and verification of the single data block and the flush procedure. For example, as shown in (a) in FIG. 6, the data blocks contained in Imm MemTable i in the computing node are first copied to the Tmp Table in the DPU by the DPA in the DPU, then the data in the Tmp Table is compressed and CRC calculated by the ARM Core in the DPU to obtain SSTable, and finally the SSTable is flushed to the remote storage node by the RDMA NIC in the DPU. By analogy, the DPU performs the above processing on the data blocks contained in Imm MemTable i+1 according to the above procedure. Due to the ordered scheduling of the data blocks, only one type of hardware resource in the DPU is used at each stage, which results in low resource utilization.

[0123] For example, as shown in (b) of FIG. 6, the DPA in the DPU copies the data blocks contained in the Imm MemTable i to the Tmp Table i in the DPU, and then the DPA can continue to process the data blocks contained in the Imm MemTable i+1, for example, copy the data blocks contained in the Imm MemTable i+1 to the Tmp Table i+1 in the DPU, and so on. Similarly, the ARM Core in the DPU compresses and calculates the CRC of the data in the Tmp Table i to obtain the SSTable i, and then in the case that the DPA in the DPU copies the data blocks contained in the Imm MemTable i+1 to the Tmp Table i+1, the ARM Core can continue to compress and calculate the CRC of the data in the Tmp Table i to obtain the SSTable i+1, and so on. Similarly, the RDMA NIC in the DPU flushes the SSTable i obtained by the ARM Core to the remote storage node, and then in the case that the ARM Core obtains the SSTable i+1, the RDMA NIC can continue to flush the SSTable i+1 to the remote storage node, and so on.

[0124] In the above pipelined processing, the DPU divides the step of processing the data blocks in an Imm MemTable into three parts, which are respectively executed by three different physical components in the DPU, namely the DPA, the ARM Core and the RDMA NIC. The three physical components in the DPU are independent of each other and do not share resources, and because there is no data dependency between the data blocks of different Imm MemTables, the physical resources in the DPU can be fully utilized through pipelining, thereby improving the throughput of the Flush operation.

[0125] In some embodiments, since the computing capabilities of different heterogeneous units are different, the time for executing the corresponding steps is also not necessarily completely matched. If the execution time of any step in the above pipeline is too long, a large pipeline bubble will be caused, resulting in the idling of certain hardware resources. For example, the data block generation throughput of a single DPA thread in the DPU is difficult to keep up with the compression throughput of a single ARM Core in the DPU. Therefore, embodiments of the present application provide a data parallel processing procedure of the DPA, in which multiple DPA threads in the DPU can be used to generate data blocks in parallel to match the compression speed of the ARM Core, thereby avoiding a large pipeline bubble and avoiding the idling of certain hardware resources in the DPU, so as to further improve the utilization of hardware resources in the DPU.

[0126] For example, as shown in FIG. 7, before starting a flush task, the CPU will first divide the KV pairs contained in the Imm MemTable into finer-grained key-value ranges, and then start multiple DPA threads in the DPU to process different key-value ranges in the Imm MemTable in parallel, so as to achieve line speed matching with the ARM Core. For example, as shown in FIG. 7, 2 DPA threads in the DPU copy data blocks contained in the Imm MemTable i to the Tmp Table i of the DPU in parallel, and then the 2 DPA threads in the DPU can continue to copy data blocks contained in the Imm MemTable i+1 to the Tmp Table i+1 of the DPU in parallel, and so on. Similarly, the ARM Core in the DPU compresses and calculates the CRC of the data in the Tmp Table i to obtain the SSTable i, and then, in the case that the 2 DPA threads in the DPU copy data blocks contained in the Imm MemTable i+1 to the Tmp Table i+1 of the DPU in parallel, the ARM Core can continue to compress and calculate the CRC of the data in the Tmp Table i to obtain the SSTable i+1, and so on. Similarly, the RDMA NIC in the DPU flushes the SSTable i obtained by the ARM Core to the remote storage node, and then the RDMA NIC can continue to flush the SSTable i+1 obtained by the ARM Core to the remote storage node, and so on.

[0127] It should be understood that, since the throughput of the RDMA NIC is higher than the maximum compression throughput of 16 ARM Cores and the memory access throughput of the DPA, the RDMA NIC will not become a bottleneck of the flush operation.

[0128] In some embodiments, normally, after the DPU generates an SSTable, the SSTable is directly flushed to the remote storage node. If the speed of the Flush operation is too fast, the SSTable of the L0 layer of the remote storage node will be excessive, which is referred to as L0 Stall. When the computing node receives information from the remote storage node indicating that L0 Stall is detected, the write traffic of the front end is throttled, thereby causing the write of the front end to stop.

[0129] Therefore, the embodiments of the present application provide a flood control strategy for the Flush operation. When L0 Stall of the remote storage node is detected, the flood control strategy can reduce the merging overhead of the remote storage node and avoid the write stop of the front end, thereby improving the write performance for periodic high write load.

[0130] For example, as shown in FIG. 8, if the computing node does not detect L0 Stall of the remote storage node, the normal Flush operation process can be performed. After the DPU generates an SSTable, the SSTable is directly flushed to the remote storage node. If the computing node detects L0 Stall of the remote storage node, the Flush message with additional information is sent to the DPU. After the DPA thread and the ARM Core execution thread in the DPU perform the normal Flush process (for example, processing the Imm MemTable and generating the SSTable), the generated SSTable is not transmitted or flushed to the remote storage node, but is cached in the memory of the DPU until the L0 Stall of the remote storage node is alleviated, and then the SSTable stored in the memory of the DPU is continued to be flushed to the remote storage node.

[0131] Optionally, as shown in FIG. 8, during the process of caching the generated SSTable in the memory of the DPU, the L0 to L1 layer compression can also be performed by using the computing power of the DPU. When the L0 Stall of the remote storage node is alleviated, the normal Flush process is restored. This can effectively reduce the damage of L0 Stall and more fully utilize the computing power of the DPU.

[0132] Optionally, in some embodiments, for the read operation, since the SSTable is partially cached in the memory of the DPU, the read performance can be improved to a certain extent. However, since the number of unordered SSTables in the memory of the DPU is increased, a global index is constructed to accelerate the search performance.

[0133] It should be understood that the above process is to use the memory of the DPU as the read-write cache of the SSTable, and the metadata of the SSTable is still stored on the computing node (which can also be referred to as a host). For a read request, the memory index on the computing node is first found, and then the metadata of the SSTable is accessed to determine whether the to-be-read data is in the DPU. If the to-be-read data is not in the DPU, the remote storage node can be directly accessed to obtain the to-be-read data. If the to-be-read data is in the DPU, the DPU can be directly accessed to obtain the to-be-read data.

[0134] The overall flow of the DPU performing the Flush operation in the embodiment of the present application will be described in detail below in conjunction with FIG. 9. It should be understood that the example of FIG. 9 is only to help those skilled in the art to understand the embodiments of the present application, and is not intended to limit the embodiments of the present application to the specific values or specific scenarios shown in the example of FIG. 9. Those skilled in the art can obviously make various equivalent modifications or changes according to the examples given below in FIG. 9, and such modifications and changes also fall within the scope of the embodiments of the present application.

[0135] For example, FIG. 9 is a schematic block diagram of an overall flow of the DPU performing the Flush operation according to an embodiment of the present application. As shown in FIG. 9, the Flush Worker in the computing node starts by connecting the ARM Core main thread at the DPU end, the computing node transmits the task information of the Flush task to the ARM Core main thread when receiving the Flush task, and the Flush Worker in the computing node ends after the task at the DPU end is completed. For the ARM Core main thread at the DPU end, it waits for the task information of the Flush task sent by the Flush Worker at the computing node end after connecting the computing node, and starts the ARM Core processing thread after receiving the task information of the Flush task. For the ARM Core processing thread at the DPU end, it starts the DPA processing thread in the DPU to copy the data blocks contained in the Imm MemTable in the computing node to the Tmp Table in the DPU. The ARM Core processing thread can compress, CRC calculate, etc. the data blocks in the Tmp Table to generate the SSTable, and the ARM Core processing thread can also transmit the generated SSTable to the RDMA NIC in the DPU and transmit the SSTable to the remote storage node through the RDMA NIC.

[0136] The method provided by the embodiments of the present application can be executed by a computing device, which can also be referred to as a computer system. The computer system includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a processing unit, a memory, and a memory control unit, and the functions and structures of the hardware are described in detail later. The operating system is any one or more computer operating systems that implement business processing through processes, such as a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system, or a windows operating system. The application layer includes application programs such as a browser, an address book, word processing software, and instant messaging software. Optionally, the computer system is a handheld device such as a smartphone or a terminal device such as a personal computer, and the present application is not particularly limited as long as the method provided by the embodiments of the present application can be executed. The execution subject of the method provided by the embodiments of the present application can be a computing device, or a functional module in the computing device that can call and execute a program.

[0137] The computing device provided by the embodiments of the present application is described in detail below in combination with FIG. 10.

[0138] FIG. 10 is an architecture schematic diagram of a computing device 1500 provided by the embodiments of the present application. The computing device 1500 can be a server or a computer or other device with computing capability. The computing device 1500 shown in FIG. 10 includes at least one processor 1510 and a memory 1520.

[0139] It should be understood that the number of processors and memories in the computing device 1500 is not limited by the present application.

[0140] The processor 1510 executes instructions in the memory 1520, so that the computing device 1500 implements the method provided by the present application. Alternatively, the processor 1510 executes instructions in the memory 1520, so that the computing device 1500 implements the functional modules provided by the present application, thereby implementing the method provided by the present application.

[0141] Optionally, the computing device 1500 further includes a communication interface 1530. The communication interface 1530 uses a transceiving module such as but not limited to a network interface card and a transceiver to implement communication between the computing device 1500 and other devices or communication networks.

[0142] Optionally, the computing device 1500 also includes a system bus 1540, wherein the processor 1510, the memory 1520 and the communication interface 1530 are connected with the system bus 1540 respectively. The processor 1510 can access the memory 1520 through the system bus 1540, for example, the processor 1510 can read and write data in the memory 1520 or execute code in the memory 1520 through the system bus 1540. The system bus 1540 is a peripheral component interconnect express (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus 1540 is divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is shown in FIG. 10, but it does not mean that there is only one bus or only one type of bus.

[0143] In one possible implementation, the function of the processor 1510 is mainly to interpret the instructions (or code) of the computer program and process the data in the computer software. Among them, the instructions of the computer program and the data in the computer software can be saved in the memory 1520 or the cache 1516.

[0144] Optionally, the processor 1510 can be an integrated circuit chip with a processing capability of signals. As an example but not limitation, the processor 1510 is a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. Among them, the general purpose processor is a microprocessor, etc. For example, the processor 1510 is a central processing unit (CPU).

[0145] Optionally, each processor 1510 includes at least one processing unit 1512 and a memory control unit 1514.

[0146] Optionally, processing units 1512, also referred to as cores or core complexes, are the most important components of a processor. Processing units 1512 are manufactured from single-crystal silicon using certain production processes. All calculations, command receiving, command storage, and data processing of a processor are performed by cores. Processing units independently execute program instructions and use parallel computing to speed up program execution. Various processing units have fixed logical structures, such as a first-level cache, a second-level cache, an execution unit, an instruction-level unit, a bus interface, and the like.

[0147] In one implementation, memory control units 1514 are configured to control data interaction between memory 1520 and processing units 1512. Specifically, memory control units 1514 receive memory access requests from processing units 1512 and control access to memory based on the memory access requests. By way of example and not limitation, a memory control unit is a memory management unit (MMU) or the like.

[0148] In one implementation, each memory control unit 1514 addresses memory 1520 through a system bus. An arbiter (not shown in FIG. 10) is configured in the system bus and is responsible for handling and coordinating competing access of multiple processing units 1512.

[0149] In one implementation, processing units 1512 and memory control units 1514 are communicatively connected through internal connections, such as address lines, of a chip, so as to realize communication between processing units 1512 and memory control units 1514.

[0150] Optionally, each processor 1510 further includes a cache 1516, which is a buffer for data exchange (referred to as a cache). When processing units 1512 want to read data, the required data is first searched in the cache. If the data is found, the data is directly executed. If the data is not found, the data is searched in the memory. Since the cache runs much faster than the memory, the cache helps processing units 1512 to run faster.

[0151] Memory 1520 can provide a running space for a process in computing device 1500. For example, a computer program (specifically, a code of the program) for generating a process is stored in memory 1520. After the computer program is executed by a processor to generate a process, the processor allocates a corresponding storage space for the process in memory 1520. Further, the storage space further includes a text segment, an initialized data segment, a bit initialized data segment, a stack segment, a heap segment, and the like. Memory 1520 stores data generated during running of the process, such as intermediate data, process data, and the like, in the storage space corresponding to the process.

[0152] Optionally, memory, also called RAM, is used to temporarily store data for processing by processor 1510. Processor 1510 moves data to and from memory for processing. When the computer is turned off, processor 1510 can lose its contents, unless a type of memory, called non-volatile memory, is used.

[0153] By way of example, and not limitation, memory 1520 is volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. By way of example, and not limitation, nonvolatile memory can be read only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically EPROM (EEPROM), or flash memory. Volatile memory is random access memory (RAM), which acts as external cache. By way of example and not limitation, many forms of RAM exist, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM). The system and method described herein can be used in connection with one or more types of memory or other disk based system for higher data storage

[0154] The structure of the computing device 1500 listed above is only an example, and the application is not limited thereto. The computing device 1500 of the embodiments of the application includes various hardware in the prior art computer system, for example, the computing device 1500 also includes other memories in addition to the memory 1520, such as disk memories and the like. Those skilled in the art should understand that the computing device 1500 can also include other devices necessary for normal operation. Meanwhile, according to specific needs, those skilled in the art should understand that the above computing device 1500 can also include hardware devices for implementing other additional functions. In addition, those skilled in the art should understand that the above computing device 1500 can also only include devices necessary for the embodiments of the application, and does not have to include all the devices shown in FIG. 10.

[0155] The embodiments of the application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server. In some embodiments, the computing device can also be a desktop computer, a notebook computer, or a terminal device such as a smart phone.

[0156] As shown in FIG. 11, the computing device cluster includes at least one computing device 1500. The memory 1520 in one or more computing devices 1500 in the computing device cluster can have the same instructions for executing the above method.

[0157] In some possible implementations, the memory 1520 in one or more computing devices 1500 in the computing device cluster can also respectively have partial instructions for executing the above method. In other words, the combination of one or more computing devices 1500 can collectively execute the instructions of the above method.

[0158] It should be noted that the memories 1520 in different computing devices 1500 in the computing device cluster can store different instructions, respectively used for executing partial functions of the above apparatus. That is, the instructions stored in the memories 1520 in different computing devices 1500 can implement the functions of one or more modules in the above apparatus.

[0159] In some possible implementations, one or more computing devices in the computing device cluster can be connected through a network. The network can be a wide area network or a local area network, etc. FIG. 12 shows a possible implementation. As shown in FIG. 12, two computing devices 1500A and 1500B are connected through a network. Specifically, the communication interface in each computing device is connected to the network.

[0160] It should be appreciated that the functionality of the computing device 1500A shown in FIG. 12 can also be accomplished by a plurality of computing devices 1500. Likewise, the functionality of the computing device 1500B can also be accomplished by a plurality of computing devices 1500.

[0161] In this embodiment, a computer program product containing instructions is also provided. The computer program product can be a software or program product containing instructions, which can be run on a computing device or stored in any available medium. When it is run on a computing device, it causes the computing device to perform the method provided above, or causes the computing device to realize the functions of the apparatus provided above.

[0162] In this embodiment, a computer readable storage medium is also provided. The computer readable storage medium can be any available medium that a computing device can store or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk), etc. The computer readable storage medium contains instructions, which, when executed on a computing device, cause the computing device to perform the method provided above.

[0163] It should be understood that the size of the sequence number of each process described above in various embodiments of the present application does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0164] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0165] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, apparatus and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0166] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely schematic. The division of the units is merely logical function division. There can be other division manners in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0167] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0168] In addition, each functional unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can be a physically independent unit, or two or more units can be integrated into one unit.

[0169] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0170] The above description is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of data processing, characterized by, The method is applied to a storage system comprising a data processing unit (DPU), a computing node and a storage node, wherein the storage node and the computing node are arranged in at least one data center providing cloud services, and the method comprises: The DPU obtains a data block to be written from the computing node and stores the data block to be written in a memory of the DPU; The DPU processes the data block to be written to generate a target sorted string table (SSTable), wherein the target SSTable comprises data blocks obtained by the DPU after compression and verification processing on the data block to be written; The DPU sends the target SSTable to the storage node.

2. The method of claim 1, wherein, The DPU comprises at least one data path accelerator (DPA), and the computing node stores a first table comprising the data block to be written, The DPU obtains the data block to be written from the computing node and stores the data block to be written in a memory of the DPU, comprising: The at least one DPA in the DPU obtains the data block to be written from the first table and stores the data block to be written in the memory of the DPU.

3. The method of claim 2, wherein, The computing node further stores a second table comprising the data block to be written, The at least one DPA in the DPU obtains the data block to be written from the first table and stores the data block to be written in the memory of the DPU, comprising: The at least one DPA stores the data block to be written obtained from the first table in the memory of the DPU and continues to store the data block to be written obtained from the second table in the memory of the DPU.

4. The method according to any one of claims 1 to 3, characterized in that, The DPU further comprises an advanced reduced instruction set machine core (ARM Core), The DPU processes the data block to be written to generate a target sorted string table (SSTable), comprising: The ARM Core in the DPU processes the data block to be written to generate the target SSTable.

5. The method according to any one of claims 1 to 4, characterized in that, The DPU further comprises a network card, The DPU sends the target SSTable to the storage node, comprising: The network card in the DPU sends the target SSTable to the storage node.

6. The method of claim 5, wherein, The network card in the DPU sends the target SSTable to the storage node, comprising: The network card stores the target SSTable in the memory of the DPU; The network card sends the target SSTable to the storage node when a number of SSTables currently processed by the storage node is less than a preset threshold.

7. The method according to any one of claims 1 to 6, characterized in that, The DPU is interposed on the computing node.

8. The method according to any one of claims 1 to 6, characterized in that, The DPU is interposed on a DPU carrier node connected with the computing node.

9. A storage system, characterized by The storage system comprises a data processing unit (DPU), a computing node and a storage node, wherein the storage node and the computing node are arranged in at least one data center providing cloud services, and wherein, The DPU is configured to obtain the data block to be written from the computing node and store the data block to be written in the memory of the DPU. The DPU is further configured to process the data block to be written to generate a target Sorted String Table (SSTable), wherein the target SSTable includes the data block obtained after the DPU performs compression and verification processing on the data block to be written. The DPU is further configured to send the target SSTable to the storage node.

10. The storage system of claim 9, wherein, The DPU includes at least one Data Path Accelerator (DPA), and the computing node stores a first table including the data block to be written. The at least one DPA is configured to obtain the data block to be written from the first table and store the data block to be written in the memory of the DPU.

11. The storage system of claim 10, wherein, The computing node further stores a second table including the data block to be written. The at least one DPA is specifically configured to: store the data block to be written obtained from the first table in the memory of the DPU, and continue to store the data block to be written obtained from the second table in the memory of the DPU.

12. The storage system of any one of claims 9 to 11, wherein, The DPU further includes an Advanced RISC Machines (ARM) Core. The ARM Core is configured to process the data block to be written to generate the target SSTable.

13. The storage system of any one of claims 9 to 12, wherein, The DPU further includes a network card. The network card is configured to send the target SSTable to the storage node.

14. The storage system of claim 13, wherein, The network card is specifically configured to: store the target SSTable in the memory of the DPU; and send the target SSTable to the storage node when the number of SSTables currently processed by the storage node is less than a preset threshold.

15. The storage system of any one of claims 9 to 14, wherein, The DPU is inserted on the computing node.

16. The storage system of any one of claims 9 to 14, wherein, The DPU is inserted on a DPU hosting node connected to the computing node.

17. A cluster of computing devices, characterized in that, The at least one computing device includes a processor and a memory. The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the computing device cluster to perform the method of any one of claims 1 to 8.

18. A computer program product comprising instructions, characterized in that, The instructions, when executed by the computing device cluster, cause the computing device cluster to perform the method of any one of claims 1 to 8.

19. A computer-readable storage medium, characterized in that, The computer program instructions, when executed by the computing device cluster, cause the computing device cluster to perform the method of any one of claims 1 to 8.

Images