Data storage method, device, and storage medium

Abstract

Embodiments of the present disclosure provide a data storage method, a device, and a storage medium. An innovative network interface card device is installed on each storage server in a storage cluster, and a non-volatile cache is arranged in the network interface card device. Upon receiving data to be stored, the network interface card device can write the data to be stored into the non-volatile cache on the network interface card device; then, the network interface card device directly writes one of a plurality of pieces of data to be persisted constructed for the data to be stored and used for persistent storage into a storage disk on the storage server where the network interface card device is located, and distributes other pieces to network interface card devices installed on other storage servers, so as to complete persistent storage. Thus, in the persistent storage process, the dependence on a CPU in a storage server is decoupled, multiple memory copies do not need to be performed in the storage server any more, data transfer is directly performed by a network interface card device, and construction processing of data to be persisted is completed nearby, thereby effectively improving the persistent storage efficiency of a storage cluster.

Description

Data storage method, device and storage medium Technical Field

[0001] The present disclosure relates to the field of storage technology, and in particular to a data storage method, device, and storage medium. Background Art

[0002] Compared to local storage, distributed storage clusters support dynamic expansion or contraction on demand, and offer high availability and strong consistency for persistent data storage. In recent years, high-bandwidth RDMA networks have reduced network transmission latency, while the design of high-performance storage servers significantly reduces the end-to-end latency of distributed storage while adapting to high throughput. This brings the performance of distributed storage clusters close to that of local storage disks, narrowing the gap, improving the user experience, and meeting the access requirements of application software.

[0003] Currently, after the computing node sends the data to be stored to the storage server, the storage server's network card will first store the data in the storage server's system memory DRAM, and then copy the data in the system memory to the persistent memory through memory copying, and return a notification of successful write to the computing node. After that, the storage server needs to copy this data from the persistent memory PMEM back to the system memory, and the storage server's central processing unit (CPU) needs to read this data from the system memory and build multiple copies of the data to be persisted, which will then be distributed to multiple storage servers and finally stored in the solid-state hard drives and other storage disks in each storage server. Only after the above complex process can the storage cluster complete the persistent storage of data.

[0004] It can be seen that although the existing persistent storage solution shortens network transmission delay, multiple memory copies are required within the storage server and it also relies on the participation of the CPU. Therefore, the storage delay within the storage server remains high, which in turn leads to the overall delay of data persistent storage based on the storage cluster still being higher than the delay of local storage.

[0005] Summary of the Invention

[0006] Various aspects of the present disclosure provide a data storage method, device, and storage medium to reduce latency of persistent storage in a storage cluster.

[0007] An embodiment of the present disclosure provides a data storage method applicable to a network card device mounted on a storage server in a storage cluster, wherein the network card device is provided with a non-volatile cache. The method includes:

[0008] Writing the received data to be stored into the non-volatile cache;

[0009] Constructing multiple copies of the data to be persisted for persistent storage for the data to be stored;

[0010] Writing one copy of the data to be persisted from the multiple copies of data to be persisted into a storage disk on the storage server;

[0011] The other copies of the data to be persisted are distributed to the network card devices installed on the other storage servers, so that the network card devices installed on the other storage servers respectively write the received data to be persisted into the storage disk of the storage server where they are located.

[0012] The embodiment of the present disclosure further provides a network card device, comprising a memory, a processor, a communication component, and a non-volatile cache;

[0013] The memory is used to store one or more computer instructions;

[0014] The processor is coupled to the memory, the communication component and the non-volatile cache, and is configured to execute the one or more computer instructions to perform the aforementioned data storage method.

[0015] An embodiment of the present disclosure further provides a storage cluster, comprising a plurality of storage servers, each of which is equipped with a network card device, and the network card device is used to execute the aforementioned data storage method to persistently store the data to be stored in the storage cluster.

[0016] An embodiment of the present disclosure further provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the aforementioned data storage method.

[0017] In the embodiment of the present disclosure, an innovative network card device is installed on each storage server in the storage cluster. The network card device is provided with a non-volatile cache. On this basis, after receiving the data to be stored, the network card device can write the data to be stored into the non-volatile cache on it, which can effectively shorten the response delay to the corresponding storage request; thereafter, the network card device can construct multiple copies of the data to be stored for persistent storage, and directly write one of the copies of the data to be persisted to the storage disk on the storage server where it is located, and distribute the other copies of the data to be persisted to the network card devices installed on other storage servers in the storage cluster, so that these network card devices respectively write the received data to be persisted to the storage disk on the storage server where they are located, thereby completing persistent storage. It can be seen that in the persistent storage process, the dependence on the CPU in the storage server is decoupled, and multiple memory copies are no longer required in the storage server. Instead, the network card device directly transfers the data and completes the construction and processing of the data to be persisted nearby, which can effectively reduce the storage delay on the storage server side, thereby effectively improving the persistent storage efficiency of the storage cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings:

[0019] Figure 1 is a logical diagram of persistent storage based on a storage cluster;

[0020] FIG2 is a logical diagram of a traditional persistent storage solution within a storage server;

[0021] FIG3 is a schematic diagram of the structure of a network card device and its assembly on a storage server provided by an exemplary embodiment of the present disclosure;

[0022] FIG4 is a schematic diagram of an exemplary structure of a network card device provided by an exemplary embodiment of the present disclosure;

[0023] FIG5 is a logical diagram of an optional implementation scheme for constructing data to be persisted in a network card device provided by an exemplary embodiment of the present disclosure;

[0024] FIG6 is a flow chart of a data storage method provided by another exemplary embodiment of the present disclosure;

[0025] FIG7 a provides a schematic diagram of an exemplary process of writing data to be stored into a storage cluster;

[0026] FIG7 b provides a schematic diagram of an exemplary process for completing persistence of data to be stored in a storage cluster. DETAILED DESCRIPTION

[0027] To make the objectives, technical solutions, and advantages of the present disclosure more clear, the technical solutions of the present disclosure will be clearly and completely described below in conjunction with the specific embodiments of the present disclosure and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present disclosure.

[0028] Figure 1 is a logical diagram of persistent storage based on a storage cluster. Referring to Figure 1, the computing node can send the data to be stored to the storage cluster through a network switch. In actual applications, the computing node usually adopts the method of submitting multiple backup data in parallel to store data based on the storage cluster. Taking three backups as an example, the computing node can initiate storage requests to the three storage servers in the storage cluster in parallel. The storage request carries the backup data, and the backup data can be used by the storage server as the data to be stored. After all three backups are successfully written, the storage cluster can select one of the three storage servers as the primary server. The primary server will then persistently store the backup data stored on it to multiple storage servers in the storage cluster to complete the persistent storage of data for the computing node.

[0029] Figure 2 is a logical diagram of a traditional persistent storage solution within a storage server. Referring to Figure 2, in traditional persistent storage solutions, to shorten the write path, storage servers typically use persistent memory (PMEM) rather than PCIe storage disks with higher latency. Based on this, after the backup data submitted by the compute node enters the storage server's network interface card (NIC), it is transmitted via the PCIe bus link to the storage server's system memory (DRAM). The data is then copied to the persistent memory (PMEM) via the system memory (DRAM). At this point, feedback is sent to the compute node indicating that the backup has been successfully written, thus completing the backup data write path. The compute node can declare the write successful after all of its submitted backups have been successfully written. Continuing with Figure 2, the storage server must read the backup data from the persistent memory (PMEM) into the system memory (DRAM). The storage server's CPU then constructs persistent data from the backup data read from the system memory (DRAM) and sends it to the NIC. The NIC then distributes this persistent data to multiple storage servers in the storage cluster via a network switch. Each storage server then stores the received data on its own solid-state drive (SSD) or other storage disk to complete the data's persistent storage. When persistent storage is successful, the corresponding data can be deleted from the persistent memory PMEM of the storage server to free up space for subsequent data.

[0030] During the research process, the inventors found that traditional persistent storage solutions have at least the following limitations:

[0031] 1. Persistent memory (PMEM) requires CPU support and stable hardware and software compatibility with the CPU, including but not limited to BIOS, firmware, transmission protocol, SI, PI, and other aspects.

[0032] 2. Memory replication is required between the network card and the system memory DRAM, and between the system memory DRAM and the persistent memory PMEM, which increases storage latency.

[0033] 3. The persistent memory PMEM supply is single and has been announced to be discontinued in the future, which cannot meet the supply.

[0034] 4. Persistent memory PMEM can only be used for CPUs from specified manufacturers and requires the CPU to comply with certain requirements. It cannot be universally adapted to CPUs from all manufacturers.

[0035] To this end, this embodiment provides an innovative network card device to shorten the persistent storage delay on the storage server side. The network card device provided by this embodiment can be installed on each storage server in the storage cluster. Figure 3 is a structural schematic diagram of a network card device provided by an exemplary embodiment of the present disclosure and a schematic diagram of its assembly on a storage server. Referring to Figure 3, the network card device provided by this embodiment can be installed on the storage server in the form of bus PCIe interconnection. The network card device provided by this embodiment can use the standard Ethernet protocol based on its communication component to connect to the network switch in Figure 1 to receive / send data through the network.

[0036] Referring to Figure 3, the network card device provided in this embodiment may include a processor, a communication component and a non-volatile cache. In this embodiment, the communication component in the network card device may retain the original network card communication function, including but not limited to the network interface NIC and the media access control MAC, etc., which will not be explained in detail here. The non-volatile cache in Figure 3 is a component added to the network card device in this embodiment, and the non-volatile cache can be coupled to the processor in the network card device through the bus PCIe. In actual applications, the assembly of the non-volatile cache can be achieved by patching on the circuit board of the network card device or by plugging in an expansion card. In addition, the network card device provided in this embodiment can also provide enhanced functions such as multi-queue reading and writing and direct memory access DMA in addition to the basic functions such as the aforementioned NIC+MAC+PCIe. Moreover, based on the independent processor in the network card device, in this embodiment, one or more computer instructions can be pre-set in the memory of the network card device (such as system memory) to enable its processor to execute the data storage method in this embodiment.

[0037] Based on this, referring to Figure 3, the network card device can use its communication component to receive data to be stored. Continuing with the multi-backup storage approach for compute nodes mentioned in Figure 1, the device to be stored here can be backup data sent by the compute node. Of course, compute nodes can also use other storage methods, which are not limited here. Here, the data to be stored received by the network card device can be understood as data that requires persistent storage within a storage cluster.

[0038] Referring to Figure 3, after receiving the data to be stored, the network card device no longer writes it into the system memory of the storage server as in the traditional solution, but writes it into the non-volatile cache in the network device. Compared with the traditional solution, this not only does not require the participation of the system memory in the storage server, but also effectively shortens the data writing path, so that the data writing can be completed with low latency. Preferably, after the data to be stored is written into the non-volatile cache, the network card device can return a write success notification to the storage requester of the data to be stored (such as the aforementioned computing node). In actual application, from the perspective of the storage requester, after receiving the write success notification, it will think that the data to be stored has been successfully written into the storage cluster, and the storage requester defaults that the storage cluster will autonomously perform the subsequent persistent storage process. Therefore, the shorter write path provided in this embodiment can feedback the write success notification to the storage requester with lower latency, thereby effectively improving the storage efficiency perceived by the storage requester.

[0039] FIG4 is a schematic diagram of an exemplary structure of a network card device provided by an exemplary embodiment of the present disclosure. Referring to FIG4 , in addition to the relevant functional components of the communication components mentioned above (e.g., the NIC, MAC, etc. in FIG4 ), the network card device is also provided with a non-volatile storage controller. As shown in FIG4 , the non-volatile storage controller can be connected to the network transmission path in a bypass manner. In this way, the non-volatile storage controller can intercept the data to be stored from the network transmission path and write the data to the non-volatile cache by performing a write operation on the non-volatile cache.

[0040] Continuing to refer to FIG3 , in order to realize the persistent storage of the data to be stored, the network card device in this embodiment can also construct the data to be persisted for persistent storage for the data to be stored. The construction logic can be preset in the processor of the network card device. Compared with the traditional solution in FIG2 , in this embodiment, when constructing the data to be persisted, the CPU in the storage server is no longer required to participate, nor is it necessary to rely on the aforementioned persistent storage PMEM. Instead, the network card device constructs the data to be persisted for the data to be stored, which can effectively avoid dependence on the CPU type. In this way, the persistent storage process on the storage server side can be completely decoupled from the CPU in the storage server. Therefore, the data storage solution implemented by the network card device in this embodiment is compatible with various CPU types in the storage server.

[0041] In addition, since the data to be stored is stored in the non-volatile cache of the network card device, the processor of the network card device is closer to the data to be stored than the PCU in the storage server, so higher construction efficiency can be achieved.

[0042] In this embodiment, the construction mechanism of the data to be persisted is not limited. The erasure coding mechanism or the multiple backup mechanism can be used as needed to construct the data to be persisted. The detailed implementation method will be described in detail later.

[0043] After completing the construction of the data to be persisted, referring to Figure 3 , the network card device can write one of the multiple copies of the data to be persisted to the storage disk on the storage server; and distribute the other copies of the data to be persisted to the network card devices installed on other storage servers. The network card devices installed on the other storage servers can then write the received data to the storage disk of the storage server on which they are located. In this embodiment, the storage disk in the storage server can be an SSD, etc., and this embodiment does not limit the type of storage disk.

[0044] It should be understood that in actual applications, it is usually necessary to build multiple copies of the data to be persisted for the data to be stored, and store the multiple copies of the data to be persisted on different storage servers in the storage cluster to ensure that the data to be stored can still be completely restored in the event of failures such as partial server downtime or data loss.

[0045] Preferably, in this embodiment, direct memory access can be used to write one copy of the data to be persisted from the multiple copies to the memory of the storage disk on the storage server where the network card device is located. Direct memory access means that the network card device directly writes the data to be persisted to the memory of the storage disk without the involvement of the CPU of the storage server where the network card device is located.

[0046] In this preferred solution, the network card device and the storage disk both serve as bus PCIe devices of the storage server. Based on this, a direct connection relationship between the two PCIe devices can be established to support direct memory access of the network card device to the storage disk. In an exemplary solution, the memory address space of the storage disk on the storage server can be mapped to the logical address space of the network card device. The memory address space of the storage disk is the BAR (Base Address Register) address space. In this way, through the BAR address space mapping technology, the memory address space of the storage disk can be exposed to the network card device, and the network card device can initiate read and write operations on the memory address space of the storage disk as if accessing local storage space. Based on this, the network card device can perform a write operation on the memory address space of the storage disk based on the mapping relationship between the memory address space of the storage disk and the logical address space of the network card device, so as to write one of the multiple copies of data to be persisted to the memory of the storage disk on the storage server. Of course, other solutions can also be used in this embodiment to achieve a direct connection between the two PCIe devices, the network card device and the storage disk, and further examples are not provided here.

[0047] The storage disk in the storage server usually uses flash memory as a persistent storage medium. Based on this, the storage disk can also continue to write the data to be persisted written in its memory into its flash memory. Among them, flash memory can also be called flash memory, which is a non-volatile storage device, usually composed of flash memory chips. It uses electronic storage technology to store data in internal flash memory units; unlike memory, the data in flash memory remains unchanged after power failure. In actual applications, the access unit of flash memory is usually a data group batch (batch), that is, the amount of data processed at one time. For this reason, after the data to be persisted in the memory of the storage disk reaches the data group specification required by its flash memory, the persistent data in the memory of the storage disk can be spliced ​​into data groups and written to the flash memory.

[0048] After other copies of data to be persisted are distributed to the network card devices installed on other storage servers, these network card devices can also write the received data to be persisted directly to the storage disk on the storage server according to the above-mentioned direct memory access method.

[0049] In the above, the network card device writes the data to be persisted directly to the storage disk of the storage server where it is located through direct memory access. There is no need for memory copying and no need for the CPU of the storage server to participate, which can effectively shorten the write delay and reduce the overhead such as memory copying.

[0050] In summary, in this embodiment, an innovative network card device can be installed on each storage server in the storage cluster, and a non-volatile cache is provided in the network card device. On this basis, after receiving the data to be stored, the network card device can write the data to be stored into the non-volatile cache thereon, which can effectively shorten the response delay to the corresponding storage request; thereafter, the network card device can construct multiple copies of the data to be persisted for persistent storage for the data to be stored, and directly write one of the copies of the data to be persisted to the storage disk on the storage server where it is located by the network card device, and distribute the other copies of the data to be persisted to the network card devices installed on other storage servers in the storage cluster, so that these network card devices can respectively write the received data to be persisted into the storage disk on the storage server where they are located, thereby completing persistent storage. It can be seen that in the persistent storage process, the dependence on the CPU in the storage server is decoupled, and multiple memory copies are no longer required in the storage server. Instead, the network card device directly transfers data and completes the construction and processing of the data to be persisted nearby. This can effectively reduce the storage latency on the storage server side, thereby effectively improving the persistent storage efficiency of the storage cluster.

[0051] In the above or following embodiments, the network card device can use various implementations to construct data to be persisted for data to be stored. Figure 5 is a logical diagram of an optional implementation scheme for constructing data to be persisted in a network card device provided by an exemplary embodiment of the present disclosure.

[0052] Referring to Figure 5, this implementation uses an erasure coding mechanism to construct the data to be persisted: the data to be stored can be read from a non-volatile cache; the data to be stored is then erasure-coded to obtain multiple erasure-coded slices corresponding to the data to be stored, each serving as the data to be persisted. The erasure coding mechanism can involve slicing the data to be stored into k original data shares, adding m redundant data shares to each of the k original shares, and then restoring the k original data shares using any k shares of the k+m shares.

[0053] 4 , a computation offloading unit may be pre-installed in a processor of a network card device, and an erasure code encoder may be deployed in the computation offloading unit to perform erasure code encoding on data to be stored.

[0054] In this implementation, the non-volatile cache in the network card device can adopt a sequential read and write mode, which can also be called a log read and write mode. In this mode, the non-volatile cache can be regarded as a continuous address space for sequential reading and writing, and the write pointer points to the current write address, storing the latest data being written. The read pointer points to the current read address, and the read data will be used for erasure coding and subsequent processing. The data that has been read will no longer be retained in the non-volatile cache. Therefore, referring to Figure 5, the data between the read pointer and the write pointer is the current valid data and needs to be erasure coded. This sequential read and write mode can ensure the accuracy of data reading and writing in the non-volatile cache.

[0055] Considering that the read and write units of the non-volatile cache may not match the amount of data to be stored, that is, it may not be possible to read all the data to be stored from the non-volatile cache at once, the network card device in this embodiment may also include a temporary data storage area. In actual applications, the temporary data storage area can be implemented using dynamic random access memory, etc. This embodiment does not limit the hardware type used for the temporary data storage area.

[0056] Based on this, in this optional implementation, on the one hand, the network card device can write the data to be stored into the non-volatile cache in a sequential write manner. On the other hand, after the data to be stored is written, the network card device can read the data from the non-volatile cache into the data temporary storage area in a sequential read manner; after the data to be stored is read into the data temporary storage area, the data to be stored is extracted from the data temporary storage area to perform erasure code encoding processing operations. Referring to Figure 4, after the data to be stored is completely read into the data temporary storage area, the data to be stored can be transmitted to the erasure code encoder to generate multiple erasure code slices for the data to be stored in the erasure code encoder, each of which serves as data to be persisted.

[0057] It should be understood that the above-mentioned implementation method of using the erasure code mechanism to construct the data to be persisted is illustrative. In this embodiment, the network card device can also adopt other implementation methods to realize the construction of the data to be persisted. For example, a multi-backup mechanism can be used to construct the data to be persisted. Under this mechanism, a backup generator can be deployed in the computing offloading unit of the aforementioned network card device to generate multiple copies of backup data for the data to be stored, respectively as the data to be persisted. No further examples are given here. Referring to Figure 4, the computing offloading unit of the network card device can also deploy compression / decompression function modules, network offloading engines, etc. as optional configuration items, and can also deploy configurable processing modules to provide optional functions such as data filtering and data format preprocessing, so as to more effectively serve scenarios such as big data, reduce the amount of data read and transmitted, and provide near-storage computing capabilities.

[0058] In this embodiment, an improved solution for the distribution process of data to be persisted is also provided.

[0059] In this improved solution, based on the aforementioned concept of setting up a data temporary storage area in the network card device, referring to FIG5 , data block buffers can be set up for different storage servers in the data temporary storage area.

[0060] Based on this, the network card device can obtain the persistent storage address information corresponding to multiple copies of the data to be persisted, and the persistent storage location information indicates the identification of the storage server; the multiple copies of the data to be persisted corresponding to the storage device are written into the data block buffer in the data temporary area of ​​the corresponding storage server; when the data to be persisted in any data block buffer reaches the transmission volume requirement, the data to be persisted in the data block buffer is encapsulated into data blocks and transmitted to the network card device installed on the corresponding storage server.

[0061] In this way, each data block buffer can store multiple copies of data to be persisted from different data to be stored; and from the perspective of the data to be stored, the corresponding multiple copies of data to be persisted will be distributed to multiple data block caches.

[0062] Referring to Figure 5, when the data to be persisted in the data block buffer corresponding to the storage server where the network card device resides reaches the required transfer volume, the network card device is triggered to encapsulate the data in the data block buffer into data blocks and write them to the memory of the storage server's storage disk through the aforementioned direct memory access method. Once the data blocks in memory accumulate to the specified data group size, they are then spliced ​​into data batches and written to the flash memory.

[0063] Referring to Figure 5, when the data to be persisted in the data block buffer corresponding to other storage servers reaches the transmission volume requirement, a network transmission can be triggered, that is, the data to be persisted in the data block buffer is encapsulated into data blocks, and the data blocks are transmitted to the network card device installed on the corresponding storage server through the network switch.

[0064] In this improved solution, by setting up a data block buffer, persistent data can be transmitted block by block, thereby fully utilizing network bandwidth, reducing the number of network transmissions, and lowering the network overhead within the storage cluster.

[0065] FIG6 is a flow chart of a data storage method provided by another exemplary embodiment of the present disclosure. The data storage method can be implemented by a network card device having a non-volatile cache, and the network card device can be installed on a storage server in a storage cluster. Referring to FIG6 , the method may include:

[0066] Step 600: writing the received data to be stored into the non-volatile cache;

[0067] Step 601: construct multiple copies of the data to be stored for persistent storage.

[0068] Step 602: writing one copy of the plurality of copies of data to be persisted into a storage disk on the storage server;

[0069] Step 603: Distribute other copies of the data to be persisted to the network card devices installed on other storage servers, so that the network card devices installed on the other storage servers respectively write the received data to be persisted into the storage disk of the storage server where they are located.

[0070] In an optional embodiment, the method may further include:

[0071] After the data to be stored is written into the non-volatile cache, a write success notification is returned to the storage requester of the data to be stored.

[0072] In an optional embodiment, constructing multiple copies of the data to be persisted for persistent storage for the data to be stored includes:

[0073] Reading the data to be stored from the non-volatile cache;

[0074] Erasure coding is performed on the data to be stored to obtain multiple erasure code slices corresponding to the data to be stored, which are respectively used as data to be persisted.

[0075] In an optional embodiment, the non-volatile cache adopts a sequential read-write mode, and the network card device is further provided with a data temporary storage area, and the received data to be stored is written into the non-volatile cache, including:

[0076] Writing the data to be stored into the non-volatile cache in a sequential writing manner;

[0077] Reading the data to be stored from the non-volatile cache includes:

[0078] Reading data from the non-volatile cache to the data temporary storage area in a sequential read manner;

[0079] After the data to be stored is read into the data temporary storage area, the data to be stored is extracted from the data temporary storage area to perform the erasure code encoding processing operation.

[0080] In an optional embodiment, a data temporary storage area is provided in the network card device, and data block buffers are respectively provided for different storage servers in the data temporary storage area. The method further includes:

[0081] Determining persistent storage address information corresponding to each of the plurality of copies of data to be persisted, wherein the persistent storage location information indicates an identifier of a storage server;

[0082] Writing the plurality of copies of data to be persisted into the data block buffer of the corresponding storage server in the data temporary storage area respectively;

[0083] When the data to be persisted in any data block buffer reaches the transmission volume requirement, the data to be persisted in the data block buffer is encapsulated as data blocks and transmitted to the network card device equipped with the corresponding storage server.

[0084] In an optional embodiment, the data temporary storage area uses a dynamic random access memory.

[0085] In an optional embodiment, writing one copy of the plurality of copies of data to be persisted to a storage disk on the storage server includes:

[0086] Direct memory access is used to write one copy of the plurality of copies of data to be persisted into the memory of the storage disk on the storage server, so that the storage disk continues to write the persistent data in its memory into its flash memory.

[0087] In an optional embodiment, the method may further include:

[0088] Mapping the memory address space of the storage disk on the storage server to the logical address space of the network card device;

[0089] The step of writing one copy of the plurality of copies of data to be persisted into the memory of the storage disk on the storage server by using direct memory access includes:

[0090] Based on the mapping relationship between the memory address space of the storage disk and the logical address space of the network card device, a write operation is performed on the memory address space of the storage disk to write one of the multiple copies of data to be persisted into the memory of the storage disk on the storage server.

[0091] In an optional embodiment, after the data to be persisted in the memory of the storage disk reaches the data group specification required by its flash memory, the data to be persisted in the memory of the storage disk is spliced ​​into data groups and written into the flash memory.

[0092] In summary, in this embodiment, the data storage method can be roughly divided into two steps: writing the data to be stored into the storage cluster and completing the persistence of the data to be stored in the storage cluster.

[0093] Figure 7a provides an exemplary process diagram for writing data to be stored into the storage cluster. Referring to Figure 7a, in this link, after the multiple copies of backup data submitted by the computing node enter the network card devices installed on each storage server, the network card device can write the backup data (as data to be stored) into the non-volatile cache integrated in the network card device in a sequential write manner, and return a write success notification to the computing node. When all the backup data submitted by the computing node are written successfully, the storage request initiated by the computing node this time will be marked as successful; if the write timeout is still unsuccessful, the computing node can re-initiate the storage request, and the multiple servers re-selected may not be exactly the same as the previous attempt. In other words, when the computing node initiates a storage request again, the multiple servers selected for writing are not exactly the same as the last time. When the data is re-written successfully, the write operation is successful, and the computing node can initiate subsequent storage requests.

[0094] FIG7b provides an exemplary process diagram for completing the persistence of data to be stored in a storage cluster. Referring to FIG7b, one of the storage servers previously written to multiple backups is selected as a primary node. The network card device on the storage server reads the data to be stored from its internal non-volatile cache. Afterwards, the data to be stored is subjected to erasure coding to construct multiple erasure code slices for it, which can be distributed to the n storage servers corresponding to this erasure code storage group (one of which is on the local SSD). The local erasure code slice can be transferred to the SSD via direct memory access. After receiving the erasure code slice, the network card devices of the remaining n-1 storage servers can also write the same to the local SSD of the storage server where they are located via direct memory access. If all n erasure code slices corresponding to the data to be stored are successfully written, it is determined that the persistent storage is successful, and the corresponding data in the non-volatile cache can be deleted, and its capacity can be used for subsequent data writing. However, if the persistent storage is unsuccessful and times out, the master node restarts the above-mentioned persistent storage logic, that is, repeating the entire process of reading the non-volatile cache, erasure code encoding processing, network transmission, local writing, etc. until n erasure code slices are successfully written.

[0095] It is worth noting that the technical details of the above data storage method can be referred to the relevant description in the aforementioned network card device embodiment. In order to save space, they will not be repeated here, but this should not cause any loss of the protection scope of this disclosure.

[0096] Accordingly, an embodiment of the present disclosure further provides a computer-readable storage medium storing a computer program, which, when executed, can implement the steps that can be executed by the network card device in the above method embodiment.

[0097] It should be noted that some of the processes described in the above embodiments and the accompanying drawings include multiple operations that appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The sequence numbers of the operations, such as 601 and 602, are merely used to distinguish between different operations and do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel.

[0098] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0099] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0100] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0101] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0102] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0103] The foregoing is merely an embodiment of the present disclosure and is not intended to limit the present disclosure. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included within the scope of protection of the present disclosure.

Claims

1. A data storage method, applicable to a network card device mounted on a storage server in a storage cluster, wherein the network card device is provided with a non-volatile cache, the method comprising: Writing the received data to be stored into the non-volatile cache; Constructing multiple copies of the data to be persisted for persistent storage for the data to be stored; Writing one copy of the data to be persisted among the multiple copies of the data to be persisted into a storage disk on the storage server; The other copies of the data to be persisted are distributed to the network card devices installed on the other storage servers, so that the network card devices installed on the other storage servers respectively write the received data to be persisted into the storage disk of the storage server where they are located.

2. The method according to claim 1, further comprising: After the data to be stored is written into the non-volatile cache, a write success notification is returned to the storage requester of the data to be stored.

3. The method according to claim 1, constructing multiple copies of the data to be stored for persistent storage for the data to be stored, comprising: Reading the data to be stored from the non-volatile cache; Erasure coding is performed on the data to be stored to obtain multiple erasure code slices corresponding to the data to be stored, which are respectively used as data to be persisted.

4. The method according to claim 3, wherein the non-volatile cache adopts a sequential read-write mode, and the network card device is further provided with a data temporary storage area, and the received data to be stored is written into the non-volatile cache, comprising: Writing the data to be stored into the non-volatile cache in a sequential writing manner; Reading the data to be stored from the non-volatile cache includes: Reading data from the non-volatile cache to the data temporary storage area in a sequential reading manner; After the data to be stored is read into the data temporary storage area, the data to be stored is extracted from the data temporary storage area to perform the erasure code encoding processing operation.

5. The method according to claim 1, wherein a data temporary storage area is set in the network card device, and data block buffers are respectively set for different storage servers in the data temporary storage area, and the method further comprises: Determine persistent storage address information corresponding to each of the plurality of copies of data to be persisted, wherein the persistent storage location information indicates an identifier of a storage server; Writing the plurality of copies of data to be persisted into the data block buffer of the corresponding storage server in the data temporary storage area respectively; When the data to be persisted in any data block buffer reaches the transmission volume requirement, the data to be persisted in the data block buffer is encapsulated as data blocks and transmitted to the network card device equipped with the corresponding storage server.

6. The method according to any one of claims 4 or 5, wherein the data temporary storage area uses a dynamic random access memory.

7. The method according to claim 1, writing one copy of the plurality of copies of data to be persisted into a storage disk on the storage server, comprising: By adopting a direct memory access method, one copy of the plurality of copies of data to be persisted is written into the memory of the storage disk on the storage server, so that the storage disk continues to write the persistent data in its memory into its flash memory.

8. The method according to claim 7, further comprising: Mapping the memory address space of the storage disk on the storage server to the logical address space of the network card device; The method of writing one copy of the plurality of copies of data to be persisted into the memory of the storage disk on the storage server by using direct memory access includes: Based on the mapping relationship between the memory address space of the storage disk and the logical address space of the network card device, a write operation is performed on the memory address space of the storage disk to write one of the multiple copies of data to be persisted into the memory of the storage disk on the storage server.

9. According to the method of claim 7, after the data to be persisted in the memory of the storage disk reaches the data group specification required by its flash memory, the data to be persisted in the memory of the storage disk is spliced ​​into a data group and written into the flash memory.

10. A network card device, comprising a memory, a processor, a communication component and a non-volatile cache; The memory is used to store one or more computer instructions; The processor is coupled to the memory, the communication component and the non-volatile cache, and is configured to execute the one or more computer instructions to execute the data storage method according to any one of claims 1 to 9.

11. A storage cluster, comprising a plurality of storage servers, each storage server being equipped with a network card device, the network card device being used to execute the data storage method according to any one of claims 1 to 9, so as to persistently store the data to be stored in the storage cluster.

12. A computer-readable storage medium storing computer instructions, which, when executed by one or more processors, causes the one or more processors to execute the data storage method according to any one of claims 1 to 9.

Images