Distributed storage management method, electronic equipment, storage medium and program product

Abstract

The invention discloses a distributed storage management method, an electronic device, a storage medium and a program product, and relates to the technical field of distributed storage. Cyclic redundancy check codes corresponding to data are written into metadata, the amount of metadata written into a distributed key value storage database is reduced, the performance of the distributed key value storage database is improved, and the reliability of the distributed key value storage database is improved. According to the method, the network transmission protocol is expanded, the metadata separation transmission mode is supported, the data logic block pointer and the independent metadata logic block pointer are supported when the distributed storage protocol access layer processes the read-write request, data copying of distributed storage is reduced through the metadata separation transmission mode, and the read-write performance is improved.

Description

Technical Field

[0001] This application relates to the field of distributed storage technology, and in particular, to a management method for distributed storage, an electronic device, a storage medium, and a program product. Background Art

[0002] In the related art, the client requests are processed by setting a primary storage node and a standby storage node. However, when each storage node processes a client request, it needs to switch threads multiple times, such as communication threads, threads for writing data to the local disk, threads for writing data to the local database, etc. Each thread switch involves enqueueing and dequeueing requests, resulting in a large processing delay for client requests and low performance of distributed storage. Summary of the Invention

[0003] This application provides a management method for distributed storage, an electronic device, a storage medium, and a program product, so as to at least solve the problems of large processing delay for client requests and low performance of distributed storage in the related art.

[0004] This application provides a management method for distributed storage, including: Receiving a write request sent by a client; In response to receiving the write request sent by the client, determining a cyclic redundancy check code for the target service data corresponding to the write request; Generating target metadata based on the cyclic redundancy check code and a data writing mechanism; Generating target storage data based on the mapping relationship between the target service data, the target metadata, and the logical block address; Writing the target storage data to the storage node corresponding to the logical block address based on a network transmission protocol; In response to the completion of the writing, writing the mapping relationship between the service object range corresponding to the target storage data and the internal object range in the storage node into a distributed key-value storage database.

[0005] This application also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the following steps of the management method for distributed storage when executing the computer program: Receiving a write request sent by a client; In response to receiving the write request sent by the client, determining a cyclic redundancy check code for the target service data corresponding to the write request; Generating target metadata based on the cyclic redundancy check code and a data writing mechanism; Generating target storage data based on the mapping relationship between the target service data, the target metadata, and the logical block address; Writing the target storage data to the storage node corresponding to the logical block address based on a network transmission protocol; In response to the completion of writing, write the mapping relationship between the business object range corresponding to the target stored data and the internal object range in the storage node into the distributed key-value storage database.

[0006] This application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the following steps of the management method for distributed storage are implemented: Receive a write request sent by a client; In response to receiving the write request sent by the client, determine the cyclic redundancy check code of the target service data corresponding to the write request; Generate target metadata based on the cyclic redundancy check code and the data writing mechanism; Generate target stored data based on the mapping relationship between the target service data, the target metadata, and the logical block address; Write the target stored data into the storage node corresponding to the logical block address based on the network transmission protocol; In response to the completion of writing, write the mapping relationship between the business object range corresponding to the target stored data and the internal object range in the storage node into the distributed key-value storage database.

[0007] This application also provides a computer program product, including a computer program. When the computer program is executed by a processor, the following steps of the management method for distributed storage are implemented: Receive a write request sent by a client; In response to receiving the write request sent by the client, determine the cyclic redundancy check code of the target service data corresponding to the write request; Generate target metadata based on the cyclic redundancy check code and the data writing mechanism; Generate target stored data based on the mapping relationship between the target service data, the target metadata, and the logical block address; Write the target stored data into the storage node corresponding to the logical block address based on the network transmission protocol; In response to the completion of writing, write the mapping relationship between the business object range corresponding to the target stored data and the internal object range in the storage node into the distributed key-value storage database.

[0008] By writing the cyclic redundancy check code corresponding to the data into the metadata, this application reduces the amount of metadata written into the distributed key-value storage database and improves the performance of the distributed key-value storage database. This application extends the network transmission protocol to support the separated transmission mode of metadata. When the access layer of the distributed storage protocol processes read and write requests, the cyclic redundancy check code is placed in an independent metadata logical block pointer to avoid data copying and improve read and write performance. Description of the Drawings

[0009] To more clearly illustrate the embodiments of the present application, the following will briefly introduce the accompanying drawings required in the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0010] Figure 1 It is a schematic diagram of the write request processing flow in the open-source distributed storage Ceph for a distributed storage management method provided by an embodiment of the present application; Figure 2 It is a schematic diagram of the logical architecture of a distributed file storage for a distributed storage management method provided by an embodiment of the present application; Figure 3 It is an application environment diagram for a distributed storage management method provided by an embodiment of the present application; Figure 4 It is a schematic diagram of the overall process for a distributed storage management method provided by an embodiment of the present application; Figure 5 It is a schematic diagram of the write request processing flow for a distributed storage management method provided by an embodiment of the present application; Figure 6 It is a schematic diagram of the process for generating NVMe metadata for a distributed storage management method provided by an embodiment of the present application; Figure 7 It is a schematic diagram of the read request processing flow for a distributed storage management method provided by an embodiment of the present application; Figure 8 It is a schematic diagram of the data copy process for a distributed storage management method provided by an embodiment of the present application; Figure 9 It is a schematic diagram of the write request processing flow based on a buffer pointer for a distributed storage management method provided by an embodiment of the present application; Figure 10 It is a schematic diagram of the read request processing flow based on a buffer pointer for a distributed storage management method provided by an embodiment of the present application; Figure 11 It is an internal structure diagram of an electronic device in an embodiment. Detailed implementation manners

[0011] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present application.

[0012] It should be noted that in the description of the present application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0013] It should be noted that the terms "S1", "S2", etc. are only used for the purpose of describing steps, and do not particularly refer to the meaning of order or sequence, nor are they used to limit the present application. They are only used to conveniently describe the method of the present application and should not be construed as indicating the order of steps. In addition, the technical solutions between various embodiments can be combined with each other, but it must be based on the ability of those of ordinary skill in the art to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present application.

[0014] To meet the requirements of high-speed storage, the NVMe (Non-Volatile Memory Express, designed to access non-volatile memory media connected via the PCI-Express bus) protocol was born in 2011. It brings more efficient performance to non-volatile storage devices such as SSDs. However, NVMe could initially only be connected to servers via the PCIe bus, limiting the scalability of storage. To solve this problem, NVMe-oF (NVMe over Fabrics, a storage network protocol that allows hosts to access remote NVMe SSD devices over the network) was introduced in 2016. It extends the NVMe protocol to the network and allows remote access to NVMe storage devices through various network fabrics. NVMe-oF supports multiple transport protocols, including Ethernet, InfiniBand, and Fibre Channel, etc. With the continuous progress of technology, the performance of NVMe-oF will be further improved and the cost will gradually decrease. If NVMe-oF technology can be used in distributed storage, the performance of distributed storage will be improved; NVMe Meta includes two transmission methods. The first is the continuous method, that is, data and metadata use one Data Buffer (data buffer), and the metadata of each logical block is stored behind the data of that logical block. The second is the separated method, that is, data and metadata each use independent Buffers (buffers). In related technologies, such as Figure 1As shown in the figure, taking the open-source distributed storage Ceph (a unified distributed storage system) as an example, the write request processing process is as follows: The protocol access layer receives a write request from the protocol client; sends the write request to the OSD (storage node) master, carrying the object ID (unique identifier) and the offset within the object; the OSD master sends the write request to 2 standby nodes, and the message communication uses the internal communication component of Ceph. Taking 3-replica redundancy as an example, 3 OSD processes need to be written. One of the 3 OSDs is used as the master node to replicate the write request to the standby nodes; the OSD master and the OSD standby allocate hard disk space for the object according to the object ID, write the data to the local disk, and then write the object size and data CRC to the local database; the data CRC is used to perform a CRC (cyclic redundancy check) when reading data to check the data correctness; the OSD master receives all the responses and sends a response to the protocol access layer. According to the background technology, the performance of the communication component of the related technology Ceph is worse than that of the open-source SPDK (Storage Performance Development Kit, an open-source storage performance development toolkit). The open-source SPDK implements the NVMe-oF protocol and adopts a polling thread model, which is lock-free during the IO processing and has higher performance. When each OSD processes a write request, it needs to switch threads multiple times, including the communication thread, the thread for writing data to the local disk, and the thread for writing to the local database. Each thread switch involves enqueueing and dequeueing requests, resulting in a large latency in processing write requests.

[0015] To solve the above technical problems, the present application provides a management method, an electronic device, a storage medium, and a program product for distributed storage. By writing the cyclic redundancy check code corresponding to the data into the metadata, the amount of metadata written into the distributed key-value storage database is reduced, and the performance of the distributed key-value storage database is improved. The present application extends the network transmission protocol to support the separate transmission mode of metadata. When the distributed storage protocol access layer processes read and write requests, the cyclic redundancy check code is placed in an independent metadata logical block pointer to avoid data copying and improve read and write performance.

[0016] For the convenience of description, first, the nouns appearing in the embodiments of the present application are uniformly explained: OSD: Object-based Storage Device, an object storage device, which is a storage node responsible for storing and retrieving data. A node has multiple OSD processes, and each OSD is a process that mounts a data hard disk, and this process can read and write data to the disk.

[0017] PG: Placement Group, a carrier for placing objects. Multiple PGs are created in a storage pool.

[0018] Object: Business object, distributed in each PG. A large number of objects are stored in one PG, and data is stored in data blocks.

[0019] RDMA: Remote Direct Memory Access, a high-performance and low-latency network data transmission technology. It can directly transfer data from the memory of one computer to the memory of another computer without passing through the CPU (Central Processing Unit), thereby reducing the time and complexity of CPU participation in transmission and improving the efficiency and throughput of data transmission.

[0020] NVMe: Non-Volatile Memory Express, a protocol for highly parallel data transmission, mainly applied to non-volatile storage devices such as solid-state drives (SSDs). NVMe commands map input / output (I / O) and responses to the memory of the host computer and are transmitted through the PCIe interface, supporting parallel I / O with multi-core processors to improve throughput and solve the CPU bottleneck problem.

[0021] NVMe SSD (Non-Volatile Memory Express Solid State Drive): A solid-state drive using the NVMe protocol, featuring high performance, low latency, multiple queues, and high concurrency.

[0022] LBA: Logical Block Addressing. In the LBA model, data is divided into multiple logical blocks, each usually 512 bytes or 4KB. These logical blocks are the basic units for data storage and access, and each logical block has a unique identifier, namely the LBA address.

[0023] NVMe-oF: Non-Volatile Memory Express over Fabrics, a technology that extends the NVMe protocol to external storage of servers using a network fabric. It inherits the low-latency and high-throughput characteristics of the NVMe protocol and can be extended through various network fabrics, including Ethernet, Fibre Channel (FC), and InfiniBand, etc. This enables flexible allocation and expansion of storage resources to meet the needs of data centers and enterprises of different scales.

[0024] NVMe metadata: Additional information for each logical block of data, usually used to store data protection information such as CRC (Cyclic Redundancy Check), ECC (Error Correction Code), etc., to ensure the reliability of data transmission and storage. There are two ways to transmit metadata. One is the continuous way, where the metadata of each logical block is stored immediately after the logical block. The other is the separated way, where metadata and data use different caches.

[0025] Distributed KV: A database that stores data in the form of key-value pairs on multiple nodes, with high availability, scalability, and fault tolerance.

[0026] The embodiments of this application are applied to a distributed file storage system, such as Figure 2 shown. Figure 2 It is a logical architecture diagram of distributed file storage. The protocol access layer processes requests from protocol clients. Taking a write request as an example, it writes data into the storage pool, sends a file metadata update request to the MDS (File Metadata Service). The MDS first updates the metadata cache, then writes the file metadata into the distributed KV database, and finally returns success to the protocol access layer. Among them, the protocol access layer is a data transmission device configured with the NVMeoF protocol.

[0027] To enable those skilled in the art of this technology to better understand the solution of this application, the following further elaborates on this application in combination with the accompanying drawings and specific implementation manners.

[0028] The management method of distributed storage provided by this application can be applied to an application environment such as Figure 1 shown. Among them, the terminal 102 communicates with the data processing platform set on the server 104 through the network. Among them, the terminal 102 can be but is not limited to various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0029] Such as Figure 4 shown, the embodiments of this application provide a management method for distributed storage. Taking the case where this method is applied to the Figure 3 terminal as an example, it includes the following steps: (Technical terms involved must be explained).

[0030] S1: Receive a write request sent by the client.

[0031] It should be noted that a write request refers to a piece of information sent by the client to the server for writing data or performing an operation. A write request usually includes three parts: a request line, a request header, and a request body. Among them, the request line refers to the method of the request (such as GET, POST, PUT, etc.) and the target URL of the request. The request header is used to provide additional information about the request, such as the user agent, Cookie, content type, etc. The request body contains the data to be sent to the server, such as form data or an uploaded file. The write request in this application contains the business data to be written and the business object corresponding to this business data. A business object refers to the business that uses this business data.

[0032] S2: In response to receiving a write request sent by a client, determine the cyclic redundancy check code for the target service data corresponding to the write request.

[0033] It should be noted that the cyclic redundancy check code refers to the check code obtained based on CRC, which is a check code calculated by a fast algorithm that generates a short fixed - length check code for data such as network data packets or computer files. It is mainly used to detect or verify possible errors that may occur after data transmission or storage.

[0034] S3: Based on the cyclic redundancy check code and the data writing mechanism, generate the target metadata.

[0035] It should be noted that the data writing mechanism is to write the cyclic redundancy check code into the NVMe metadata of the corresponding logical block to obtain the final target metadata.

[0036] S4: Based on the mapping relationship between the target service data, the target metadata, and the logical block address, generate the target storage data.

[0037] It should be noted that a logical block refers to the smallest storage unit of the hard disk space, and the logical block address refers to the LBA address, which is determined by the logical block where the data needs to be written.

[0038] S5: Based on the network transmission protocol, write the target storage data to the storage node corresponding to the logical block address.

[0039] It should be noted that the network transmission protocol refers to the NVMe - oF protocol, which is a storage network protocol. The storage nodes refer to the primary storage node and the secondary storage nodes, such as the primary OSD, standby OSD1, standby OSD2, etc. shown in the figure.

[0040] S6: In response to the completion of the write, write the mapping relationship between the service object range corresponding to the target storage data and the internal object range in the storage node into the distributed key - value storage database.

[0041] It should be noted that the distributed key - value storage database refers to the distributed KV. The mapping relationship is used to record the correspondence between the service object range corresponding to the stored data and the internal object range. The internal object refers to the hard disk space in the pre - applied storage node. The service object range refers to the data range corresponding to the service object, and the internal object range refers to the range of data stored in the internal object. The distributed KV is used to store the correspondence between the internal object and the addresses of the logical blocks in each storage node.

[0042] In the above embodiments, writing the data CRC into the NVMe metadata reduces the amount of metadata written to the distributed KV, improving the performance of the distributed KV. Exemplarily, taking 4MB of data as an example, for every 4KB NVMe logical block, 4 bytes of CRC need to be calculated. 1MB of data corresponds to 256 CRCs, occupying 4KB of space. When the value of the distributed KV is greater than 1KB, modification will cause a large write amplification, resulting in a decline in the performance of the distributed KV. Therefore, reducing the amount of metadata written can improve the performance of the distributed KV.

[0043] In some specific embodiments, such as Figure 5 shown, before determining the cyclic redundancy check code of the target service data corresponding to the write request in response to receiving the write request sent by the client, the method includes: Detecting whether the process of the protocol access layer corresponding to the network transmission protocol has been started, where the network transmission protocol is the above-mentioned NVMe-oF protocol; In response to the process of the protocol access layer corresponding to the network transmission protocol having been started, performing an internal object pre-application operation, that is, after the process of the protocol access layer is started, it begins to perform the internal object pre-application operation, so that when receiving a client write request, it can directly write to the internal object. The role of the internal object is to pre-apply for hard disk space on the data OSD.

[0044] In some specific embodiments, in response to the process of the protocol access layer corresponding to the network transmission protocol having been started, performing the internal object pre-application operation includes: In response to the process of the protocol access layer corresponding to the network transmission protocol having been started, sending an internal object pre-application request to the primary storage node; In response to the primary storage node receiving the internal object pre-application request, sending the internal object pre-application request to multiple secondary storage nodes through the primary storage node, that is, after the primary OSD receives the internal object pre-application request, it sends a pre-application request to the data OSD (i.e., the secondary storage node). Among them, a relatively large space can be applied for in one internal object pre-application, for example, 64MB. Assuming that the size of each service request is 1MB, one pre-application can process 64 service requests; In response to the secondary storage node receiving the internal object pre-application request, allocating hard disk space for the internal object through the secondary storage node according to the size of the hard disk space corresponding to the internal object pre-application request; In response to the allocation being completed, returning the response result of the internal object pre-application request to the primary storage node. The response result includes the logical block address of the allocated hard disk space, that is, the data OSD allocates hard disk space for the internal object and returns the response result of the pre-application request. The response result includes the lba address of the locally allocated space.

[0045] In some specific embodiments, after returning the response result of the internal object pre-application request to the main storage node, the method further includes: The main storage node writes the logical block address into the distributed key-value storage database and returns the logical block address to the protocol access layer. That is, the main OSD writes the lba addresses of the internal object on each data OSD into the distributed KV. Taking 3 replicas as an example, 3 lba addresses are recorded. Then, the lba addresses on each OSD are carried in the pre-application response result returned by the main OSD. In the above embodiment, after the protocol access layer starts the process, it performs the internal object pre-application operation, so that when receiving a client data write request, it can directly write the internal object, improving the data write efficiency.

[0046] In some specific embodiments, obtain the hard disk space size required for multiple task applications corresponding to the internal object pre-application request; Classify the space size required for the target task according to a preset space step. The preset space step can be set according to actual needs, such as 4KB. That is, the applied space of 0 - 4KB is a range and belongs to the first category, and the applied space of 4 - 8KB is a range and belongs to the second category, etc. If the number of occurrences in the target category corresponding to the space size of the target task is greater than the preset threshold, map the space range value corresponding to the target category to the target task to generate a mapping relationship and save the mapping relationship. If it is detected again that the target task applies for hard disk space, determine the corresponding space range value based on the mapping relationship, select the maximum value in the range value as the hard disk space to be applied, and allocate hard disk space based on the hard disk space to be applied.

[0047] In the above embodiment, by determining the mapping relationship between the target task and the space range value, the accuracy of hard disk space allocation can be improved, thereby avoiding the situation of resource waste or insufficient resource allocation and enhancing the performance of the distributed storage system.

[0048] In some specific embodiments, in response to receiving a write request sent by the client, determining the cyclic redundancy check code corresponding to the target service data of the write request includes: Initialize the cyclic redundancy check register, which is used to calculate and determine the cyclic redundancy check code of the target service data; Slice the target service data according to the size of the logical block, and process the sliced target service data based on the cyclic redundancy check register to obtain the cyclic redundancy check code corresponding to the sliced target service data.

[0049] In some specific embodiments, based on the cyclic redundancy check code and the data writing mechanism, generating the target metadata includes: Determine the target logical block corresponding to the target cyclic redundancy check code, and the metadata of the target logical block; Write the target cyclic redundancy check code into the metadata of the target logical block to obtain the target metadata corresponding to the target logical block.

[0050] Specifically, as Figure 6 shown, after the protocol access layer receives the write request from the client, it calculates the CRC and generates nvme metadata. In the process of generating nvme metadata, data copying is required. The original data (i.e., the target service data) is sliced at a granularity of 4KB (the size of the nvme logical block), and then the CRC is calculated for each 4KB data and written into the nvme metadata of the logical block to obtain the target metadata.

[0051] In the above embodiment, by calculating the CRC of the service data and writing it into the nvme metadata, it is convenient to verify the data after performing the data transmission operation, thereby improving the security of reading the data.

[0052] In some specific embodiments, based on the mapping relationship between the target service data, the target metadata, and the logical block address, generating the target storage data includes: Obtain the hard disk space size required for the target service data and the target metadata corresponding to the target service data; Based on the hard disk space size, determine the target service data and the logical block corresponding to the target service data; Obtain the address of the logical block; Based on the target service data, the target metadata, and the address of the logical block, generate a mapping relationship; Based on the mapping relationship, generate the target storage data.

[0053] In some specific embodiments, based on the network transmission protocol, writing the target storage data into the storage node corresponding to the logical block address includes: Obtain the logical block address in the target storage data, and define the network transmission protocol as the storage network protocol; Based on the storage network protocol, write the target storage data into the storage node corresponding to the logical block address.

[0054] In some specific embodiments, in response to the completion of writing, writing the mapping relationship between the service object range corresponding to the target storage data and the internal object range in the storage node into the distributed key-value storage database includes: In response to the completion of writing, obtain the data range stored in the corresponding hard disk space of the target storage data and the data range corresponding to the target storage data; Generate a mapping relationship based on the data range of the target stored data in the corresponding hard disk space and the data range corresponding to the target stored data; Write the mapping relationship into the distributed key-value storage database.

[0055] Specifically, the protocol access layer writes messages including data, nvme metadata, and lba addresses to each data OSD through the nvme-of protocol; after the protocol access layer receives all responses, that is, after the write operation is completed, it writes the forward relationship (mapping relationship) into the distributed KV, where the forward relationship records the correspondence between the service object range and the internal object range. Exemplarily, the range from 0 to 1MB of the service object obj_service is written to the range from 2MB to 3MB of obj_internal.

[0056] In the above embodiment, by writing the mapping relationship into the distributed KV, the corresponding data can be directly obtained through the mapping relationship when responding to subsequent read requests, improving the processing efficiency of data read requests.

[0057] In some specific embodiments, the method further includes: In response to receiving a read request sent by the client, send a mapping relationship query request to the distributed key-value storage database through the protocol access layer; Receive the response result of the mapping relationship query request returned by the distributed key-value storage database. The response result at least includes the mapping relationship between the service object range corresponding to the target stored data and the internal object range in the storage node, and the logical block address of the read range corresponding to the read request on the internal object; Based on the logical block address, obtain the data corresponding to the read request and the metadata corresponding to the data.

[0058] In some specific embodiments, obtaining the data corresponding to the read request based on the logical block address includes: Based on the storage network protocol and the logical block address, send a read request to the primary storage node and / or the standby storage node, and receive the data corresponding to the read request and the metadata corresponding to the data.

[0059] In some specific embodiments, after obtaining the data corresponding to the read request based on the logical block address, the method includes: In response to the data corresponding to the read request being received completely, verify the data according to the cyclic redundancy check code in the metadata corresponding to the data; In response to successful verification, delete the metadata and return the data to the client.

[0060] Specifically, as Figure 7 and Figure 8As shown in the figure, for a read request, the processing flow is as follows: The protocol access layer receives a read request from the client; the protocol access layer sends a query mapping relationship request to the distributed KV; the protocol access layer obtains the lba address of the client's read range on the internal object according to the mapping relationship, and sends an nvme-of read request to the data OSD; after the protocol access layer receives the response result from the data OSD, it performs a crc check based on the nvme metadata; after successful verification, the protocol access layer makes a data copy, deletes the nvme metadata, carries the copied data in the response message sent to the client, and sends it to the client.

[0061] In the above embodiment, the data to be read is determined through the mapping relationship stored in the distributed KV, and a verification operation is performed after reading the data, which improves the data reading efficiency while ensuring the reliability of the read data.

[0062] In some specific embodiments, the method further includes: In response to the first logical block pointer of the target metadata and the second logical block pointer of the target service data included in the write request, the read request sent by the client is responded to according to the first logical block pointer and the second logical block pointer, where the first logical block pointer refers to the nvme metadata buffer pointer, and the second logical block pointer refers to the original data buffer pointer.

[0063] Specifically, as Figure 9 shown, the protocol access layer receives a write request from the client, calculates the crc, generates nvme metadata, and the calculation process does not require copying the original data. When sending an nvme-of write request, it carries the original data buffer pointer and the nvme metadata buffer pointer. As Figure 10 shown, when the protocol access layer processes a read request, it sends an nvme-of read request to the OSD, carrying the data buffer pointer and the nvme metadata buffer pointer. After the protocol access layer receives the correct result of the crc check data, it sends the data in the data buffer to the client without making a data copy.

[0064] In the above embodiment, by extending the nvme-of protocol to support the separated transmission mode of nvme metadata, when the distributed storage protocol access layer processes read and write requests, the crc is placed in an independent nvme metadata buffer, avoiding data copying and improving the read and write performance.

[0065] In the above-described management method of distributed storage, the method includes: receiving a write request sent by a client; in response to receiving the write request sent by the client, determining a cyclic redundancy check code of the target service data corresponding to the write request; generating target metadata based on the cyclic redundancy check code and a data writing mechanism; generating target storage data based on the mapping relationship between the target service data, the target metadata, and a logical block address; writing the target storage data to a storage node corresponding to the logical block address based on a network transmission protocol; in response to the completion of the writing, writing the mapping relationship between the service object range corresponding to the target storage data and the internal object range in the storage node to a distributed key-value storage database. This application supports the protocol access layer to directly write data to an SSD through the nvme-of protocol, and utilizes the low latency and high throughput characteristics of nvme-of to improve the performance of distributed storage. This application supports writing data CRC to nvme metadata, reducing the writing to distributed KV, and improving the performance of distributed KV. This application extends the nvme-of protocol to support the separated transmission mode of nvme metadata, and the protocol access layer of distributed storage reduces memory copying, improving the writing performance.

[0066] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation manner.

[0067] It should be understood that although Figures 4 - 10 the steps in the flowchart of Figures 4 - 10 are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover,

[0068] In one embodiment, an electronic device is provided. The electronic device may be a terminal, and its internal structure diagram may be as Figure 11As shown in the figure. The electronic device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. The computer program, when executed by the processor, implements a method for managing distributed storage. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer covered on the display screen, or a button, a trackball, or a touchpad provided on the housing of the electronic device, or an external keyboard, a touchpad, or a mouse, etc.

[0069] Those skilled in the art can understand that Figure 11 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0070] An embodiment of the present application provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in the embodiment of the method for managing distributed storage, including: S1: Receive a write request sent by a client; S2: In response to receiving the write request sent by the client, determine the cyclic redundancy check code of the target service data corresponding to the write request; S3: Generate target metadata based on the cyclic redundancy check code and the data writing mechanism; S4: Generate target storage data based on the mapping relationship between the target service data, the target metadata, and the logical block address; S5: Write the target storage data into the storage node corresponding to the logical block address based on the network transmission protocol; S6: In response to the completion of writing, write the mapping relationship between the service object range corresponding to the target storage data and the internal object range in the storage node into the distributed key-value storage database.

[0071] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps in the embodiment of the method for managing distributed storage when running, including: S1: Receive a write request sent by a client; S2: In response to receiving a write request sent by a client, determine the cyclic redundancy check code of the target service data corresponding to the write request; S3: Generate target metadata based on the cyclic redundancy check code and the data writing mechanism; S4: Generate target storage data based on the mapping relationship among the target service data, the target metadata, and the logical block address; S5: Write the target storage data to the storage node corresponding to the logical block address based on the network transmission protocol; S6: In response to the completion of the writing, write the mapping relationship between the service object range corresponding to the target storage data and the internal object range in the storage node to the distributed key-value storage database.

[0072] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media such as a USB flash drive, a read-only memory (ROM for short), a random access memory (RAM for short), a mobile hard disk, a magnetic disk, or an optical disc that can store computer programs.

[0073] An embodiment of the present application also provides a computer program product. The above computer program product includes a computer program. When the computer program is executed by a processor, the steps in the embodiment of the management method for distributed storage are implemented, including: S1: Receive a write request sent by a client; S2: In response to receiving a write request sent by a client, determine the cyclic redundancy check code of the target service data corresponding to the write request; S3: Generate target metadata based on the cyclic redundancy check code and the data writing mechanism; S4: Generate target storage data based on the mapping relationship among the target service data, the target metadata, and the logical block address; S5: Write the target storage data to the storage node corresponding to the logical block address based on the network transmission protocol; S6: In response to the completion of the writing, write the mapping relationship between the service object range corresponding to the target storage data and the internal object range in the storage node to the distributed key-value storage database.

[0074] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the steps in the embodiment of the management method for distributed storage are implemented, including: S1: Receive a write request sent by a client; S2: In response to receiving a write request sent by a client, determine the cyclic redundancy check code of the target service data corresponding to the write request; S3: Generate target metadata based on the cyclic redundancy check code and the data writing mechanism; S4: Generate target stored data based on the mapping relationship among the target service data, the target metadata, and the logical block address; S5: Write the target stored data to the storage node corresponding to the logical block address based on the network transmission protocol; S6: In response to the completion of the writing, write the mapping relationship between the service object range corresponding to the target stored data and the internal object range in the storage node to the distributed key-value storage database.

[0075] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0076] The above has introduced in detail a management method, device, electronic device, and storage medium for distributed storage provided by this application. Specific examples are used herein to elaborate on the principle and implementation manner of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of this application, several improvements and modifications can be made to this application, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A management method for distributed storage, characterized in that, The method includes: Receiving a write request sent by a client; In response to receiving the write request sent by the client, determining a cyclic redundancy check code for target service data corresponding to the write request; Generating target metadata based on the cyclic redundancy check code and a data writing mechanism; Generating target storage data based on a mapping relationship among the target service data, the target metadata, and a logical block address; Writing the target storage data to a storage node corresponding to the logical block address based on a network transmission protocol; In response to completion of the writing, writing a mapping relationship between a service object range corresponding to the target storage data and an internal object range in the storage node to a distributed key-value storage database.

2. The management method of distributed storage according to claim 1, wherein Before determining, in response to receiving the write request sent by the client, the cyclic redundancy check code for the target service data corresponding to the write request, the method includes: Detecting whether a process of a protocol access layer corresponding to the network transmission protocol has been started; In response to the process of the protocol access layer corresponding to the network transmission protocol having been started, performing an internal object pre-application operation.

3. The management method of distributed storage according to claim 2, wherein Performing the internal object pre-application operation in response to the process of the protocol access layer corresponding to the network transmission protocol having been started includes: In response to the process of the protocol access layer corresponding to the network transmission protocol having been started, sending an internal object pre-application request to a primary storage node; In response to the primary storage node receiving the internal object pre-application request, sending the internal object pre-application request to a plurality of standby storage nodes through the primary storage node; In response to the standby storage node receiving the internal object pre-application request, allocating hard disk space for an internal object through the standby storage node according to a hard disk space size corresponding to the internal object pre-application request; In response to completion of the allocation, returning a response result of the internal object pre-application request to the primary storage node, where the response result includes a logical block address of the allocated hard disk space.

4. The management method of distributed storage according to claim 3, wherein, After returning the response result of the internal object pre-application request to the primary storage node, the method further includes: Writing the logical block address to the distributed key-value storage database through the primary storage node, and returning the logical block address to the protocol access layer.

5. The management method of distributed storage according to claim 1, characterized in that, Determining, in response to receiving the write request sent by the client, the cyclic redundancy check code for the target service data corresponding to the write request includes: Initializing a cyclic redundancy check register; Segmenting the target service data according to a size of a logical block, and processing the segmented target service data based on the cyclic redundancy check register to obtain a cyclic redundancy check code corresponding to the segmented target service data.

6. The management method of distributed storage according to claim 5, characterized in that Generating target metadata based on the cyclic redundancy check code and the data writing mechanism includes: Determining a target logical block corresponding to the target cyclic redundancy check code and metadata of the target logical block; Writing the target cyclic redundancy check code into the metadata of the target logical block to obtain target metadata corresponding to the target logical block.

7. The management method of distributed storage according to claim 1, wherein Generating target storage data based on the mapping relationship among the target service data, the target metadata, and the logical block address includes: The size of the hard disk space required to obtain the target service data and the target metadata corresponding to the target service data; Based on the size of the hard disk space, determine the target service data and the logical blocks corresponding to the target service data; Obtain the addresses of the logical blocks; Based on the target service data, the target metadata, and the addresses of the logical blocks, generate a mapping relationship; Based on the mapping relationship, generate the target stored data.

8. The management method of distributed storage according to claim 1, characterized in that, Writing the target stored data to the storage node corresponding to the logical block address based on the network transmission protocol includes: Obtain the logical block address in the target stored data, and define the network transmission protocol as a storage network protocol; Based on the storage network protocol, write the target stored data to the storage node corresponding to the logical block address.

9. The management method of distributed storage according to claim 1, characterized in that, In response to the completion of the writing, writing the mapping relationship between the service object range corresponding to the target stored data and the internal object range in the storage node to the distributed key-value storage database includes: In response to the completion of the writing, obtain the data range stored in the corresponding hard disk space of the target stored data and the data range corresponding to the target stored data; Based on the data range stored in the corresponding hard disk space of the target stored data and the data range corresponding to the target stored data, generate a mapping relationship; Write the mapping relationship to the distributed key-value storage database.

10. The management method of distributed storage according to claim 1, characterized in that The method further includes: In response to receiving a read request sent by the client, send a mapping relationship query request to the distributed key-value storage database through the protocol access layer; Receive the response result of the mapping relationship query request returned by the distributed key-value storage database, and the response result at least includes the mapping relationship between the service object range corresponding to the target stored data and the internal object range in the storage node, and the logical block address of the read range corresponding to the read request on the internal object; Based on the logical block address, obtain the data corresponding to the read request and the metadata corresponding to the data.

11. The management method of distributed storage according to claim 10, characterized in that, Based on the logical block address, obtaining the data corresponding to the read request includes: Based on the storage network protocol and the logical block address, send the read request to the primary storage node and / or the standby storage node, and receive the data corresponding to the read request and the metadata corresponding to the data.

12. The management method of distributed storage according to claim 11, wherein, After obtaining the data corresponding to the read request based on the logical block address, the method includes: In response to the completion of receiving the data corresponding to the read request, verify the data according to the cyclic redundancy check code in the metadata corresponding to the data; In response to successful verification, delete the metadata and return the data to the client.

13. The management method of distributed storage according to claim 1, characterized in that, The method further includes: In response to the write request including the first logical block pointer of the target metadata and the second logical block pointer of the target service data, respond to the read request sent by the client according to the first logical block pointer and the second logical block pointer.

14. An electronic device, characterized in that, Includes: A memory for storing a computer program; A processor for implementing the steps of the management method of distributed storage as described in any one of claims 1 to 13 when executing the computer program.

15. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, wherein when the computer program is executed by a processor, the steps of the management method of the distributed storage according to any one of claims 1 to 13 are implemented.

16. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, the steps of the management method of the distributed storage according to any one of claims 1 to 13 are implemented.

Images