Data storage method and device, equipment and storage medium

Abstract

The embodiment of the invention relates to the technical field of data storage, in particular to a data storage method and device, equipment and a storage medium, and aims at improving the data read-write efficiency of distributed storage. The method comprises the following steps: receiving a data writing request sent by a client; performing erasure code calculation on business data corresponding to the data writing request to obtain storage data; through a preset network transmission protocol, the storage data is written into a hard disk space pre-applied from at least one data storage node, and the minimum storage unit of the hard disk space is a data block; and recording a mapping relationship between the business object range corresponding to the storage data and the internal object range in the data storage node in the metadata storage node.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of data storage, and in particular, to a data storage method, apparatus, device and storage medium. Background Art

[0002] The NVMe-oF (NVMe over Fabrics) protocol is a protocol that extends the NVMe (Non-Volatile Memory Express) protocol to the network. This protocol can provide efficient data storage and reading services for non-volatile storage devices. In related technologies, a primary data storage node and a backup data storage node are generally set up. The data to be written is sent to the primary data storage node through the NVMe-oF protocol, and then the data is sent to each backup storage node through the primary data storage node.

[0003] In the related art, data transmission between the primary data storage node and the backup data storage node consumes a large amount of bandwidth, message forwarding and copying also consumes processor computing resources, and the request processing delay when processing read and write requests is large. Summary of the invention

[0004] The embodiments of the present application provide a data storage method, apparatus, device and storage medium, aiming to improve the data reading and writing efficiency of distributed storage.

[0005] A first aspect of an embodiment of the present application provides a data storage method, the method comprising: Receive data writing request sent by the client; Performing erasure code calculation on the business data corresponding to the data write request to obtain storage data; Writing the stored data into a hard disk space pre-applied from at least one data storage node through a preset network transmission protocol, wherein the minimum storage unit of the hard disk space is a data block; A mapping relationship between a business object range corresponding to the stored data and an internal object range in the data storage node is recorded in the metadata storage node.

[0006] Optionally, before receiving the data writing request sent by the client, the method further includes: When the process corresponding to the network transmission protocol is started, sending an internal object pre-application request to the metadata storage node; The metadata storage node, upon receiving the internal object pre-application request, sends the internal object pre-application request to the at least one data storage node; Upon receiving the internal object pre-application request, the at least one data storage node determines a hard disk space corresponding to the internal object pre-application request; The at least one data storage node allocates the corresponding hard disk space in the hard disk; The at least one data storage node determines the address of the data block corresponding to the hard disk space; The at least one data storage node returns a pre-application response to the metadata storage node, wherein the pre-application response carries the address of the data block; When receiving the pre-application response, the metadata storage node records the address of the data block carried in the pre-application response; Receive the address of the data block sent by the metadata storage node.

[0007] Optionally, the step of writing the stored data into a hard disk space pre-applied from at least one data storage node according to a preset network transmission protocol, wherein the minimum storage unit of the hard disk space is a data block, comprises: Determine the space capacity of the hard disk space corresponding to the stored data; Determine, according to the space capacity, at least one data block corresponding to the stored data; Obtaining an address of the at least one data block; The stored data is written into the data block corresponding to the address of the at least one data block through the network transmission protocol.

[0008] Optionally, the method further comprises: Receiving a data reading request sent by the client; When receiving the data read request, sending a mapping relationship query request to the metadata storage node; Upon receiving the mapping relationship query request, the metadata storage node returns a mapping relationship query response, wherein the mapping relationship query response carries a mapping relationship between a business object range and an internal object range and an address of a data block of the internal object in the at least one data storage node; Determining the address of the data block corresponding to the data read request according to the query mapping relationship response; Sending a data acquisition request to the at least one data storage node according to the address of the data block; The acquired data is sent to the client.

[0009] Optionally, the method further comprises: The management node receives heartbeat information sent by at least one of the data storage nodes; The management node determines that the data storage node is a faulty node when the management node does not receive the heartbeat information sent by the data storage node; The management node deletes the faulty node; The management node adds the newly added data storage node to a placement group, where the placement group is used to place the at least one data storage node.

[0010] Optionally, the method further comprises: The management node sends the updated data storage node view to the metadata storage node; The metadata storage node reconstructs data of internal objects on the failed node according to the data storage node view.

[0011] Optionally, the metadata storage node reconstructs data of internal objects on the failed node according to the data storage node view, including: The metadata storage node sends a hard disk space application message to the newly added data storage node, wherein the hard disk space application message is used to apply for a corresponding hard disk space for each of the internal objects; The metadata storage node deletes the address of the data block corresponding to the failed node; The metadata storage node records the address of the data block corresponding to the newly added data storage node in the mapping relationship; The metadata storage node reads the backup data of the failed node from the data storage nodes other than the newly added data storage node; The metadata storage node performs erasure code calculation on the backup data to obtain storage data corresponding to the faulty node; The metadata node stores the storage data corresponding to the failed node in the newly added data storage node.

[0012] A second aspect of an embodiment of the present application provides a data storage device, the device comprising: A data writing request receiving module, used for receiving a data writing request sent by a client; A storage data acquisition module, used to perform erasure code calculation on the business data corresponding to the data write request to obtain storage data; A data writing module, used to write the stored data into a hard disk space pre-applied from at least one data storage node through a preset network transmission protocol, wherein the minimum storage unit of the hard disk space is a data block; The mapping relationship updating module is used to record in the metadata storage node the mapping relationship between the business object range corresponding to the stored data and the internal object range in the data storage node.

[0013] Optionally, the device further comprises: A pre-application sending module, configured to send an internal object pre-application request to the metadata storage node when a process corresponding to the network transmission protocol is started; a pre-application forwarding module, configured for the metadata storage node to send the internal object pre-application request to the at least one data storage node when the metadata storage node receives the internal object pre-application request; a hard disk space determination module, configured for the at least one data storage node to determine the hard disk space corresponding to the internal object pre-application request when receiving the internal object pre-application request; A hard disk space allocation module, used for allocating the corresponding hard disk space in the hard disk to the at least one data storage node; An address determination module, used for the at least one data storage node to determine the address of the data block corresponding to the hard disk space; A pre-application response returning module, configured for the at least one data storage node to return a pre-application response to the metadata storage node, wherein the pre-application response carries the address of the data block; An address recording module, configured for the metadata storage node to record the address of the data block carried in the pre-application response when the metadata storage node receives the pre-application response; The data address receiving module is used to receive the address of the data block sent by the metadata storage node.

[0014] Optionally, the data writing module includes: A hard disk space determination submodule, used to determine the space capacity of the hard disk space corresponding to the stored data; A data block determination submodule, configured to determine at least one data block corresponding to the stored data according to the space capacity; A data block address acquisition submodule, used to acquire the address of the at least one data block; The data transmission submodule is used to write the stored data into the data block corresponding to the address of the at least one data block through the network transmission protocol.

[0015] Optionally, the device further comprises: A read request receiving module, used for receiving a data read request sent by the client; A mapping relationship query request sending module, configured to send a mapping relationship query request to the metadata storage node upon receiving the data read request; A mapping relationship query response module is used for the metadata storage node to return a mapping relationship query response when receiving the mapping relationship query request, wherein the mapping relationship query response carries a mapping relationship between a business object range and an internal object range and an address of a data block of the internal object in the at least one data storage node; A data block address determination module, configured to determine the address of the data block corresponding to the data read request according to the query mapping relationship response; A data acquisition request sending module, used to send a data acquisition request to the at least one data storage node according to the address of the data block; The data forwarding module is used to send the acquired data to the client.

[0016] Optionally, the device further comprises: A heartbeat information receiving module, used for the management node to receive the heartbeat information sent by at least one of the data storage nodes; A fault node determination module, configured for the management node to determine that the data storage node is a fault node when the management node does not receive the heartbeat information sent by the data storage node; A faulty node deletion module, used for the management node to delete the faulty node; A newly added node module is used for the management node to add a newly added data storage node to a placement group, and the placement group is used to place the at least one data storage node.

[0017] Optionally, the device further comprises: A node view updating module, used for the management node to send the updated data storage node view to the metadata storage node; The data reconstruction module is used for the metadata storage node to reconstruct the data of the internal objects on the failed node according to the data storage node view.

[0018] Optionally, the data reconstruction module includes: A hard disk space application submodule, used for the metadata storage node to send a hard disk space application message to the newly added data storage node, wherein the hard disk space application message is used to apply for a corresponding hard disk space for each of the internal objects; A data block address deletion submodule, used for the metadata storage node to delete the address of the data block corresponding to the faulty node; A newly added address recording submodule is used for the metadata storage node to record the address of the data block corresponding to the newly added data storage node in the mapping relationship; A backup data reading submodule, used for the metadata storage node to read the backup data of the failed node from the data storage nodes other than the newly added data storage node; A storage data recovery submodule, configured for the metadata storage node to perform erasure code calculation on the backup data to obtain storage data corresponding to the faulty node; A newly added data node data storage submodule is used for the metadata node to store the storage data corresponding to the failed node in the newly added data storage node.

[0019] A third aspect of an embodiment of the present application provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps in the method described in the first aspect of the present application are implemented.

[0020] A fourth aspect of an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method described in the first aspect of the present application are implemented.

[0021] The data storage method provided by the present application is adopted to receive a data write request sent by a client; perform error correction and erasure code calculation on the business data corresponding to the data write request to obtain storage data; write the storage data into a hard disk space pre-applied from at least one data storage node through a preset network transmission protocol, wherein the minimum storage unit of the hard disk space is a data block; and record in a metadata storage node a mapping relationship between a business object range corresponding to the storage data and an internal object range in the data storage node.

[0022] In this method, erasure codes are calculated for data to be written to obtain storage data, which is then written directly into a data block pre-applied from a data storage node through a preset network transmission protocol, and the correspondence between the business object range and the internal object range corresponding to the storage data is recorded, thereby realizing the use of a network protocol to directly store data in the corresponding data storage node, eliminating the need for data forwarding between data nodes, reducing the delay in data reading and writing, and improving the data reading and writing speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0024] Figure 1This is a logical architecture diagram of distributed file storage; Figure 2 It is a schematic diagram of a data writing process in the related art; Figure 3 It is a schematic diagram of a data writing process in the related art; Figure 4 is a flow chart of a data storage method proposed in one embodiment of the present application; Figure 5 It is a flowchart of a data storage method proposed in an embodiment of the present application; Figure 6 This is a schematic diagram of a data reading process proposed in an embodiment of the present application; Figure 7 This is a schematic diagram of a fault repair process proposed in an embodiment of the present application; Figure 8 is a schematic diagram of a data storage device proposed in an embodiment of the present application; Fig. 9 It is a schematic diagram of an electronic device proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] For the convenience of explanation, the terms appearing in the embodiments of the present application are first explained uniformly: OSD: Object-based Storage Device, object storage device, is a storage node responsible for storing and retrieving data. A node has multiple osd processes, each osd is a process that mounts a data hard disk and can read and write data to the disk.

[0027] Monitor (MON, management node): The main task is to maintain the consistency of the cluster view. When maintaining consistency, the Paxos algorithm (protocol) is used and instantiated into the database for subsequent access. Monitor is one of the key management nodes, monitoring the health status of all node osd services, monitoring metadata and configuration, and ensuring the consistency of various parts of the cluster.

[0028] PG: Placement Group, a carrier for placing objects. Multiple PGs are created in a storage pool.

[0029] RDMA: Remote Direct Memory Access is a high-performance, low-latency network data transmission technology that can directly transfer data from the memory of one computer to the memory of another computer without passing through the CPU, thereby reducing the time and complexity of CPU participation in transmission and improving the efficiency and throughput of data transmission.

[0030] NVMe: Non-Volatile Memory Express, is a protocol for highly parallel data transmission, mainly used in non-volatile storage devices such as solid-state drives (SSDs). NVMe commands map input / output (I / O) and responses to the host computer's memory and transmit them through the PCIe interface. It supports parallel I / O with multi-core processors to increase throughput and solve CPU bottleneck problems.

[0031] NVMe SSD: (Non-Volatile Memory Express Solid State Drive) is a solid state drive that uses the NVMe protocol and has the characteristics of high performance, low latency, multi-queue, and high concurrency.

[0032] NVMe-oF: Non-Volatile Memory Express over Fabrics, is a technology that uses network structures to extend the NVMe protocol to server external storage. It inherits the low latency and high throughput characteristics of the NVMe protocol and can be extended through various network structures, including Ethernet, Fibre Channel (FC), and InfiniBand, etc. This allows storage resources to be flexibly allocated and expanded to meet the needs of data centers and enterprises of different sizes.

[0033] EC: Erasure Coding is a method of optimizing the data recovery process by splitting data and adding redundant information. Unlike traditional copy technology, erasure coding stores data in multiple locations in a more efficient way, mainly for fault tolerance and data recovery.

[0034] This application embodiment is applied to a distributed file storage system, refer to Figure 1 , Figure 1 This is a logical architecture diagram of distributed file storage, such as Figure 1As shown in the figure, the protocol access layer processes requests from the protocol client. Taking a write request as an example, the data is written to the OSD (storage pool), and then a file metadata update request is sent to the MDS (file metadata service). The MDS first updates the metadata cache, writes the metadata operation log (used for metadata recovery processing during abnormal restart) to the storage pool, and then returns success to the protocol access layer. The protocol access layer is a data transmission device configured with the NVMe-oF protocol.

[0035] refer to Figure 2 , Figure 2 is a schematic diagram of the data writing process in the related technology, such as Figure 2 As shown in the figure, the protocol access layer receives the write request from the protocol client and sends a write request to the primary data storage node (primary osd). The write request carries the object ID and the offset in the object. The primary osd performs EC encoding calculations. Taking the EC redundancy ratio of 4+2 as an example, 6 osd processes need to be written, and one of the 6 osds is the primary node. The primary osd sends the EC-encoded data to each osd standby process through an internal message; taking the EC redundancy ratio of 4+2 as an example, EC calculation obtains 6 copies of EC shard data; taking EC 4+2 as an example, there are 5 osd standby processes. The above figure simplifies the processing and only draws 2 osd standby processes; the primary osd and osd standby allocate hard disk space for the object according to the object ID and write the EC shard data to the local disk; the osd master receives the response and sends a response to the protocol access layer.

[0036] In this technology, write requests need to be forwarded to the OSD master, doubling bandwidth consumption. Message forwarding also causes memory copying, increasing CPU consumption.

[0037] For example, the bandwidth of external requests received by the protocol access layer is 1GB / S per second, the bandwidth forwarded to the OSD master is 1GB / S, and the bandwidth sent from the OSD master to the OSD backup is 1GB / S per second. In this way, the bandwidth sent within the cluster is 2GB / S, which is doubled.

[0038] refer to Figure 3 , Figure 3 is a schematic diagram of the data writing process in the related technology, such as Figure 3As shown, the protocol access layer receives the write request from the client and performs EC encoding calculation; the protocol access layer sends a write notification of numerical control separation to the OSD master; different from the first technique, the write notification does not carry data content, but carries the RDMA memory address of the data in the protocol access layer, for example, for EC 4+2 redundancy, it carries the RDMA memory address of 6 EC shards; the OSD master broadcasts the write notification of numerical control separation to the OSD standby; the OSD master and standby processes read data from the protocol access layer through RDMA unilateral reading; because the RDMA unilateral reading is hardware unloaded by the RDMA network card and does not pass through the CPU, the latency of the RDMA unilateral reading is very small and does not consume the CPU; the OSD master and OSD standby allocate hard disk space for the object according to the object ID and write the EC shard data to the local disk; the OSD master receives the response and sends a response to the protocol access layer.

[0039] In this technology, control messages need to be forwarded by the OSD master, and the delay in writing request processing is still relatively large.

[0040] refer to Figure 4 , Figure 4 1 is a flow chart of a data storage method proposed in one embodiment of the present application. Figure 4 As shown, the method comprises the following steps: S11: Receive a data writing request sent by the client.

[0041] In this embodiment, Figure 5 As shown, Figure 5 is a flow chart of a data storage method proposed in an embodiment of the present application. Figure 5 As shown, when the client needs to write data into the database, it sends a data write request to the protocol access layer, and the protocol access layer receives the data write request.

[0042] In this embodiment, the data write request carries the business data to be written and the business object corresponding to the business data, where the business object refers to the business using the business data.

[0043] S12: Perform erasure code calculation on the business data corresponding to the data write request to obtain storage data.

[0044] In this embodiment, the erasure code calculation can divide the original data into multiple data blocks, and generate check blocks through the encoding algorithm. These data blocks and check blocks are stored in different locations. For example, 4+2 means 4 data blocks and 2 check blocks. If any data block is damaged, the data in the damaged data block can be calculated through data blocks stored elsewhere, and then the damaged data can be restored.

[0045] In this embodiment, Figure 5As shown, the protocol access layer calculates the erasure code for the business data corresponding to the data write request to obtain the storage data.

[0046] S13: writing the stored data into a hard disk space pre-applied from at least one data storage node through a preset network transmission protocol, wherein the minimum storage unit of the hard disk space is a data block.

[0047] In this embodiment, the preset network transmission protocol is the NVMe-oF protocol, and the smallest storage unit in the hard disk space is a data block.

[0048] In this embodiment, Figure 5 As shown, the protocol access layer writes the storage data into the hard disk space pre-applied from at least one data storage node (data osd) through a preset network transmission protocol. The specific steps include: S13-1: Determine the space capacity of the hard disk space corresponding to the stored data.

[0049] In this embodiment, when writing storage data, the protocol access layer first determines the space capacity of the hard disk space corresponding to the storage data, and then allocates the corresponding space capacity for the storage data.

[0050] S13-2: Determine at least one data block corresponding to the stored data according to the space capacity.

[0051] In this embodiment, after determining the size of the hard disk space, multiple data blocks of corresponding sizes are determined in multiple data storage nodes, and then the storage data is stored in the multiple data blocks respectively.

[0052] S13-3: Obtain the address of the at least one data block.

[0053] In this embodiment, after determining the data blocks where the storage data can be stored, the LBA addresses of the multiple data blocks are obtained.

[0054] S13-4: Writing the stored data into the data block corresponding to the address of the at least one data block through the network transmission protocol.

[0055] In this embodiment, after determining the address of the data block to be written, the protocol access layer stores the storage data in the corresponding data block through the network transmission protocol.

[0056] S14: Recording in the metadata storage node a mapping relationship between a business object range corresponding to the stored data and an internal object range in the data storage node.

[0057] In this embodiment, the mapping relationship is used to record the correspondence between the business object range corresponding to the storage data and the internal object range. The internal object is the hard disk space in the data storage node that is pre-applied. The business object range refers to the range of data corresponding to the business object, and the internal object range refers to the range of data stored in the internal object. The metadata storage node (metadata osd) is used to store the correspondence between the internal object and the address of the data block in each data storage node.

[0058] In this embodiment, Figure 5 As shown, after writing data to the data block in the corresponding storage node, the protocol access layer sends an update mapping relationship request to the metadata storage node, and then records the correspondence between the business object range corresponding to the written storage data and the internal object range in the mapping relationship.

[0059] For example, 0~1MB of obj_service is written to 2MB~3MB of obj_internal.

[0060] In this embodiment, data is directly written into each master data storage node through the nvme-of protocol without the need for data forwarding. Based on the high-speed data transmission characteristic of the protocol, the data transmission speed is further improved.

[0061] In this embodiment, before receiving the data writing request sent by the client, the method further includes: S21: When the process corresponding to the network transmission protocol is started, an internal object pre-application request is sent to the metadata storage node.

[0062] In this embodiment, the internal object pre-application request is used to request hard disk space in the primary data storage node.

[0063] In this embodiment, Figure 5 As shown, when the process corresponding to the network transmission protocol is started, the protocol access layer sends an internal object pre-application request to the metadata storage node.

[0064] S22: When receiving the internal object pre-application request, the metadata storage node sends the internal object pre-application request to the at least one data storage node.

[0065] In this embodiment, Figure 5 As shown, upon receiving the internal object pre-application request, the metadata storage node sends the internal object pre-application request to all data storage nodes in the storage pool.

[0066] S23: Upon receiving the internal object pre-application request, the at least one data storage node determines a hard disk space corresponding to the internal object pre-application request.

[0067] In this embodiment, Figure 5 As shown, when receiving the internal object pre-application request, the data storage node determines the hard disk space corresponding to the internal object pre-application request.

[0068] S24: The at least one data storage node allocates the corresponding hard disk space in the hard disk.

[0069] In this embodiment, Figure 5 As shown, after the data storage node determines the hard disk space that needs to be allocated, it allocates the corresponding hard disk space in the hard disk. After allocating the hard disk space to the internal object, it is bound to the internal object and is no longer allocated to other internal objects.

[0070] For example, a pre-application for an internal object can apply for a larger space, such as 64MB. Assuming that the size of each business request is 1MB, a pre-application can process 64 business requests. Osd is divided into metadata osd and data osd, and data osd is no longer divided into primary and backup osd. Metadata osd and data osd use different redundancy. Generally, metadata osd uses 3 copies, and data osd uses EC, such as EC 4+2.

[0071] S25: The at least one data storage node determines the address of the data block corresponding to the hard disk space.

[0072] In this embodiment, after allocating hard disk space for the internal object, the data storage node determines the address of the data block corresponding to the hard disk space.

[0073] S26: The at least one data storage node returns a pre-application response to the metadata storage node, where the pre-application response carries the address of the data block.

[0074] In this embodiment, Figure 5 As shown, the data storage node returns a pre-application response to the metadata storage node, and the pre-application response carries the address of the data block allocated to the internal object.

[0075] S27: When receiving the pre-application response, the metadata storage node records the address of the data block carried in the pre-application response.

[0076] In this embodiment, Figure 5 As shown, when receiving the pre-application response, the metadata storage node records the address of the internal object carried in the pre-application response on each data storage node.

[0077] S28: Receive the address of the data block sent by the metadata storage node.

[0078] In this embodiment, Figure 5 As shown, the protocol access layer receives the addresses of the internal objects on each primary data storage node sent by the metadata storage node.

[0079] In this embodiment, when the process is started, hard disk space is applied for in advance in each main data storage node, and there is no need to apply for space again when a data write request is sent, thereby improving the data write speed.

[0080] In another embodiment of the present application, the method further includes: S31: receiving a data reading request sent by the client.

[0081] In this embodiment, Figure 6 As shown, Figure 6 This is a data reading process diagram proposed in an embodiment of the present application. When the client needs to read data in the main data storage node, it sends a data reading request to the protocol access layer. The data reading request carries the data range of the data to be read.

[0082] S32: When receiving the data read request, sending a mapping relationship query request to the metadata storage node.

[0083] In this embodiment, Figure 6 As shown, upon receiving a data read request, the protocol access layer sends a mapping relationship query request to the metadata storage node, where the request is used to query the mapping relationship stored in the metadata storage node.

[0084] S33: Upon receiving the mapping relationship query request, the metadata storage node returns a query mapping relationship response, wherein the query mapping relationship response carries a mapping relationship between the business object range and the internal object range and the address of the data block of the internal object in the at least one data storage node.

[0085] In this embodiment, Figure 6 As shown, when the metadata storage node receives a mapping relationship query request, it returns a query mapping relationship response to the protocol access layer. The query mapping relationship response carries the mapping relationship between the business object range and the internal object range corresponding to the data stored in the data storage node and the address of the data block of the internal object in at least one data storage node.

[0086] S34: Determine the address of the data block corresponding to the data read request according to the query mapping relationship response.

[0087] In this embodiment, Figure 6 As shown, the protocol access layer determines the internal object corresponding to the data read request according to the query mapping relationship response, and then determines the address of the data block corresponding to the internal object.

[0088] S35: Sending a data acquisition request to the at least one data storage node according to the address of the data block.

[0089] In this embodiment, Figure 6 As shown, after obtaining the address of the data block in the range of the internal object to be read by the client, the protocol access layer sends a data acquisition request, that is, an nvme-of read request, to the corresponding data storage node.

[0090] S36: Send the acquired data to the client.

[0091] In this embodiment, Figure 6 As shown, after receiving the response from the data storage node, the protocol access layer sends the read data to the client in the response.

[0092] In this embodiment, when the client needs to read data, it directly reads data from each data storage node through the protocol access layer, and quickly reads data in the data storage node based on the nvme-of protocol, thereby improving the speed of data reading.

[0093] In another embodiment of the present application, the method further includes: S41: The management node receives heartbeat information sent by at least one of the data storage nodes.

[0094] In this embodiment, Figure 7 As shown, Figure 7 is a schematic diagram of a fault repair process proposed in an embodiment of the present application, such as Figure 7 As shown, the management node mom receives the heartbeat information sent by the data storage node.

[0095] S42: When the management node does not receive the heartbeat information sent by the data storage node, the management node determines that the data storage node is a faulty node.

[0096] In this embodiment, Figure 7 As shown, when the management node does not receive the heartbeat information sent by the data storage node, it determines that the data storage node is a faulty node. If the management node does not receive the heartbeat information sent by the data storage node within a preset time period, it is deemed that the fault time exceeds the threshold. At this time, the management node determines that the data storage node is a faulty node.

[0097] S43: The management node deletes the faulty node.

[0098] In this embodiment, Figure 7 As shown, the management node deletes the faulty node and the data storage node 1.

[0099] S44: The management node adds the newly added data storage node to a placement group, where the placement group is used to place the at least one data storage node.

[0100] In this embodiment, Figure 7 As shown, the management node counts the new data storage node (data storage node 3) into the placement group. The placement group is used to store multiple data nodes. One placement group includes a metadata storage node and multiple data storage nodes.

[0101] In this embodiment, the method further includes: S45: The management node sends the updated data storage node view to the metadata storage node.

[0102] In this embodiment, the data storage node view (osd view) is a view that includes information of each storage node.

[0103] In this embodiment, Figure 7 As shown, after the management node adds the new data storage node to the storage pool, that is, replaces the failed node with the new data storage node, the management node sends the updated data storage node view to the metadata storage node.

[0104] S46: The metadata storage node reconstructs data of internal objects on the failed node according to the data storage node view.

[0105] In this embodiment, Figure 7 As shown, the metadata storage node reconstructs the data of the internal objects on the failed node according to the updated data storage node view. The specific steps include: S46-1: The metadata storage node sends a hard disk space application message to the newly added data storage node, where the hard disk space application message is used to apply for corresponding hard disk space for each of the internal objects.

[0106] In this embodiment, Figure 7 As shown, the metadata storage node sends a hard disk space application message to the newly added data storage node (osd3), which is used to apply for the corresponding hard disk space for the internal objects originally stored in the failed node (osd1).

[0107] S46-2: The metadata storage node deletes the address of the data block corresponding to the faulty node.

[0108] In this embodiment, Figure 7As shown, the metadata storage node deletes the storage address of the data block corresponding to the failed node.

[0109] S46-3: The metadata storage node records the address of the data block corresponding to the newly added data storage node in the mapping relationship.

[0110] In this embodiment, the metadata storage node records the address of the data block corresponding to the newly added data storage node in the mapping relationship.

[0111] S46-4: The metadata storage node reads the backup data of the failed node from the data storage nodes except the newly added data storage node.

[0112] In this embodiment, Figure 7 As shown, the metadata storage node reads the backup data from the data storage nodes other than the newly added data storage nodes, that is, uses the nvme-of protocol to read data from the normal data osd.

[0113] For example, taking EC 4+2 as an example, the data is divided into 6 blocks. If osd1 fails, the data is read from the 5 data osds other than osd1.

[0114] S46-5: The metadata storage node performs erasure code calculation on the backup data to obtain storage data corresponding to the faulty node.

[0115] In this embodiment, Figure 7 The metadata storage node performs erasure code calculation on the backup data to obtain storage data corresponding to the failed node.

[0116] S46-6: The metadata node stores the storage data corresponding to the failed node in the newly added data storage node.

[0117] In this embodiment, Figure 7 As shown, the metadata node stores the storage data corresponding to the failed node in the newly added data storage node, thus completing the data reconstruction of the failed node.

[0118] In another embodiment of the present application, the protocol access layer monitors the usage of internal objects in real time. When all the applied internal objects are used up or the stored data exceeds a preset threshold, the protocol access layer sends an internal object pre-application request to the metadata node again to expand the internal objects in a timely manner to ensure that the received data can be stored in a timely manner. The management node also monitors the hard disk space of each data storage node in real time. When the hard disk space of each data storage node exceeds a certain load threshold, a new data storage node is added to the placement group, and the corresponding mapping relationship is updated in the metadata node to ensure that sufficient storage resources are provided to users, ensure the normal operation of the business, and improve the operating efficiency of the system.

[0119] Based on the same inventive concept, an embodiment of the present application provides a data storage device. Figure 8 , Figure 8 FIG. 8 is a schematic diagram of a data storage device 800 according to an embodiment of the present application. Figure 8 As shown, the device comprises: The data writing request receiving module 801 is used to receive the data writing request sent by the client; The storage data acquisition module 802 is used to calculate the erasure code for the business data corresponding to the data write request to obtain storage data; The data writing module 803 is used to write the stored data into a hard disk space pre-applied from at least one data storage node through a preset network transmission protocol, wherein the minimum storage unit of the hard disk space is a data block; The mapping relationship updating module 804 is used to record in the metadata storage node the mapping relationship between the business object range corresponding to the stored data and the internal object range in the data storage node.

[0120] Optionally, the device further comprises: A pre-application sending module, used for sending an internal object pre-application request to the metadata storage node when the process corresponding to the network transmission protocol is started; a pre-application forwarding module, configured for the metadata storage node to send the internal object pre-application request to the at least one data storage node when the metadata storage node receives the internal object pre-application request; a hard disk space determination module, configured for the at least one data storage node to determine the hard disk space corresponding to the internal object pre-application request when receiving the internal object pre-application request; A hard disk space allocation module, used for allocating the corresponding hard disk space in the hard disk to the at least one data storage node; An address determination module, used for the at least one data storage node to determine the address of the data block corresponding to the hard disk space; A pre-application response returning module, configured for the at least one data storage node to return a pre-application response to the metadata storage node, wherein the pre-application response carries the address of the data block; An address recording module, configured for the metadata storage node to record the address of the data block carried in the pre-application response when the metadata storage node receives the pre-application response; The data address receiving module is used to receive the address of the data block sent by the metadata storage node.

[0121] Optionally, the data writing module includes: A hard disk space determination submodule, used to determine the space capacity of the hard disk space corresponding to the stored data; A data block determination submodule, configured to determine at least one data block corresponding to the stored data according to the space capacity; A data block address acquisition submodule, used to acquire the address of the at least one data block; The data transmission submodule is used to write the stored data into the data block corresponding to the address of the at least one data block through the network transmission protocol.

[0122] Optionally, the device further comprises: A read request receiving module, used for receiving a data read request sent by the client; A mapping relationship query request sending module, configured to send a mapping relationship query request to the metadata storage node upon receiving the data read request; A mapping relationship query response module is used for the metadata storage node to return a mapping relationship query response when receiving the mapping relationship query request, wherein the mapping relationship query response carries a mapping relationship between a business object range and an internal object range and an address of a data block of the internal object in the at least one data storage node; A data block address determination module, configured to determine the address of the data block corresponding to the data read request according to the query mapping relationship response; A data acquisition request sending module, used to send a data acquisition request to the at least one data storage node according to the address of the data block; The data forwarding module is used to send the acquired data to the client.

[0123] Optionally, the device further comprises: A heartbeat information receiving module, used for the management node to receive the heartbeat information sent by at least one of the data storage nodes; A fault node determination module, configured for the management node to determine that the data storage node is a fault node when the management node does not receive the heartbeat information sent by the data storage node; A faulty node deletion module, used for the management node to delete the faulty node; A newly added node module is used for the management node to add a newly added data storage node to a placement group, and the placement group is used to place the at least one data storage node.

[0124] Optionally, the device further comprises: A node view updating module, used for the management node to send the updated data storage node view to the metadata storage node; The data reconstruction module is used for the metadata storage node to reconstruct the data of the internal objects on the failed node according to the data storage node view.

[0125] Optionally, the data reconstruction module includes: A hard disk space application submodule, used for the metadata storage node to send a hard disk space application message to the newly added data storage node, wherein the hard disk space application message is used to apply for a corresponding hard disk space for each of the internal objects; A data block address deletion submodule, used for the metadata storage node to delete the address of the data block corresponding to the faulty node; A newly added address recording submodule is used for the metadata storage node to record the address of the data block corresponding to the newly added data storage node in the mapping relationship; A backup data reading submodule, used for the metadata storage node to read the backup data of the failed node from the data storage nodes other than the newly added data storage node; A storage data recovery submodule, configured for the metadata storage node to perform erasure code calculation on the backup data to obtain storage data corresponding to the faulty node; A newly added data node data storage submodule is used for the metadata node to store the storage data corresponding to the failed node in the newly added data storage node.

[0126] Based on the same inventive concept, another embodiment of the present application provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the data storage method described in any of the above embodiments of the present application.

[0127] Based on the same inventive concept, another embodiment of the present application provides an electronic device, Fig. 9It is a schematic diagram of an electronic device 900 proposed in one embodiment of the present application, including a memory 902, a processor 901 and a computer program stored in the memory and executable on the processor, and the processor, when executed, implements the steps in the data storage method described in any of the above embodiments of the present application.

[0128] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0129] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0130] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0131] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0132] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable terminal device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0134] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0135] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0136] The data storage method, device, equipment and storage medium provided by the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data storage method, characterized in that: The method comprises: Receive data writing request sent by the client; Performing erasure code calculation on the business data corresponding to the data write request to obtain storage data; Writing the stored data into a hard disk space pre-applied from at least one data storage node through a preset network transmission protocol, wherein the minimum storage unit of the hard disk space is a data block; A mapping relationship between a business object range corresponding to the stored data and an internal object range in the data storage node is recorded in the metadata storage node.

2. The data storage method according to claim 1, characterized in that: Before receiving the data writing request sent by the client, the method further includes: When the process corresponding to the network transmission protocol is started, sending an internal object pre-application request to the metadata storage node; The metadata storage node, upon receiving the internal object pre-application request, sends the internal object pre-application request to the at least one data storage node; Upon receiving the internal object pre-application request, the at least one data storage node determines a hard disk space corresponding to the internal object pre-application request; The at least one data storage node allocates the corresponding hard disk space in the hard disk; The at least one data storage node determines the address of the data block corresponding to the hard disk space; The at least one data storage node returns a pre-application response to the metadata storage node, wherein the pre-application response carries the address of the data block; When receiving the pre-application response, the metadata storage node records the address of the data block carried in the pre-application response; Receive the address of the data block sent by the metadata storage node.

3. The data storage method according to claim 1, characterized in that: The step of writing the stored data into a hard disk space pre-applied from at least one data storage node according to a preset network transmission protocol includes: Determine the space capacity of the hard disk space corresponding to the stored data; Determine, according to the space capacity, at least one data block corresponding to the stored data; Obtaining an address of the at least one data block; The stored data is written into the data block corresponding to the address of the at least one data block through the network transmission protocol.

4. The data storage method according to claim 1, characterized in that: The method further comprises: Receiving a data reading request sent by the client; When receiving the data read request, sending a mapping relationship query request to the metadata storage node; Upon receiving the mapping relationship query request, the metadata storage node returns a mapping relationship query response, wherein the mapping relationship query response carries a mapping relationship between a business object range and an internal object range and an address of a data block of the internal object in the at least one data storage node; Determining the address of the data block corresponding to the data read request according to the query mapping relationship response; Sending a data acquisition request to the at least one data storage node according to the address of the data block; The acquired data is sent to the client.

5. The data storage method according to claim 1, characterized in that: The method further comprises: The management node receives heartbeat information sent by at least one of the data storage nodes; The management node determines that the data storage node is a faulty node when the management node does not receive the heartbeat information sent by the data storage node; The management node deletes the faulty node; The management node adds the newly added data storage node to a placement group, where the placement group is used to place the at least one data storage node.

6. The data storage method according to claim 5, characterized in that: The method further comprises: The management node sends the updated data storage node view to the metadata storage node; The metadata storage node reconstructs data of internal objects on the failed node according to the data storage node view.

7. The data storage method according to claim 6, characterized in that: The metadata storage node reconstructs data of the internal object on the failed node according to the data storage node view, including: The metadata storage node sends a hard disk space application message to the newly added data storage node, wherein the hard disk space application message is used to apply for a corresponding hard disk space for each of the internal objects; The metadata storage node deletes the address of the data block corresponding to the failed node; The metadata storage node records the address of the data block corresponding to the newly added data storage node in the mapping relationship; The metadata storage node reads the backup data of the failed node from the data storage nodes other than the newly added data storage node; The metadata storage node performs erasure code calculation on the backup data to obtain storage data corresponding to the faulty node; The metadata node stores the storage data corresponding to the failed node in the newly added data storage node.

8. A data storage device, characterized in that: The device comprises: A data writing request receiving module, used for receiving a data writing request sent by a client; A storage data acquisition module, used to perform erasure code calculation on the business data corresponding to the data write request to obtain storage data; A data writing module, used to write the stored data into a hard disk space pre-applied from at least one data storage node through a preset network transmission protocol, wherein the minimum storage unit of the hard disk space is a data block; The mapping relationship updating module is used to record in the metadata storage node the mapping relationship between the business object range corresponding to the stored data and the internal object range in the data storage node.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images