Data storage method, subsystem, distributed storage system and storage medium

Abstract

The data storage method, subsystem, distributed storage system, and storage medium provided by the present invention include: synchronizing the write operation log received by the master logical node to all slave logical nodes; when it is determined that the master logical node has received a message indicating a log synchronization failure, determining the abnormal logical node corresponding to the message; wherein the log synchronization failure indicates that data on the target physical storage node corresponding to the abnormal logical node is missing; generating missing data based on a preset erasure code and the data logs of the remaining slave logical nodes except the abnormal logical node and the master logical node; the data log is used to record data written to the physical storage node corresponding to the logical node; and recovering data on the target physical storage node based on the missing data. The data loss detection of the present invention is dependent on data writing, and does not require service nodes to schedule and recover data, thus avoiding the problem of recovering data loss due to server downtime and effectively reducing the complexity of the recovery process.

Description

Technical Field

[0001] The present invention relates to the field of data storage technology, and in particular to a data storage method, subsystem, distributed storage system and storage medium. Background Art

[0002] To avoid errors during data writing, RS(N,M) erasure coding can be used to generate M copies of check data from N copies of original data. When any M copies of the N+M copies of data are lost, they can be regenerated from the remaining N copies of data, thereby achieving the effect of data recovery.

[0003] In existing distributed storage systems, after a data write failure, the server side needs to report to the metadata server, which schedules data recovery. The recovery operation is actually performed on the data storage node side. It can be seen that in the existing technology, the recovery data can only be perceived by the system after the server side reports it to the metadata server. When the server crashes, the recovery data is easily lost. At this time, the data inconsistency on the data storage node side cannot be immediately perceived. If an anomaly occurs again during this period, the risk of system data inconsistency or complete damage will be significantly increased. Summary of the Invention

[0004] One of the purposes of the present invention is to provide a data storage method, subsystem, distributed storage system and storage medium to reduce the risk of data inconsistency or complete damage and the complexity of data recovery.

[0005] In a first aspect, the present invention provides a data storage method, which is applied to a data storage subsystem in a distributed storage system, wherein the data storage subsystem includes multiple logical nodes, each logical node corresponding to a physical storage node; the method includes: synchronizing the write operation log received by the master logical node to all slave logical nodes; wherein the master logical node is one of all the logical nodes; the slave logical node is a logical node other than the master logical node; when it is determined that the master logical node receives a message of log synchronization failure, determining the abnormal logical node corresponding to the message; wherein the log synchronization failure indicates that data on the target physical storage node corresponding to the abnormal logical node is missing; based on a preset erasure code and the data logs of the remaining slave logical nodes except the abnormal logical node and the master logical node, generating missing data; the data log is used to record data written to the physical storage node corresponding to the logical node; and based on the missing data, performing data recovery on the target physical storage node.

[0006] In a second aspect, the present invention provides a data storage subsystem, which includes multiple logical nodes, each logical node corresponding to a physical storage node, and includes: a synchronization module, used to synchronize the write operation log received by the master logical node to all slave logical nodes; wherein the master logical node is one of all the logical nodes; the slave logical node is a logical node other than the master logical node; a determination module, used to determine the abnormal logical node corresponding to the message when it is determined that the master logical node receives a message of log synchronization failure; wherein the log synchronization failure indicates that data on the target physical storage node corresponding to the abnormal logical node is missing; a generation module, used to generate missing data based on a preset erasure code and the data logs of the remaining slave logical nodes except the abnormal logical node and the master logical node; the data log is used to record data written to the physical storage node corresponding to the logical node; a storage module, used to recover data on the target physical storage node based on the missing data.

[0007] In a third aspect, the present invention provides a distributed storage system, which includes a data storage subsystem, the data storage subsystem is composed of multiple logical nodes, each logical node corresponds to a physical storage node, and the data storage subsystem is used to execute the data storage method described in the first aspect.

[0008] In a fourth aspect, the present invention provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the data storage method as described in the first aspect.

[0009] The data storage method, subsystem, distributed storage system and storage medium provided by the present invention include: after receiving a write operation log, the master logical node synchronizes the write operation log to the slave logical node; upon receiving a message indicating that the log synchronization fails from the slave logical node, indicating that data on the physical storage node corresponding to the abnormal logical node that fed back the message is missing, the master logical node can generate missing data based on a preset erasure code and the data logs of the remaining slave logical nodes other than the abnormal logical node and the master logical node; and based on the missing data, data recovery is performed on the physical storage node corresponding to the abnormal logical node; the data storage subsystem provided by the present invention can monitor whether there is a physical storage node with missing data; if so, the missing data can be generated based on the recorded data on other slave logical nodes, and then the physical storage node with missing data is recovered; the entire process makes anomaly detection dependent on data writing, and does not require service nodes to schedule and recover, thus avoiding the problem of recovery data loss due to server downtime, and effectively reducing the complexity of the recovery process. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0011] Figure 1 This is an example diagram of an existing data storage method;

[0012] Figure 2 A system structure diagram of a distributed storage system provided by an embodiment of the present invention;

[0013] Figure 3 A schematic diagram of the structure of a data storage subsystem provided by an embodiment of the present invention;

[0014] Figure 4 A schematic flow chart of a data storage method provided in an embodiment of the present invention;

[0015] Figure 5 A schematic diagram of a data storage scenario provided in an embodiment of the present invention;

[0016] Figure 6 A schematic flow chart of another data storage method provided by an embodiment of the present invention;

[0017] Figure 7 A schematic diagram of another scenario of data storage provided in an embodiment of the present invention;

[0018] Figure 8 A functional module diagram of a data storage subsystem provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0020] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0021] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0022] In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the accompanying drawings, or is the orientation or position relationship in which the product of the invention is usually placed when in use. It is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be understood as a limitation on the present invention.

[0023] In addition, the terms "first", "second", etc., if used, are merely used to distinguish and describe, and should not be understood as indicating or implying relative importance.

[0024] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0025] The following first explains the relevant terms involved in the embodiments of this application.

[0026] RS(N,M) erasure code: Generates M parity data from N original data. If any M of the N+M data are lost, they can be regenerated from the remaining N data.

[0027] The RAFT consensus algorithm is a distributed strong consistency solution that replicates logs from a master node to slave nodes. This algorithm treats the system as a state machine and operations on the state machine as log entries. Consistency is ensured by achieving the same target state when the master and slave systems execute the same operations from the same initial state. The algorithm maintains a log sequence internally and ensures that the logs are correctly and orderly replicated to each replica through master node selection and synchronous replication.

[0028] See Figure 1 , Figure 1 This is an example diagram of an existing data storage method, such as Figure 1 As shown, in the existing data storage method, the data node needs to store the stripe data and version number at the same time. A single write on the server side will generate two inputs and two outputs. After the stripe is divided, the data written by the client to the server side needs to be aligned to the stripe size. Data alignment requires two additional read requests, one for reading the version number and the other for reading the data. This will significantly increase the write operation latency when the cache misses.

[0029] For example, the data is divided into 4KB stripes, each 4KB stripe corresponds to an 8-byte version number, and an additional 128KB version number needs to be stored for every 64MB of data. The service node needs to update the version number and data content at the same time when writing. At this time, two inputs and two outputs will be generated. These two inputs and two outputs are written to different locations, which increases the seek overhead. After striping, the data written to the data node needs to be aligned to the stripe size. That is, the data offset and length written to the data node need to be aligned to 4KB, and the data written to the SDK needs to be aligned to 4KB*N.

[0030] When the data written by the client to the server cannot be aligned to 4KB*N, it is necessary to first read 4KB*(N+M) data from the DN side to overwrite it and then write it to the data node again. In extreme cases, the data written to the server side needs to be aligned at both the beginning and the end. At this time, a write on the server side will generate 2*(N+M)*2 reads and (N+M)*2 writes. If the cache is not hit, the write operation latency will be significantly increased. In the sequential write scenario, hitting the cache can eliminate half of the read operation latency. At this time, the SDK side write latency is MAX(N +M)(read version number)+MAX(N+M)(read data)+MAX(N+M)(write data)+MAX(N+M)(commit data). The existing system uses NVM for write acceleration. The latency of MAX(N+M)(write data)+MAX(N+M)(commit data) is mainly affected by the network and is generally much shorter than MAX(N+M)(read version number)+MAX(N+M)(read data). The written data spans two stripe groups. The first and last stripes must be read to the SDK before the erasure data can be calculated.

[0031] On the other hand, after detecting a data storage failure, the service node needs to report a data recovery request to the metadata server, which will schedule and generate the missing data. The data recovery operation is actually performed on the data node side. At this time, the data node side will receive the data that needs to be sent and the data to be written from the service node at the same time, which increases the complexity of the system. Moreover, since the data recovery request is reported by the service node, once the service node goes down, the data recovery request may be lost, and inconsistent data needs to be discovered through background scanning. The background scanning cycle is long, and any abnormality during this period will significantly increase the risk of inconsistent or complete damage to the system data.

[0032] In order to solve the above problems, an embodiment of the present invention provides an improved distributed storage system, see Figure 2 , Figure 2A system structure diagram of a distributed storage system provided in an embodiment of the present invention includes a client 201, a service node 202, a data node 203 (DN), and a data storage subsystem 205 based on the RAFT consensus algorithm. The client 201, the service node 202, and the data node 203 are communicatively connected. The above-mentioned distributed storage system may also include a metadata server (MS) (omitted here).

[0033] The service node 202 can actually be a server or an SDK. The client 201 and the service node 202 can exchange data through, but not limited to, the iscsi protocol. An RPC connection can be established between the service node 202 and the data node 203, and the RPC communication timeout is set to 5 seconds.

[0034] The data storage subsystem 205 is used to take over the data writing business and data recovery, and ensure the consistency of the data written to the data node 103 side through a strong consistency algorithm and data recovery.

[0035] The data storage subsystem 205 is bound to the block storage erasure object (OBJ). When the OBJ needs to write or recover data, the data storage subsystem 205 can be created in advance to ensure the consistency of the OBJ data. When the idle time of the data storage subsystem 205 reaches the preset time, the destruction of the data storage subsystem 205 can be initiated.

[0036] The data storage subsystem 205 provided by an embodiment of the present invention can remove the stripes and version numbers on the data node side during the process of writing data. At this time, the data node side can receive writes of any offset and granularity, postpone the data alignment operation to the NVM disk, and eliminate the overhead of writing the version number on the data node side.

[0037] For example, assume that block storage provides external services via the ISCSI protocol with a minimum write granularity of 512 bytes. Selecting an appropriate N value (2, 4, 8, 16, etc.) ensures that 512 % N == 0 and sets the stripe size to 512 / N. This allows data written by the server to be evenly distributed across N + M data nodes. The server can then directly write data to the data nodes without having to read data from the data nodes for stripe group alignment. The write latency is MAX(N + M) (write data) + MAX(N + M) (synchronize operation logs). Taking the 8 + 2 erasure ratio as an example, setting the stripe size to 64 bytes in the old system eliminates the stripe alignment read operation, but this significantly increases the number of version numbers. In this case, an 8-byte version number is required for every 64 bytes of data, resulting in a space utilization of 64 / (64 + 8) = 88.9%. This solution eliminates the stripe version number, allowing for arbitrary stripe size settings without occupying any version number space.

[0038] See Figure 3 , Figure 3 A structural diagram of the data storage subsystem provided in an embodiment of the present invention, wherein the data storage subsystem 205 is composed of an algorithm core, logical nodes, operation logs, and data logs, and uses LUNID+OBJID as the identifier of the data storage subsystem 205. For example, assuming that the LUNID is 1 and the OBJID is 0, the identifier of the data storage subsystem can be expressed as r-1-0.

[0039] The core of the algorithm is the RAFT algorithm, which consists of two parts: leader election and log replication. When a slave logical node does not receive a keepalive from the master logical node, it initiates the leader election process. In the original algorithm, the elected node becomes the master logical node after receiving more than half of the votes. However, in the embodiment of the present invention, a one-to-one mapping with the data log is required. Therefore, it can only become the master logical node when it receives greater than or equal to N votes, where N is the total number of original data copies after the data to be written is divided.

[0040] A logical node is a virtual node created by a data node based on a request from a metadata server or service node. It is located on the physical node where the corresponding data block resides, according to its sequence number. Each logical node is identified based on the data storage subsystem's identifier and the corresponding data node's identifier. For example, a logical node on data node DN-1 can be represented as r-1-0-1, where r-1-0 is the data storage subsystem's identifier and 1 is the identifier of data node DN-1. Each logical node contains information about all logical nodes in data storage subsystem 205, primarily including the status of each logical node and the identifier of the physical node it resides on. Each logical node consists of an operation log and a data log, which are stored in the transaction log of data node 203.

[0041] Operation logs are divided into write operation logs and full data recovery logs. Write operation logs contain the offset within the OBJ (that is, the starting position of data writing), data length, and transaction ID; full data recovery logs record the start and end status.

[0042] The data log records the data written and distributed to the current data node. The service node writes it to the data node's transaction log. The write operation log controls the data storage. When the write operation log is written to the data node, it is checked once to identify data log loss and repair it. Logs with log IDs less than or equal to the RAFT commit ID are applied to the storage.

[0043] It can be understood that the data storage subsystem provided by the embodiment of the present invention is a virtual subsystem in a distributed storage system, which is the main difference from the existing distributed storage system. The data storage subsystem is composed of the RAFT algorithm and various logical nodes. Each logical node is used to maintain operation logs and data logs. The data logs can be used to maintain the data to be written, avoiding the impact of data failure caused by data node failure. When there is a data write request, a request can be made to create a data storage subsystem, and the data storage subsystem will complete data writing, data recovery and other functions. In this way, it is possible to avoid reporting data write or data recovery requests through the server, and avoid the adverse effects of server downtime. At the same time, when there is no data write request, the data storage subsystem can be deleted to avoid occupying additional resources.

[0044] Based on the data storage subsystem 205, the embodiment of the present invention provides a data storage method, see Figure 4 , Figure 4 A schematic flow chart of a data storage method provided in an embodiment of the present invention, which may include:

[0045] S401: Synchronize the write operation log received by the master logical node to all slave logical nodes.

[0046] The master logical node is one of all logical nodes; the slave logical node is a logical node other than the master logical node.

[0047] S402, when it is determined that the master logical node receives a message about log synchronization failure, determining the abnormal logical node corresponding to the message;

[0048] Among them, log synchronization failure indicates that data on the target physical storage node corresponding to the abnormal logical node is missing;

[0049] S403: Generate missing data based on a preset erasure code and data logs of the remaining slave logical nodes except the abnormal logical node and the master logical node.

[0050] The data log is used to record the data written to the physical storage node corresponding to the logical node;

[0051] S404: Perform data recovery on the target physical storage node based on the missing data.

[0052] According to the data storage method provided by an embodiment of the present invention, after receiving the write operation log, the master logical node synchronizes the write operation log to the slave logical node. After receiving the message of log synchronization failure fed back by the slave logical node, it indicates that the data on the physical storage node corresponding to the abnormal logical node that fed back the message is missing. At this time, missing data can be generated based on the preset erasure code and the data logs of the remaining slave logical nodes except the abnormal logical node and the master logical node, and based on the missing data, data recovery is performed on the physical storage node corresponding to the abnormal logical node. The data storage subsystem provided by the present invention can monitor whether there is a physical storage node with missing data. If so, missing data can be generated based on the recorded data on other slave logical nodes, and then the physical storage node with missing data is recovered. The entire process makes anomaly detection dependent on data writing, and does not require the service node to schedule and recover, avoiding the problem of recovery data loss due to server downtime, and effectively reducing the complexity of the recovery process.

[0053] The following is combined with Figure 4 To the attached Figure 7 , the above steps S401 to S404 are introduced in detail.

[0054] In step S401, the write operation log received by the master logical node is synchronized to all slave logical nodes.

[0055] In an embodiment of the present invention, the above-mentioned write operation log is an operation log generated after the service node distributes the data to be written to multiple physical storage nodes. That is, in an optional implementation manner, the operation log is generated as follows:

[0056] a1. Obtain the original data and verification data based on the data to be written received by the service node.

[0057] In an embodiment of the present application, the data to be written is divided according to a preset erasure code. For example, assuming that the preset erasure code is R (N, M), where N is the total number of original data, and M is the total number of verification data, erasure calculation is performed on the data slices, and the original data is divided into N original data according to size, and M verification data is generated through erasure coding. The offset and data length corresponding to each piece of data are 1 / N of the data length of the data to be written.

[0058] a2. Distribute the transaction ID, original data, and verification data generated by the service node to multiple physical storage nodes, so that each physical storage node writes the received transaction ID, original data, or verification data into the transaction log.

[0059] The physical storage node in the embodiment of the present application is Figure 2The data nodes shown, that is, before generating the operation log, the service node first sends the original data and verification data together with the transaction ID to the physical storage node. After each physical storage node receives the original data or verification data, it writes the transaction ID as well as the original data and verification data into the transaction log maintained by itself.

[0060] In an embodiment of the present invention, the transaction ID can be based on the maximum value of the existing transaction log in the data storage subsystem + 1 as the initial value, and is increased by 1 each time the service node uses it, and LUNID-OBJID is added as a globally unique value.

[0061] In an embodiment of the present invention, the service node can determine in advance the N+M physical storage nodes that receive the original data, verification data, and transaction ID based on N+M copies of data. That is, in an optional implementation, the service node can first send query information of the data storage subsystem to the metadata server. After receiving the query information, the metadata server queries the created OBJ information, that is, determines the physical storage nodes where the N+M data blocks (BLKs) contained in the OBJ are located, as the N+M physical storage nodes.

[0062] In the above process of determining N+M physical storage nodes, it can also be determined whether a logical node is created in each physical storage node. If not, a creation request can be sent to the physical storage node to enable the physical storage node to create the logical node. If so, no creation is performed.

[0063] a3, determining whether the number of data sending success messages received by the service node is greater than or equal to the total number of original data copies.

[0064] In an embodiment of the present invention, after receiving the original data or verification data and the transaction ID, the physical storage node may return a data transmission success message or a data transmission failure message to the service node. The service node may determine based on the returned result that if the number of data transmission success messages is greater than or equal to N, it will generate a write operation log and send it to the master logical node; otherwise, it will send a rollback message to the physical storage node.

[0065] a4. If yes, generate a write operation log through the service node.

[0066] The write operation log in the embodiment of the present invention can, on the one hand, indicate whether the data to be written exists from the logical node, and on the other hand, can also apply the operation log to write the data to be written into the data storage of the physical storage node.

[0067] In an embodiment of the present invention, the write operation log includes a transaction ID. After receiving the write operation log, the master logical node may write the write operation log into the transaction log of the physical storage node corresponding to the master logical node, and then initiate write operation log synchronization.

[0068] In an embodiment of the present invention, by synchronizing the write operation log, the original 2PC write mode is adjusted to write data + synchronize the operation log, and the submitted data version number is replaced by the synchronized operation log to avoid increasing network pressure due to synchronized data content.

[0069] It can also be seen that in the embodiment of the present invention, before log synchronization is performed, the service node first distributes the data to the physical storage node, and then data loss anomaly detection can be performed during the log synchronization process. That is to say, the implementation of the present invention can make anomaly detection and data writing dependent, without the need for the service node to detect and report the anomaly, thereby reducing the data loss loss caused by service node downtime.

[0070] To understand the above, please refer to Figure 5 , Figure 5 A schematic diagram of a data storage scenario provided in an embodiment of the present invention.

[0071] like Figure 5 As shown, assuming that the storage space is divided into 64M BLKs, using a 4+2 erasure ratio, and each OJB is 256MB in size, the client can first query the OBJ information from the MS through the SDK based on LUN=1, and determine that the OBJ identifier is 0. The data to be written needs to be stored on the six data nodes DN-0 to DN-5. The created data storage subsystem is r-1-0, including six logical nodes r-1-0-0 to r-1-0-5, and each logical node corresponds to a data node.

[0072] After the client sends the data to be written to the SDK, the SDK divides the data into four copies and generates two copies of verification data. The data to be written includes a length of 512 bytes and a starting write position offset of 512. Therefore, each copy has a length of 128 bytes and a starting write position of 128. The SDK first generates a transaction ID (id=1) and distributes the transaction ID and the six copies of data to DN-0 through DN-5. After receiving the data and transaction ID, each data node writes it to the transaction log. After receiving four or more successful responses, the SDK generates a write operation log (log=2) based on the transaction ID, the starting write position (offset=512), and the data length (length=512). The log is then sent to the master logical node from r-1-0-0 to r-1-0-5, which then synchronizes the log to the slave logical nodes.

[0073] It can also be understood that for each data node, after the logical node applies the write operation log, the obtained data can be written into the data storage. For example, taking data node DN-0 as an example, the obtained data is written into the data block identified as 1-0-0, and the data starting write position offset=128.

[0074] In step S402, when it is determined that the master logical node receives a message indicating a log synchronization failure, an abnormal logical node corresponding to the message is determined.

[0075] In an embodiment of the present invention, after receiving a write operation log from a logical node, the corresponding physical storage node transaction log can be queried based on the transaction ID in the write operation log. If the transaction log is successfully queried, a synchronization success message is returned; if not, a synchronization failure message is returned. Therefore, in an optional embodiment, after receiving a log synchronization request from a logical node, the following steps can be performed:

[0076] b1, determine whether the physical storage node corresponding to the slave logical node has the transaction ID in the write operation log;

[0077] b2. If the transaction ID does not exist, the slave logical node will send back a message indicating that the log synchronization has failed to complete.

[0078] b3. If the transaction ID exists, the slave logical node sends a message to the master logical node indicating that the log synchronization is successful.

[0079] For easier understanding, please refer to Figure 5Taking DN-0 as an example, after receiving the transaction ID (i.e., id=1) and the original data or verification data, DN-0 writes the transaction ID (i.e., id=1) into the transaction log. The logical node r-1-0-0 corresponding to DN-1 receives the synchronized write operation log. It can query whether the transaction ID (i.e., id=1) in the write operation log exists in DN-0. If so, it means that the original data or verification data to be written already exists in DN-0. If not, it means that the data of DN-0 is missing.

[0080] For the slave logical node whose write operation log is successfully synchronized, the write operation log can be applied according to the transaction ID in the write operation log, so that the physical storage node where the slave logical node is located writes the original data or verification data in the transaction log corresponding to the transaction ID into the data storage.

[0081] For the slave logical node that fails to synchronize the write operation log, after receiving the synchronization failure message, the master logical node determines the logical node corresponding to the message as an abnormal logical node, and then performs steps S403 and S404 to achieve data recovery.

[0082] In step S403, missing data is generated based on the preset erasure code and the data logs of the remaining slave logical nodes except the abnormal logical node and the master logical node; the data logs are used to record the data written to the physical storage nodes corresponding to the logical nodes.

[0083] In an embodiment of the present invention, the data log corresponding to each logical node is used to record the original data or verification data to be written to the physical storage node. When data is missing in the physical storage node where the abnormal logical node is located, the original data or verification data in the data logs of other logical nodes can be obtained, and the missing data can be generated through a preset erasure code.

[0084] In step S404, data recovery is performed on the target physical storage node based on the missing data.

[0085] In an embodiment of the present invention, the master logical node generates missing data, sends the missing data to the target physical storage node, causes the target physical storage node to write the missing data to the transaction log, and then resynchronizes the write operation log to the abnormal logical node.

[0086] In an optional implementation manner, in order to determine whether the data to be written is successfully written or failed, the embodiment of the present invention further provides the following implementation manners of steps c1 to c3:

[0087] c1 determines whether the number of successful log synchronization messages received by the master logical node is greater than or equal to the total number of original data after the data to be written is divided;

[0088] c2, if yes, then the main logic node will send a message to the service node indicating that the data has been written successfully;

[0089] c3. If not, the operation log is discarded through the main logic node, and a message indicating data writing failure is fed back to the service node.

[0090] In an embodiment of the present application, the service node may also feed back the received write result to the client, and promptly notify the user of the data write status.

[0091] In an optional embodiment, since there are multiple logical nodes in the data storage subsystem, in order to determine the master logical node and the slave logical node, the present invention also provides the following implementation of steps d1 to d3:

[0092] d1 determines whether the first slave logical node receives the keep-alive information from the master logical node; the first slave logical node is any one other than the master logical node;

[0093] d2, if not, initiate the master logical node election process;

[0094] d3: When there is a target logical node that receives a number of votes greater than or equal to the total number of copies of the original data after the data to be written is divided, the target logical node is determined as the master logical node.

[0095] In an optional implementation, when a data node is offline, data loss may occur. Therefore, an embodiment of the present invention provides a data recovery method. Figure 6 , Figure 6 This is a schematic flow chart of another data storage method provided by an embodiment of the present invention. That is, the data storage method provided by an embodiment of the present invention may further include the following steps:

[0096] S405: When the master logical node determines that there are offline slave logical nodes, and the offline slave logical nodes have not come online within a preset time period, it is detected whether the number of online slave logical nodes is greater than or equal to the total number of copies of the original data after the data to be written is divided.

[0097] In an embodiment of the present invention, the node offline determination time of the data storage subsystem can be set according to actual needs, for example, set to 1 minute. After being offline and not online within a preset time period, a new data block can be applied to the metadata server for data recovery. If no new data block can be applied, it will be removed from the data storage subsystem and the remaining node logs will be submitted.

[0098] S406: If yes, then send a request for a new data block to the metadata server through the master logical node, so that the metadata server creates a new data block and a new logical node on the physical storage node where the new data block is located.

[0099] S407: Update the full data recovery log of the master logical node to the startup state, and synchronize the full data recovery log to the slave logical nodes other than the offline slave logical node and the new logical node;

[0100] It should be noted that after receiving the full recovery message, all slave logical nodes suspend submitting updated data logs to the data storage. Data log submission can only be resumed when the full data recovery logs of all nodes are in the end state.

[0101] S408 : Generate new missing data based on the preset erasure code and the online data log of the slave logical node, and send the new missing data to the new logical node.

[0102] To understand the above, please refer to Figure 7 , Figure 5 FIG. 1 is another schematic diagram of a data storage scenario provided in an embodiment of the present invention. Figure 7 As shown, assuming that data with a length of 128 is originally to be written to the data block blk=1-0-4 of DN-5, and the starting write position is 128, when the data node DN-5 is offline, the data sent to DN-5 will be lost. At this time, you can first apply for a new data block. Assuming that the new data block corresponds to the data node DN-6, you can first create a new logical node on DN-6. After the new missing data is written by the main logical node, the missing data can be written to the new data block applied on DN-6. The data length is 128 and the starting write position is 128.

[0103] Based on the same invention concept, see Figure 8 , Figure 8 The functional module diagram of the data storage subsystem provided in an embodiment of the present invention includes:

[0104] Synchronization module 205-1, configured to synchronize the write operation log received by the master logical node to all slave logical nodes; wherein the master logical node is one of all logical nodes; and the slave logical node is a logical node other than the master logical node;

[0105] Determination module 205-2 is configured to, upon determining that the master logical node receives a message indicating a log synchronization failure, determine an abnormal logical node corresponding to the message; wherein the log synchronization failure indicates that data on a target physical storage node corresponding to the abnormal logical node is missing;

[0106] A generation module 205-3 is configured to generate missing data based on a preset erasure code and data logs of the remaining slave logical nodes excluding the abnormal logical node and the master logical node; the data logs are configured to record data written to the physical storage nodes corresponding to the logical nodes;

[0107] The storage module 205 - 4 is configured to recover data on the target physical storage node based on the missing data.

[0108] It is understandable that the synchronization module 205-1, the determination module 205-2, the generation module 205-3 and the storage module 205-4 can be executed in coordination. Figure 4 Each step in the process is performed to achieve the corresponding technical effects.

[0109] In an optional embodiment, the data storage subsystem may further include a sending module, and the determining module 205-2, the generating module 205-3 and the sending module may be executed in a coordinated manner. Figure 7 Steps S405 to S408 in the embodiment are performed to achieve the corresponding technical effects.

[0110] In an optional implementation, the generation module 205 - 3 is further configured to execute steps a1 to a2 to achieve corresponding technical effects.

[0111] In an optional implementation, the determination module 205 - 2 and the sending module may also be used to execute steps b1 to b3 and steps c1 to c3 to achieve corresponding technical effects.

[0112] In an optional implementation, the data storage subsystem may further include an election module, and the determination module 205 - 2 and the election module may further collaboratively perform steps d1 to d3 to achieve corresponding technical effects.

[0113] In an optional embodiment, the operation log is generated in the following manner: dividing the data to be written received by the service node to obtain the original data and verification data; distributing the transaction ID generated by the service node, as well as the original data and the verification data to the multiple physical storage nodes, so that each of the physical storage nodes writes the received transaction ID, and the original data or the verification data into the transaction log; determining whether the number of data sending success messages received by the service node is greater than or equal to the total number of copies of the original data; if so, generating the write operation log through the service node.

[0114] In an optional embodiment, the storage module 205-4 can also be used to: write the write operation log into the transaction log of the physical storage node corresponding to the master logical node; apply the write operation log through the master logical node so that the physical storage node where the master logical node is located writes the received original data or verification data into the data storage.

[0115] An embodiment of the present invention further provides a storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the data storage method described in any of the aforementioned embodiments. The computer-readable storage medium may be, but is not limited to, a USB flash drive, a mobile hard drive, ROM, RAM, PROM, EPROM, EEPROM, a magnetic disk, or an optical disk, among other media capable of storing program code.

[0116] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, improvements, and the like made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention. It should be noted that similar reference numerals and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it need not be further defined or explained in subsequent figures.

Claims

1. A data storage method, characterized in that: A data storage subsystem applied to a distributed storage system, the data storage subsystem comprising a plurality of logical nodes, each logical node corresponding to a physical storage node; the method comprising: Synchronize the write operation log received by the master logical node to all slave logical nodes; wherein the master logical node is one of all the logical nodes; and the slave logical nodes are logical nodes other than the master logical node; Determine whether the physical storage node corresponding to the slave logical node has the transaction ID in the write operation log; if the transaction ID does not exist, feedback a message indicating that log synchronization failed to the master logical node through the slave logical node; if the transaction ID exists, feedback a message indicating that log synchronization succeeded to the master logical node through the slave logical node; When it is determined that the master logical node has received a message indicating a log synchronization failure, determining an abnormal logical node corresponding to the message; wherein the log synchronization failure indicates that data on a target physical storage node corresponding to the abnormal logical node is missing; Generate missing data based on a preset erasure code and data logs of the remaining slave logical nodes except the abnormal logical node and the master logical node; the data logs are used to record data written to the physical storage nodes corresponding to the logical nodes; Data recovery is performed on the target physical storage node based on the missing data.

2. The data storage method according to claim 1, wherein: The distributed storage system includes a metadata server, and the metadata server interacts with the data storage subsystem for data. The method further includes: When the master logical node determines that there is an offline slave logical node, and the offline slave logical node has not come online within a preset time period, detecting whether the number of online slave logical nodes is greater than or equal to the total number of copies of the original data after the data to be written is divided; If yes, sending a request for a new data block to the metadata server through the master logical node, so that the metadata server creates a new data block and creates a new logical node on the physical storage node where the new data block is located; Generate a full data recovery log through the master logical node, and synchronize the full data recovery log to the slave logical nodes other than the offline slave logical node and the new logical node; Based on the preset erasure code and the data log of the online slave logical node, new missing data is generated, and the new missing data is sent to the new logical node.

3. The data storage method according to claim 1, wherein: The distributed storage system includes a metadata server, and the metadata server interacts with the data storage subsystem for data. The method further includes: Determine whether the number of successful log synchronization messages received by the master logical node is greater than or equal to the total number of copies of the original data after the data to be written is divided; If so, a message indicating that the data has been written successfully is fed back to the service node through the main logic node; If not, the operation log is discarded through the main logic node, and a message indicating that the data writing failed is fed back to the service node.

4. The data storage method according to claim 1, wherein: The method further comprises: determining whether a first slave logical node receives a keep-alive message from a master logical node; the first slave logical node is any one other than the master logical node; If not, initiate the master logical node election process; When there is a target logical node whose number of votes received is greater than or equal to the total number of copies of the original data after the data to be written is divided, the target logical node is determined as the master logical node.

5. The data storage method according to claim 3, characterized in that: The operation log is generated in the following way: Dividing the data to be written received by the service node to obtain the original data and verification data; Distributing the transaction ID generated by the service node, the original data, and the verification data to the plurality of physical storage nodes, so that each of the physical storage nodes writes the received transaction ID and the original data or the verification data into a transaction log; Determining whether the number of data sending success messages received by the service node is greater than or equal to the total number of copies of the original data; If so, the write operation log is generated by the service node.

6. The data storage method according to claim 1, wherein: The method further comprises: Writing the write operation log into the transaction log of the physical storage node corresponding to the master logical node; The write operation log is applied by the master logical node, so that the physical storage node where the master logical node is located writes the received original data or verification data into data storage.

7. A data storage subsystem, characterized in that: The data storage subsystem includes multiple logical nodes, each of which corresponds to a physical storage node, including: A synchronization module, configured to synchronize the write operation log received by the master logical node to all slave logical nodes; wherein the master logical node is one of all the logical nodes; and the slave logical nodes are logical nodes other than the master logical node; a determination module, configured to determine whether the physical storage node corresponding to the slave logical node has the transaction ID in the write operation log; if the transaction ID does not exist, feeding back a message indicating a log synchronization failure to the master logical node via the slave logical node; and if the transaction ID exists, feeding back a message indicating a log synchronization success to the master logical node via the slave logical node; The determining module is further configured to, when determining that the master logical node has received a message indicating a log synchronization failure, determine an abnormal logical node corresponding to the message; wherein the log synchronization failure indicates that data on a target physical storage node corresponding to the abnormal logical node is missing; a generation module, configured to generate missing data based on a preset erasure code and data logs of the remaining slave logical nodes except the abnormal logical node and the master logical node; the data logs are used to record data written to the physical storage nodes corresponding to the logical nodes; A storage module is used to recover data on the target physical storage node based on the missing data.

8. A distributed storage system, characterized in that: The distributed storage system includes a data storage subsystem, which is composed of multiple logical nodes, each logical node corresponds to a physical storage node, and the data storage subsystem is used to execute the data storage method according to any one of claims 1 to 6.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data storage method according to any one of claims 1 to 6 is implemented.

Images