Data recovery method, device and equipment

Abstract

The present invention provides a data recovery method, device and equipment, which can be applied to the field of computer technology. The method is applied to a data storage cluster, the data storage cluster includes multiple storage nodes, each storage node includes multiple data shards, and the method includes: in response to a data query request for the data storage cluster, based on the same data shard identification information, traversing the data table structure information of each data shard of each storage node; according to the traversal result, determining the abnormal data shard corresponding to the same data shard identification information; using the data shards on the normal storage nodes in the data storage cluster to vote on the abnormal data shards, and determining the target abnormal data shard from the abnormal primary data shard and the abnormal backup data shard; calling the data recovery component to perform data recovery processing on the target abnormal data shard.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more specifically to a data recovery method, device, equipment, medium and program product. Background Art

[0002] Distributed relational database clusters are used to split the stored data to ensure high availability of the cluster. However, in a production environment, when a physical machine has no battery or fails, causing a power outage, data may be written abnormally, resulting in inconsistency between the primary data shard and the backup data shard.

[0003] In the related technology, using related execution statements to query the distributed data storage cluster can only determine the problem of abnormal data writing, but cannot accurately identify the abnormal data shards. Therefore, when repairing the data, redeployment is required, which wastes system resources and reduces the availability of the cluster. Summary of the invention

[0004] In view of the above problems, the present invention provides a data recovery method, apparatus, device, medium and program product.

[0005] According to a first aspect of the present invention, a data recovery method is provided, which is applied to a data storage cluster, wherein the data storage cluster includes a plurality of storage nodes, each storage node includes a plurality of data shards, and includes:

[0006] In response to a data query request for a data storage cluster, based on the same data shard identification information, the data table structure information of each data shard of each storage node is traversed, wherein the data content stored on each data shard of each storage node is different, and two data shards with the same data shard identification information are respectively arranged on different storage nodes, and the two data shards with the same data shard identification information include a primary data shard and a backup data shard, and the data content stored in the primary data shard and the backup data shard is the same.

[0007] According to the traversal results, the abnormal data shards corresponding to the same data shard identification information are determined, wherein the abnormal data shards include abnormal primary data shards and abnormal backup data shards, the data structure information of the abnormal primary data shards and the data structure information of the abnormal backup data shards are inconsistent, and the storage nodes corresponding to the abnormal data shards are abnormal storage nodes.

[0008] The data shards on the normal storage nodes in the data storage cluster are used to vote on the abnormal data shards, and the target abnormal data shards are determined from the abnormal primary data shards and the abnormal backup data shards.

[0009] Call the data recovery component to perform data recovery processing on the target abnormal data shard.

[0010] According to an embodiment of the present invention, the data table structure information includes row number information and column number information; determining the abnormal data shard corresponding to the same data shard identification information according to the traversal result includes:

[0011] For the primary data shard and the backup data shard with the same data shard identification information, the row number information of the primary data shard and the row number information of the backup data shard are compared to obtain a row number comparison result.

[0012] The column number information of the primary data shard is compared with the column number information of the backup data shard to obtain a column number comparison result.

[0013] When the row number comparison result or the column number comparison result is inconsistent, the primary data shard and the backup data shard are determined as abnormal data shards corresponding to the same data shard identification information.

[0014] According to an embodiment of the present invention, voting is performed on abnormal data shards using data shards on normal storage nodes in a data storage cluster to determine a target abnormal data shard from abnormal primary data shards and abnormal backup data shards, including:

[0015] Determine the row and column information of the data shards on the normal storage nodes.

[0016] The row number information of the data shards on the normal storage node is compared with the row number information of the abnormal primary data shard and the row number information of the abnormal backup data shard to obtain a first voting result.

[0017] The column number information of the data shards on the normal storage nodes is compared with the column number information of the abnormal primary data shards and the column number information of the abnormal backup data shards to obtain the second voting result; the target voting result is determined according to the first voting result and the second voting result.

[0018] According to the target voting result, the target abnormal data shard is determined from the abnormal primary data shard and the abnormal backup data shard.

[0019] According to an embodiment of the present invention, calling a data recovery component to perform data recovery processing on a target abnormal data slice includes:

[0020] Delete the data table structure information and stored data content of the target abnormal data shard.

[0021] Acquire data table structure information and stored data content of a normal data shard corresponding to a target abnormal data shard, and transmit the data table structure information and stored data content of the normal data shard to the target abnormal data shard through a network connection channel, wherein the normal data shard is determined in the abnormal primary data shard and the abnormal backup data shard.

[0022] According to the data table structure information of the normal data shard, a new data table structure is constructed on the target abnormal data shard.

[0023] The stored data content of the normal data shard is imported into the new data table structure so as to perform data recovery processing on the target abnormal data shard.

[0024] According to an embodiment of the present invention, using the data table structure information of the normal data shard corresponding to the target abnormal data shard, performing data recovery processing on the target abnormal data shard also includes:

[0025] When performing data recovery processing on the target abnormal data slice, the data slice identification information corresponding to the target abnormal data slice is stored in the abnormal record file.

[0026] When data recovery of the target abnormal data shard fails, a recovery abnormality identifier is added to the data shard identifier information corresponding to the target abnormal data shard in the abnormal record file.

[0027] When the data recovery process is completed for the target abnormal data slice, the data slice identification information corresponding to the target abnormal data slice in the abnormal record file is deleted.

[0028] According to an embodiment of the present invention, data content update time information is also stored on the data slice; the above method further includes:

[0029] For the primary data shard and the backup data shard with the same data shard identification information, when the data table structure information of the primary data shard is consistent with the data table structure information of the backup data shard, the data content update time information of the primary data shard and the data content update time information of the backup data shard are compared to obtain a time comparison result.

[0030] When the time comparison results are inconsistent, the target update time information is determined from the data content update time information of the primary data shard and the data content update time information of the backup data shard.

[0031] The data shard corresponding to the target update time information of the data content is used as the new primary data shard.

[0032] Build a new backup data shard based on the new primary data shard.

[0033] According to an embodiment of the present invention, determining abnormal data fragments corresponding to the same data fragment identification information according to the traversal result also includes:

[0034] Determine the target data table structure based on the traversal results.

[0035] For the primary data shard and the backup data shard with the same data shard identification information, when the data table structure information of the primary data shard is consistent with the data table structure information of the backup data shard, the data table structure information of the primary data shard and the data table structure information of the backup data shard are compared with the target data table structure to obtain the data table structure comparison result.

[0036] When the structure comparison results are inconsistent, the primary data shard and the backup data shard are determined as abnormal data shards.

[0037] According to an embodiment of the present invention, the method further includes:

[0038] Before traversing the data table structure information of each data shard of each storage node, the control lock component is enabled.

[0039] After completing data recovery for the target abnormal data shard, close the control lock component.

[0040] A second aspect of the present invention provides a data recovery device, including a traversal module, a first determination module, a voting module and a recovery module.

[0041] A traversal module is used to respond to a data query request for a data storage cluster, and based on the same data shard identification information, traverse the data table structure information of each data shard of each storage node, wherein the data content stored on each data shard of each storage node is different, and two data shards with the same data shard identification information are respectively arranged on different storage nodes, and the two data shards with the same data shard identification information include a primary data shard and a backup data shard, and the data content stored in the primary data shard and the backup data shard is the same.

[0042] The first determination module is used to determine the abnormal data shard corresponding to the same data shard identification information based on the traversal results, wherein the abnormal data shard includes an abnormal primary data shard and an abnormal backup data shard, the data structure information of the abnormal primary data shard is inconsistent with the data structure information of the abnormal backup data shard, and the storage node corresponding to the abnormal data shard is an abnormal storage node.

[0043] The voting module is used to use the data shards on the normal storage nodes in the data storage cluster to vote on the abnormal data shards, and determine the target abnormal data shard from the abnormal primary data shards and the abnormal backup data shards.

[0044] The recovery module is used to call the data recovery component to perform data recovery processing on the target abnormal data shard.

[0045] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0046] The fourth aspect of the present invention further provides a computer-readable storage medium on which a computer program or instruction is stored, and the steps of the above method are implemented when the above computer program or instruction is executed by a processor.

[0047] The fifth aspect of the present invention also provides a computer program product, including a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0048] According to an embodiment of the present invention, in response to a data query request for a data storage cluster, based on the same data shard identification information, the data table structure information of each data shard of each storage node is traversed; based on the traversal result, the abnormal data shard corresponding to the same data shard identification information is determined; the data shards on the normal storage nodes in the data storage cluster are used to vote on the abnormal data shards, and the target abnormal data shards are determined from the abnormal primary data shards and the abnormal backup data shards; and the data recovery component is called to perform data recovery processing on the target abnormal data shard.

[0049] By using the data shards on the normal storage nodes to vote on the abnormal data shards, the target abnormal data shard with data writing abnormality due to power failure is determined among the identified abnormal data shards, and the target abnormal data shard is restored. This avoids the problem of not being able to accurately identify the abnormal data shards and needing to redeploy all data shards, thereby achieving the technical effect of reducing system resource loss and improving the availability of the data storage cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0051] Figure 1 An application scenario diagram of a data recovery method, apparatus, device, medium, and program product according to an embodiment of the present invention is shown;

[0052] Figure 2 A flow chart of a data recovery method according to an embodiment of the present invention is shown;

[0053] Figure 3 A flowchart of determining abnormal data slices corresponding to the same data slice identification information according to an embodiment of the present invention is shown;

[0054] Figure 4 A flowchart of determining a target abnormal data shard from an abnormal primary data shard and an abnormal backup data shard according to an embodiment of the present invention is shown;

[0055] Figure 5 A schematic diagram of a data storage cluster according to an embodiment of the present invention is shown;

[0056] Figure 6 A flowchart of performing data recovery processing on a target abnormal data slice according to an embodiment of the present invention is shown;

[0057] Figure 7 A schematic diagram showing a data recovery method according to an embodiment of the present invention is shown;

[0058] Figure 8 A structural block diagram of a data recovery device according to an embodiment of the present invention is shown; and

[0059] Fig. 9 A block diagram of an electronic device suitable for implementing a data recovery method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0060] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0061] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0062] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0063] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0064] In the related technology, using related execution statements to query the distributed data storage cluster can only determine the problem of abnormal data writing, but cannot accurately identify the abnormal data shards. Therefore, when repairing the data, redeployment is required, which wastes system resources and reduces the availability of the cluster.

[0065] An embodiment of the present invention provides a data recovery method, which is applied to a data storage cluster, wherein the data storage cluster includes multiple storage nodes, each storage node includes multiple data shards, and includes: in response to a data query request for the data storage cluster, based on the same data shard identification information, traversing the data table structure information of each data shard of each storage node, wherein the data content stored on each data shard of each storage node is different, two data shards with the same data shard identification information are respectively arranged on different storage nodes, the two data shards with the same data shard identification information include a primary data shard and a backup data shard, the primary data shard and The data content stored in the backup data shard is the same; according to the traversal result, the abnormal data shard corresponding to the same data shard identification information is determined, wherein the abnormal data shard includes an abnormal primary data shard and an abnormal backup data shard, the data structure information of the abnormal primary data shard is inconsistent with the data structure information of the abnormal backup data shard, and the storage node corresponding to the abnormal data shard is an abnormal storage node; the data shards on the normal storage nodes in the data storage cluster are used to vote on the abnormal data shards, and the target abnormal data shard is determined from the abnormal primary data shard and the abnormal backup data shard; the data recovery component is called to perform data recovery on the target abnormal data shard.

[0066] Figure 1 The application scenario diagram of the data recovery method, device, equipment, medium and program product according to the embodiment of the present invention is shown.

[0067] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0068] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0069] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0070] The server 105 may be a server that provides various services, such as a background management server (only as an example) that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as the data query request, and perform data recovery processing on the target abnormal data shards in the data storage cluster.

[0071] It should be noted that the data recovery method provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the data recovery device provided in the embodiment of the present invention can generally be set in the server 105. The data recovery method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the data recovery device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0072] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0073] Figure 2 A flow chart of a data recovery method according to an embodiment of the present invention is shown.

[0074] like Figure 2As shown, the data recovery method 200 of this embodiment includes operations S210 to S240, which are applied to a data storage cluster. The data storage cluster includes multiple storage nodes, and each storage node includes multiple data shards.

[0075] In operation S210, in response to a data query request for the data storage cluster, based on the same data shard identification information, the data table structure information of each data shard of each storage node is traversed, wherein the data content stored on each data shard of each storage node is different, and two data shards with the same data shard identification information are respectively arranged on different storage nodes, and the two data shards with the same data shard identification information include a primary data shard and a backup data shard, and the data content stored in the primary data shard and the backup data shard are the same.

[0076] According to an embodiment of the present invention, the data storage cluster may be a distributed cluster.

[0077] The data storage cluster includes multiple storage nodes, and uses the cluster layer associated with the data storage cluster to obtain data to be stored, splits the data to be stored into multiple data slices and stores them on multiple storage nodes to achieve efficient data storage.

[0078] According to an embodiment of the present invention, a data query request may be issued by a maintenance person to the data storage cluster when the data storage cluster loses power. The data query request may be a SQL statement such as load, insert select or commit.

[0079] The data query request may be sent to a cluster layer corresponding to the data storage cluster, and then the cluster layer sends the data query request to multiple storage nodes of the data storage cluster respectively.

[0080] According to an embodiment of the present invention, the data stored on the data shard may be in the form of a data table, and the table structure of the data table is used as the data table structure information.

[0081] Among them, each data shard can store multiple data tables with different table names. Before traversing the data table structure information of each data shard of each storage node, the table name to be queried is determined according to the data query request, and the data table to be traversed is determined based on the table name to be queried.

[0082] For example, data table 1, data table 2, and data table 3 are stored on the data shard of the data storage cluster, and the data query request is to query data table 1, then only data table 1 is traversed during traversal.

[0083] According to an embodiment of the present invention, the data slice identification information may be identification information used to distinguish data slices storing different data contents.

[0084] For example, a stored data is split into three parts and stored in three data shards respectively, and the data shard identification information thereof may be n1, n2 and n3 respectively.

[0085] In order to realize backup storage of the stored data, each data shard also has a corresponding backup data shard. For example, data shard n1 has a corresponding backup data shard, which is used to back up the data content on data shard n1. At this time, data shard n1 can be used as the main data shard n1, and the data identification information of the backup data shard is consistent with the data identification information of data shard n1, which is also n1, and data shard n1 and backup data shard n1 are stored on different storage nodes.

[0086] Similarly, data shard n2 has a corresponding backup data shard, which is used to back up the data content stored on data shard n2, and data shard n2 and backup data shard n2 are stored on different storage nodes; data shard n3 has a corresponding backup data shard, which is used to back up the data content stored on data shard n3, and data shard n3 and backup data shard n3 are stored on different storage nodes.

[0087] According to an embodiment of the present invention, for primary data shards and backup data shards with the same data shard identification information, by storing the primary data shards and backup data shards on different storage nodes, it is avoided that when one storage node is damaged, the primary data shards and backup data shards with the same data shard identification information on the storage node both become abnormal, thereby causing the data to be unrecoverable.

[0088] According to an embodiment of the present invention, when traversing the data table structure information of each data shard of each storage node, the data table structure information corresponding to all data shard identification information is obtained.

[0089] For example, for the data shard identification information n1, the data table structure information of the primary data shard and the data table structure information of the backup data shard corresponding to the data shard are obtained.

[0090] In operation S220, based on the traversal results, the abnormal data shards corresponding to the same data shard identification information are determined, wherein the abnormal data shards include abnormal primary data shards and abnormal backup data shards, the data structure information of the abnormal primary data shards is inconsistent with the data structure information of the abnormal backup data shards, and the storage node corresponding to the abnormal data shards is an abnormal storage node.

[0091] According to an embodiment of the present invention, based on the traversal results, it is determined whether the data structure information of the primary data shard and the data structure information of the backup data shard with the same data shard identification information are consistent. If they are inconsistent, the primary data shard and the backup data shard are abnormal data shards, that is, abnormal primary data shards and abnormal backup data shards.

[0092] According to an embodiment of the present invention, when there is at least one abnormal data shard on a storage node, the storage node is determined to be an abnormal storage node.

[0093] In operation S230, the abnormal data shards are voted on using the data shards on the normal storage nodes in the data storage cluster, and a target abnormal data shard is determined from the abnormal primary data shards and the abnormal backup data shards.

[0094] According to an embodiment of the present invention, a normal storage node may be a storage node other than an abnormal storage node among all storage nodes, that is, the normal storage node does not include an abnormal data slice.

[0095] According to an embodiment of the present invention, the target abnormal data shard may be a data shard among the abnormal primary data shard and the abnormal backup data shard, in which data writing abnormality occurs due to power failure of the data storage cluster.

[0096] According to an embodiment of the present invention, voting processing can be to use data shards on multiple normal storage nodes to judge the abnormal primary data shards and abnormal backup data shards in the abnormal data shards respectively, and select the target abnormal data shards from the abnormal primary data shards and the abnormal backup data shards.

[0097] In operation S240, a data recovery component is called to perform data recovery processing on the target abnormal data slice.

[0098] According to an embodiment of the present invention, the data recovery component may be a component for reconstructing a data table.

[0099] According to an embodiment of the present invention, after a target abnormal data shard is selected from the abnormal primary data shard and the abnormal backup data shard, another data shard other than the target abnormal data shard is used as a normal data shard, and data recovery processing is performed on the target abnormal data shard based on the normal data shard.

[0100] For example, among the abnormal primary data shard and the abnormal backup data shard, according to the target voting result, the abnormal primary data shard is determined to be the target abnormal data shard, and the abnormal backup data shard is determined to be the normal data shard. Based on the data table structure information of the abnormal backup data shard, data recovery processing is performed on the abnormal primary data shard.

[0101] According to an embodiment of the present invention, in response to a data query request for a data storage cluster, based on the same data shard identification information, the data table structure information of each data shard of each storage node is traversed; based on the traversal result, the abnormal data shard corresponding to the same data shard identification information is determined; the data shards on the normal storage nodes in the data storage cluster are used to vote on the abnormal data shards, and the target abnormal data shards are determined from the abnormal primary data shards and the abnormal backup data shards; and the data recovery component is called to perform data recovery processing on the target abnormal data shard.

[0102] By using the data shards on the normal storage nodes to vote on the abnormal data shards, the target abnormal data shard with data writing abnormality due to power failure is determined among the identified abnormal data shards, and the target abnormal data shard is restored. This avoids the problem of not being able to accurately identify the abnormal data shards and needing to redeploy all data shards, thereby achieving the technical effect of reducing system resource loss and improving the availability of the data storage cluster.

[0103] Figure 3 A flow chart of determining abnormal data slices corresponding to the same data slice identification information according to an embodiment of the present invention is shown.

[0104] According to an embodiment of the present invention, the data table structure information includes row number information and column number information; according to the traversal result, determining the abnormal data shard corresponding to the same data shard identification information includes operations S310 to S330.

[0105] In operation S310, for a primary data shard and a backup data shard having the same data shard identification information, row number information of the primary data shard and row number information of the backup data shard are compared to obtain a row number comparison result.

[0106] In operation S320, the column number information of the primary data shard and the column number information of the backup data shard are compared to obtain a column number comparison result.

[0107] In operation S330, when the row number comparison result or the column number comparison result is inconsistent, the primary data shard and the backup data shard are determined as abnormal data shards corresponding to the same data shard identification information.

[0108] According to an embodiment of the present invention, the row number information may be the number of row data in the data table of the data shard.

[0109] According to an embodiment of the present invention, the column number information may be the number of column data in the data table of the data shard, and the table header name corresponding to each column data.

[0110] According to an embodiment of the present invention, for a primary data shard and a backup data shard having the same data shard identification information, the number of row data of the primary data shard and the number of row data of the backup data shard are compared to determine whether the number of row data of the two are consistent, thereby obtaining a row number comparison result.

[0111] According to an embodiment of the present invention, the number of column data of the primary data shard and the table header name corresponding to each column data are compared with the number of column data of the backup data shard and the table header name corresponding to each column data to determine whether the number of column data and the table header name corresponding to each column data of the two are consistent, thereby obtaining a column number comparison result.

[0112] According to an embodiment of the present invention, when the row number comparison result or the column number comparison result is inconsistent, it is considered that one of the primary data shard and the backup data shard is abnormal, and the primary data shard and the backup data shard are both regarded as abnormal data shards.

[0113] According to an embodiment of the present invention, by comparing the data table structure information of the primary data shard and the data table structure information of the backup data shard with the same data shard identification information, the scope of abnormal storage nodes can be narrowed, avoiding the problem of performing abnormal detection on each storage node and thus wasting system resources.

[0114] Figure 4 A flow chart of determining a target abnormal data shard from abnormal primary data shards and abnormal backup data shards according to an embodiment of the present invention is shown.

[0115] According to an embodiment of the present invention, data shards on normal storage nodes in a data storage cluster are used to vote on abnormal data shards, and target abnormal data shards are determined from abnormal primary data shards and abnormal backup data shards, including operations S410 to S450.

[0116] In operation S410, row number information and column number information of a data slice on a normal storage node are determined.

[0117] In operation S420, the row number information of the data shards on the normal storage nodes is compared with the row number information of the abnormal primary data shards and the row number information of the abnormal backup data shards to obtain a first voting result.

[0118] In operation S430, the column number information of the data shards on the normal storage nodes is compared with the column number information of the abnormal primary data shards and the column number information of the abnormal backup data shards to obtain a second voting result.

[0119] In operation S440, a target voting result is determined according to the first voting result and the second voting result.

[0120] In operation S450, a target abnormal data shard is determined from the abnormal primary data shard and the abnormal backup data shard according to the target voting result.

[0121] According to an embodiment of the present invention, the row number information of the data shards on the normal storage node is compared with the row number information of the abnormal primary data shard and the row number information of the abnormal backup data shard.

[0122] If the row number information of the data shard on the normal storage node is consistent with the row number information of the abnormal primary data shard, the abnormal primary data shard will be recorded as 1 vote; if they are inconsistent, the abnormal primary data shard will be recorded as 0 votes.

[0123] If the row number information of the data shard on the normal storage node is consistent with the row number information of the abnormal standby data shard, the abnormal standby data shard will be recorded as 1 vote; if they are inconsistent, the abnormal standby data shard will be recorded as 0 votes.

[0124] After the data shards on all normal storage nodes have voted on the row number information of the abnormal primary data shard and the row number information of the abnormal backup data shard, the total number of votes for the row number information of the abnormal primary data shard and the total number of votes for the row number information of the abnormal backup data shard are counted and used as the first voting result.

[0125] According to an embodiment of the present invention, the column number information of the data shards on the normal storage node is compared with the column number information of the abnormal primary data shard and the column number information of the abnormal backup data shard.

[0126] If the column number information of the data shard on the normal storage node is consistent with the column number information of the abnormal primary data shard, the abnormal primary data shard will be recorded with 1 vote; if they are inconsistent, the abnormal primary data shard will be recorded with 0 votes.

[0127] If the column number information of the data shard on the normal storage node is consistent with the column number information of the abnormal standby data shard, the abnormal standby data shard will be recorded with 1 vote; if they are inconsistent, the abnormal standby data shard will be recorded with 0 votes.

[0128] After the data shards on all normal storage nodes have voted on the column number information of the abnormal primary data shard and the column number information of the abnormal backup data shard, the total votes for the column number information of the abnormal primary data shard and the total votes for the column number information of the abnormal backup data shard are counted respectively and used as the second voting result.

[0129] According to an embodiment of the present invention, based on the first voting result and the second voting result, the total number of votes for the row number information and the total number of votes for the column number information of the abnormal primary data shard are counted to obtain the total number of votes for the abnormal primary data shard. The total number of votes for the row number information and the total number of votes for the column number information of the abnormal backup data shard are counted to obtain the total number of votes for the abnormal backup data shard.

[0130] According to an embodiment of the present invention, the total number of votes for the abnormal primary data shards and the total number of votes for the abnormal backup data shards are taken as target voting results.

[0131] According to an embodiment of the present invention, among the abnormal primary data shard and the abnormal backup data shard, the data shard with the least total votes indicated in the target voting result is the target abnormal data shard.

[0132] Figure 5 A schematic diagram of a data storage cluster according to an embodiment of the present invention is shown.

[0133] According to an embodiment of the present invention, Figure 5 As shown, the data storage cluster 500 includes three storage nodes as an example, namely, storage node gnode1, storage node gnode2 and storage node gnode3, wherein storage node gnode1 includes primary data shard n1 and backup data shard n2, storage node gnode2 includes primary data shard n2 and backup data shard n3, and storage node gnode3 includes primary data shard n3 and backup data shard n1.

[0134] According to the traversal results, it is determined that the primary data shard n1 includes 4 column data and 3 row data, and the backup data shard n1 includes 3 column data and 3 row data, that is, the data table structure information of the primary data shard n1 and the data table structure information of the backup data shard n1 are inconsistent. The primary data shard n1 and the backup data shard n1 are abnormal data shards.

[0135] The storage node gnode1 corresponding to the primary data shard n1 and the storage node gnode3 corresponding to the backup data shard n1 are abnormal storage nodes, that is, the storage node gnode2 is a normal storage node.

[0136] The primary data shard n2 and the backup data shard n3 of the normal storage node gnode2 are used to vote for the primary data shard n1 and the backup data shard n1 respectively.

[0137] For example, the primary data shard n2 includes 3 column data and 3 row data, and the backup data shard n3 includes 3 column data and 3 row data.

[0138] If both primary data shard n2 and primary data shard n1 have 3 rows of data, primary data shard n1 will be counted as 1 vote; if both backup data shard n3 and primary data shard n1 have 3 rows of data, primary data shard n1 will be counted as 1 vote. The total number of votes for the number of rows of primary data shard n1 is 2.

[0139] If both primary data shard n2 and backup data shard n1 have 3 rows of data, then backup data shard n1 will have 1 vote; if both backup data shard n3 and backup data shard n1 have 3 rows of data, then backup data shard n1 will have 1 vote. The total number of votes for the number of rows of backup data shard n1 is 2.

[0140] If the data of the three columns of the primary data shard n2 is inconsistent with the data of the four columns of the primary data shard n1, the primary data shard n1 will be recorded with 0 votes; if the data of the three columns of the backup data shard n3 is inconsistent with the data of the four columns of the primary data shard n1, the primary data shard n1 will be recorded with 0 votes. The total number of votes for the column number information of the primary data shard n1 is 0 votes.

[0141] If both the primary data shard n2 and the backup data shard n1 have 3 columns of data, then the backup data shard n1 will be recorded with 1 vote; if both the backup data shard n3 and the backup data shard n1 have 3 columns of data, then the backup data shard n1 will be recorded with 1 vote. The total number of votes for the number of columns of the backup data shard n1 is 2.

[0142] After counting the above votes, the total votes of the primary data shard n1 include 2 votes for the total number of row information and 0 votes for the total number of column information, totaling 2 votes; the total votes of the backup data shard n1 include 2 votes for the total number of row information and 2 votes for the total number of column information, totaling 4 votes.

[0143] If the total number of votes for the primary data shard n1 is less than the total number of votes for the backup data shard n1, the primary data shard n1 is the target abnormal data shard, and the backup data shard n1 is the normal data shard.

[0144] According to an embodiment of the present invention, by using the data shards of normal storage nodes to vote for abnormal data shards, the problem of failure in identifying abnormal data shards caused by operations such as random selection can be avoided, and the recognition accuracy of abnormal data shards can be improved so as to perform targeted data recovery processing.

[0145] Figure 6 A flow chart of performing data recovery processing on a target abnormal data slice according to an embodiment of the present invention is shown.

[0146] According to an embodiment of the present invention, calling a data recovery component to perform data recovery processing on a target abnormal data slice includes operations S610 to S640.

[0147] In operation S610, the data table structure information and stored data content of the target abnormal data segment are deleted.

[0148] In operation S620, data table structure information and stored data content of a normal data shard corresponding to a target abnormal data shard are obtained, and the data table structure information and stored data content of the normal data shard are transmitted to the target abnormal data shard through a network connection channel, wherein the normal data shard is determined in the abnormal primary data shard and the abnormal backup data shard.

[0149] In operation S630, a new data table structure is constructed on the target abnormal data segment according to the data table structure information of the normal data segment.

[0150] In operation S640, the stored data content of the normal data segment is imported into the new data table structure, so as to perform data recovery processing on the target abnormal data segment.

[0151] According to an embodiment of the present invention, the data table structure information and the stored data content of the target abnormal data segment are deleted so as not to affect the data table structure information and the stored data content after data recovery.

[0152] According to an embodiment of the present invention, each data slice has a network connection channel for data transmission. The data table structure information and stored data content of the normal data slice are sent to the abnormal data slice through the network connection channel between the target abnormal data slice and the normal data slice.

[0153] For example, if the target abnormal data shard is an abnormal primary data shard, the abnormal backup data shard is a normal data shard, and the data table structure information and stored data content of the abnormal backup data shard are sent to the abnormal primary data shard through a network connection channel.

[0154] According to an embodiment of the present invention, after receiving the data table structure information and stored data content of the normal data shard, the abnormal data shard constructs a new data table structure based on the data table structure information of the normal data shard. The new data table structure is consistent with the data table structure information of the normal data shard. After the construction is completed, the stored data content of the normal data shard is imported into the new data table structure to complete the data recovery process.

[0155] According to an embodiment of the present invention, using the data table structure information of the normal data shard corresponding to the target abnormal data shard, performing data recovery processing on the target abnormal data shard also includes:

[0156] When performing data recovery processing on the target abnormal data slice, the data slice identification information corresponding to the target abnormal data slice is stored in the abnormal record file.

[0157] When data recovery of the target abnormal data shard fails, a recovery abnormality identifier is added to the data shard identifier information corresponding to the target abnormal data shard in the abnormal record file.

[0158] When the data recovery process is completed for the target abnormal data slice, the data slice identification information corresponding to the target abnormal data slice in the abnormal record file is deleted.

[0159] According to an embodiment of the present invention, when data recovery processing is performed on a target abnormal data slice, data slice identification information corresponding to the target abnormal data slice is stored in an abnormal record file so that maintenance personnel can confirm the abnormal data slice.

[0160] According to an embodiment of the present invention, when data recovery of a target abnormal data slice fails, if there is only the target abnormal data slice currently, data recovery processing is performed again, and a recovery abnormality identifier is added to the corresponding data slice identification information in the abnormal record file; if there are multiple target abnormal data slices currently, the target abnormal data slice is placed at the end of the data recovery queue and waits for data recovery processing. When data recovery of the target abnormal data slice fails again, a new recovery abnormality identifier is added to the corresponding data slice identification information in the abnormal record file.

[0161] According to an embodiment of the present invention, when data recovery processing is completed for a target abnormal data slice, the data slice identification information corresponding to the target abnormal data slice in the abnormal record file and the recovery abnormality identification corresponding to the data slice identification information are deleted.

[0162] According to an embodiment of the present invention, an abnormal record file is used to record the recovery status of the target abnormal data slice during data recovery processing, so that maintenance personnel can promptly process or issue corresponding instructions for the target abnormal data slice that has failed to recover multiple times.

[0163] According to an embodiment of the present invention, data content update time information is also stored on the data shard; the above method also includes: for the primary data shard and the backup data shard with the same data shard identification information, when the data table structure information of the primary data shard and the data table structure information of the backup data shard are consistent, the data content update time information of the primary data shard and the data content update time information of the backup data shard are compared to obtain a time comparison result; when the time comparison result is inconsistent, the target update time information is determined from the data content update time information of the primary data shard and the data content update time information of the backup data shard; the data shard corresponding to the data content target update time information is used as a new primary data shard; and a new backup data shard is constructed based on the new primary data shard.

[0164] According to an embodiment of the present invention, for a primary data shard and a backup data shard having the same data shard identification information, when the data table structure information of the primary data shard is consistent with the data table structure information of the backup data shard, it is necessary to judge the data content stored in the primary data shard and the data content stored in the backup data shard to confirm whether the two are consistent.

[0165] According to an embodiment of the present invention, the data content update time information may be the time when the data of the data slice was last updated.

[0166] According to an embodiment of the present invention, the data content update time information of the primary data shard and the data content update time information of the backup data shard are compared to determine whether the two are consistent. If they are consistent, it proves that the data content stored in the primary data shard is consistent with the data content stored in the backup data shard.

[0167] If there is inconsistency, it is necessary to determine the time when the data was last updated from the current time from the data content update time information of the primary data shard and the data content update time information of the backup data shard, that is, the target update time information, so as to determine the latest data content in the data content stored in the primary data shard and the data content stored in the backup data shard.

[0168] According to an embodiment of the present invention, the data shard corresponding to the data content target update time information is used as a new primary data shard. Without modifying the data table structure information, the data content stored in the new primary data shard is used to update the data content stored in the new backup data shard.

[0169] According to an embodiment of the present invention, if both the primary data shard n2 and the backup data shard n2 have 3 rows of data and 3 columns of data, that is, the data table structure information of the primary data shard n2 is consistent with the data table structure information of the backup data shard n2.

[0170] Determine that the data content update time information of the primary data shard n2 is updated at 8 o'clock, the data content update time information of the backup data shard n2 is updated at 9 o'clock, and the update time information closest to the current time is updated at 9 o'clock. The corresponding backup data shard n2 is used as the new primary data shard, and the data content stored in the original primary data shard n2 is updated.

[0171] According to an embodiment of the present invention, by using data content update time information to compare the primary data shard and the backup data shard with consistent data table structure information, omission of data shards with abnormal data content can be avoided, making data recovery processing more complete and improving the availability of the data storage cluster.

[0172] According to an embodiment of the present invention, based on the traversal results, the abnormal data shard corresponding to the same data shard identification information is determined, and the method also includes: based on the traversal results, determining the target data table structure; for the primary data shard and the backup data shard with the same data shard identification information, when the data table structure information of the primary data shard and the data table structure information of the backup data shard are consistent, the data table structure information of the primary data shard and the data table structure information of the backup data shard are compared with the target data table structure to obtain the data table structure comparison result; when the structure comparison results are inconsistent, the primary data shard and the backup data shard are determined as abnormal data shards.

[0173] According to an embodiment of the present invention, the target data table structure may be the data table structure that accounts for the largest proportion among the multiple data table structures. For the primary data shard and the backup data shard with the same data shard identification information, when the data table structure information of the primary data shard and the data table structure information of the backup data shard are consistent, for example, there are 7 primary data shards and backup data shards corresponding to the data shard identification information.

[0174] The data table structure information of the primary data shard and the data table structure information of the backup data shard corresponding to the data shard identification information n1 to the data shard identification information n5 are both 3 rows of data and 3 columns of data.

[0175] The data table structure information of the primary data shard and the data table structure information of the backup data shard corresponding to the data shard identification information n6 to the data shard identification information n7 are both 2 row data and 2 column data.

[0176] It can be seen that the data table structure information including 3 row data and 3 column data accounts for 5 / 7 of all the data table structure information, while the data table structure information including 2 row data and 2 column data accounts for 2 / 7 of all the data table structure information.

[0177] The data table structure information including 3 row data and 3 column data is used as the target data table structure information.

[0178] According to an embodiment of the present invention, the data table structure information of the primary data shard and the data table structure information of the backup data shard corresponding to each of the same data identification information are respectively compared with the target data table structure, and the data shard corresponding to the data table structure information inconsistent with the target data table structure is determined as an abnormal data shard.

[0179] For example, the target data table structure determined above is 3 rows of data and 3 columns of data. By comparing the data table structure information corresponding to the data shard identification information n1 to the data shard identification information n7 one by one, it can be determined that the data table structure information of the primary data shard and the data table structure information of the backup data shard corresponding to the data shard identification information n1 to the data shard identification information n5 are respectively consistent with the target data table structure information, then there is no abnormality in the primary data shard and the backup data shard corresponding to the data shard identification information n1 to the data shard identification information n5.

[0180] If the data table structure information of the primary data shard and the data table structure information of the backup data shard corresponding to the data shard identification information n6~data shard identification information n7 are inconsistent with the target data table structure information, then the primary data shard and the backup data shard corresponding to the data shard identification information n6~data shard identification information n7 are abnormal data shards.

[0181] According to an embodiment of the present invention, the method further includes: before traversing the data table structure information of each data slice of each storage node, opening the control lock component; after completing data recovery of the target abnormal data slice, closing the control lock component.

[0182] According to an embodiment of the present invention, the control lock component is used to prevent other operations from operating on data in the data storage cluster during data recovery.

[0183] According to an embodiment of the present invention, a control lock component is provided to ensure the stability of the data storage cluster during data recovery.

[0184] Figure 7 A schematic diagram of a data recovery method according to an embodiment of the present invention is shown.

[0185] Data recovery method 700 Figure 7 As shown, the data recovery method 700 includes operations S701 to S713.

[0186] In operation S701, data table structure information of each data slice of each storage node is traversed.

[0187] According to the traversal result, operation S702 is performed to determine whether the data table structure information of the primary data shard and the data table structure information of the backup data shard having the same data shard identification information are consistent.

[0188] If they are inconsistent, operation S703 is performed to determine the primary data shard and the backup data shard as abnormal data shards.

[0189] Operation S704 is executed to perform voting processing on the abnormal data shards using the data shards on the normal storage nodes, and to confirm the target abnormal data shards among the abnormal primary data shards and the abnormal backup data shards.

[0190] In operation S705, data recovery processing is performed on the target abnormal data slice determined above.

[0191] If it is determined to be consistent in operation S702, operation S706 is performed to compare the data content update time information of the primary data shard with the data content update time information of the backup data shard to obtain a time comparison result.

[0192] In operation S707, it is determined whether the time comparison result indicates consistency.

[0193] If they are consistent, operation S708 is executed without processing the primary data shard and the backup data shard.

[0194] If they are inconsistent, operation S709 is performed to determine the target update time information, and a new primary data shard is determined according to the target update time information.

[0195] If it is determined to be consistent in operation S702, operation S710 is performed to determine the target data table structure according to the traversal result.

[0196] Operation S711 is performed to compare the data table structure information of the data shard and the data table structure information of the backup data shard with the target data table structure to obtain a data table structure comparison result.

[0197] In operation S712, it is determined whether the data table structure comparison result indicates consistency.

[0198] If they are consistent, operation S708 is executed without processing the primary data shard and the backup data shard.

[0199] If they are inconsistent, operation S713 is performed to determine the primary data shard and the backup data shard as abnormal data shards.

[0200] Based on the above data recovery method, the present invention also provides a data recovery device. Figure 8 The device is described in detail.

[0201] Figure 8 A structural block diagram of a data recovery device according to an embodiment of the present invention is shown.

[0202] like Figure 8 As shown, the data recovery device 800 of this embodiment includes a traversal module 810 , a first determination module 820 , a voting module 830 and a recovery module 840 .

[0203] The traversal module 810 is used to respond to a data query request for the data storage cluster and traverse the data table structure information of each data shard of each storage node based on the same data shard identification information, wherein the data content stored on each data shard of each storage node is different, and two data shards with the same data shard identification information are respectively arranged on different storage nodes, and the two data shards with the same data shard identification information include a primary data shard and a backup data shard, and the data content stored in the primary data shard and the backup data shard is the same. In one embodiment, the traversal module 810 can be used to perform the operation S210 described above, which will not be repeated here.

[0204] The first determination module 820 is used to determine the abnormal data shard corresponding to the same data shard identification information according to the traversal result, wherein the abnormal data shard includes an abnormal primary data shard and an abnormal backup data shard, the data structure information of the abnormal primary data shard is inconsistent with the data structure information of the abnormal backup data shard, and the storage node corresponding to the abnormal data shard is an abnormal storage node. In one embodiment, the first determination module 820 can be used to perform the operation S220 described above, which will not be repeated here.

[0205] The voting module 830 is used to perform voting on the abnormal data shards using the data shards on the normal storage nodes in the data storage cluster, and determine the target abnormal data shards from the abnormal primary data shards and the abnormal backup data shards. In one embodiment, the voting module 530 can be used to perform the operation S230 described above, which will not be repeated here.

[0206] The recovery module 840 is used to call the data recovery component to perform data recovery processing on the target abnormal data slice. In one embodiment, the recovery module 840 can be used to perform the operation S240 described above, which will not be described in detail here.

[0207] According to an embodiment of the present invention, in response to a data query request for a data storage cluster, based on the same data shard identification information, the data table structure information of each data shard of each storage node is traversed; based on the traversal result, the abnormal data shard corresponding to the same data shard identification information is determined; the data shards on the normal storage nodes in the data storage cluster are used to vote on the abnormal data shards, and the target abnormal data shards are determined from the abnormal primary data shards and the abnormal backup data shards; and the data recovery component is called to perform data recovery processing on the target abnormal data shard.

[0208] By using the data shards on the normal storage nodes to vote on the abnormal data shards, the target abnormal data shard with data writing abnormality due to power failure is determined among the identified abnormal data shards, and the target abnormal data shard is restored. This avoids the problem of not being able to accurately identify the abnormal data shards and needing to redeploy all data shards, thereby achieving the technical effect of reducing system resource loss and improving the availability of the data storage cluster.

[0209] According to an embodiment of the present invention, any multiple modules of the traversal module 810, the first determination module 820, the voting module 830 and the recovery module 840 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the traversal module 810, the first determination module 820, the voting module 830 and the recovery module 840 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in a suitable combination of any of them. Alternatively, at least one of the traversal module 810, the first determination module 820, the voting module 830, and the recovery module 840 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be performed.

[0210] According to an embodiment of the present invention, the first determination module 820 includes a first comparison submodule, a second comparison submodule and a first determination submodule.

[0211] The first comparison submodule is used to compare the row number information of the primary data shard and the row number information of the backup data shard with respect to the primary data shard and the backup data shard having the same data shard identification information, and obtain a row number comparison result.

[0212] The second comparison submodule is used to compare the column number information of the primary data shard with the column number information of the backup data shard to obtain a column number comparison result.

[0213] The first determination submodule is used to determine the primary data shard and the backup data shard as abnormal data shards corresponding to the same data shard identification information when the row number comparison result or the column number comparison result is inconsistent.

[0214] According to an embodiment of the present invention, the voting module 830 includes a second determination submodule, a third comparison submodule, a fourth comparison submodule, a third determination submodule and a fourth determination submodule.

[0215] The second determination submodule is used to determine the row number information and column number information of the data shards on the normal storage node.

[0216] The third comparison submodule is used to compare the row number information of the data shards on the normal storage node with the row number information of the abnormal primary data shards and the row number information of the abnormal backup data shards to obtain a first voting result.

[0217] The fourth comparison submodule is used to compare the column number information of the data shards on the normal storage node with the column number information of the abnormal primary data shard and the column number information of the abnormal backup data shard, respectively, to obtain a second voting result.

[0218] The third determination submodule is used to determine the target voting result according to the first voting result and the second voting result.

[0219] The fourth determination submodule is used to determine a target abnormal data shard from the abnormal primary data shard and the abnormal backup data shard according to the target voting result.

[0220] According to an embodiment of the present invention, the recovery module 840 includes a first deletion submodule, an acquisition submodule, a construction submodule and an import submodule.

[0221] The first deletion submodule is used to delete the data table structure information and stored data content of the target abnormal data segment.

[0222] The acquisition submodule is used to obtain the data table structure information and stored data content of the normal data shard corresponding to the target abnormal data shard, and transmit the data table structure information and stored data content of the normal data shard to the target abnormal data shard through a network connection channel, wherein the normal data shard is determined in the abnormal primary data shard and the abnormal backup data shard.

[0223] The construction submodule is used to construct a new data table structure on the target abnormal data shard according to the data table structure information of the normal data shard.

[0224] The import submodule is used to import the stored data content of the normal data segment into the new data table structure so as to perform data recovery processing on the target abnormal data segment.

[0225] According to an embodiment of the present invention, the recovery module 840 further includes a storage submodule, an addition submodule and a second deletion submodule.

[0226] The storage submodule is used to store the data slice identification information corresponding to the target abnormal data slice into the abnormal record file when performing data recovery processing on the target abnormal data slice.

[0227] Add a submodule to add a recovery exception identifier to the data slice identifier information corresponding to the target abnormal data slice in the abnormal record file when data recovery of the target abnormal data slice fails.

[0228] The second deletion submodule is used to delete the data slice identification information corresponding to the target abnormal data slice in the abnormal record file when the data recovery process is completed for the target abnormal data slice.

[0229] According to an embodiment of the present invention, the data recovery device 800 further includes a comparison module, a second determination module, a serving module and a construction module.

[0230] The comparison module is used to compare the data content update time information of the primary data shard and the data content update time information of the backup data shard with the same data shard identification information, and obtain a time comparison result when the data table structure information of the primary data shard is consistent with the data table structure information of the backup data shard.

[0231] The second determination module is used to determine the target update time information from the data content update time information of the primary data shard and the data content update time information of the backup data shard when the time comparison results are inconsistent.

[0232] As a module, it is used to take the data slice corresponding to the data content target update time information as the new main data slice.

[0233] The construction module is used to construct a new backup data shard based on the new primary data shard.

[0234] According to an embodiment of the present invention, the first determination module 820 further includes a fifth determination submodule, a fifth comparison submodule and a sixth determination submodule.

[0235] The fifth determination submodule is used to determine the target data table structure according to the traversal result.

[0236] The fifth comparison submodule is used to compare the data table structure information of the primary data shard and the data table structure information of the backup data shard with the target data table structure for the primary data shard and the backup data shard with the same data shard identification information, so as to obtain the data table structure comparison result.

[0237] The sixth determination submodule is used to determine the primary data shard and the backup data shard as abnormal data shards when the structure comparison results are inconsistent.

[0238] According to an embodiment of the present invention, the data recovery device 800 further includes an opening module and a closing module.

[0239] The opening module is used to open the control lock component before traversing the data table structure information of each data slice of each storage node.

[0240] The closing module is used to close the control lock component after completing data recovery of the target abnormal data shard.

[0241] Fig. 9 A block diagram of an electronic device suitable for implementing a data recovery method according to an embodiment of the present invention is shown.

[0242] like Fig. 9 As shown, the electronic device 900 according to an embodiment of the present invention includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 to a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0243] In RAM 603, various programs and data required for the operation of electronic device 900 are stored. Processor 901, ROM 902 and RAM 903 are connected to each other via bus 904. Processor 901 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 902 and / or RAM 903. It should be noted that the program can also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0244] According to an embodiment of the present invention, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the input / output (I / O) interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0245] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0246] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 902 and / or RAM 903 described above and / or one or more memories other than ROM 902 and RAM 903.

[0247] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the data recovery method provided by the embodiment of the present invention.

[0248] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 901. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0249] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0250] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0251] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0252] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0253] It will be appreciated by those skilled in the art that the various embodiments of the present invention and / or the features described therein may be combined in a variety of ways and / or combinations, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the various embodiments of the present invention and / or the features described therein may be combined in a variety of ways and / or combinations. All of these combinations and / or combinations fall within the scope of the present invention.

[0254] The embodiments of the present invention have been described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment has been described above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. The scope of the present invention is limited by the appended and its equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A data recovery method, applied to a data storage cluster, wherein the data storage cluster includes a plurality of storage nodes, each of which includes a plurality of data shards, characterized in that: The method comprises: In response to a data query request for the data storage cluster, based on the same data shard identification information, traverse the data table structure information of each data shard of each storage node, wherein the data content stored on each data shard of each storage node is different, and the two data shards with the same data shard identification information are respectively arranged on different storage nodes, and the two data shards with the same data shard identification information include a primary data shard and a backup data shard, and the data content stored in the primary data shard and the backup data shard is the same; According to the traversal result, an abnormal data shard corresponding to the same data shard identification information is determined, wherein the abnormal data shard includes an abnormal primary data shard and an abnormal backup data shard, the data structure information of the abnormal primary data shard is inconsistent with the data structure information of the abnormal backup data shard, and the storage node corresponding to the abnormal data shard is an abnormal storage node; The abnormal data shards are voted on by using the data shards on the normal storage nodes in the data storage cluster, and a target abnormal data shard is determined from the abnormal primary data shards and the abnormal backup data shards; The data recovery component is called to perform data recovery processing on the target abnormal data slice.

2. The method according to claim 1, characterized in that The data table structure information includes row number information and column number information; The determining, according to the traversal result, the abnormal data slice corresponding to the same data slice identification information comprises: For the primary data shard and the backup data shard having the same data shard identification information, comparing the row number information of the primary data shard with the row number information of the backup data shard to obtain a row number comparison result; Comparing the column number information of the primary data shard with the column number information of the backup data shard to obtain a column number comparison result; When the row number comparison result or the column number comparison result is inconsistent, the primary data shard and the backup data shard are determined as abnormal data shards corresponding to the same data shard identification information.

3. The method according to claim 2, characterized in that The method of voting the abnormal data shards by using the data shards on the normal storage nodes in the data storage cluster to determine the target abnormal data shards from the abnormal primary data shards and the abnormal backup data shards includes: Determine the row number information and column number information of the data shards on the normal storage node; Compare the row number information of the data shards on the normal storage node with the row number information of the abnormal primary data shards and the row number information of the abnormal backup data shards to obtain a first voting result; The column number information of the data shards on the normal storage node is compared with the column number information of the abnormal primary data shard and the column number information of the abnormal backup data shard to obtain a second voting result; Determining a target voting result according to the first voting result and the second voting result; According to the target voting result, a target abnormal data shard is determined from the abnormal primary data shard and the abnormal backup data shard.

4. The method according to claim 1, characterized in that: The calling of the data recovery component to perform data recovery processing on the target abnormal data slice includes: Delete the data table structure information and stored data content of the target abnormal data slice; Acquire data table structure information and stored data content of a normal data shard corresponding to the target abnormal data shard, and transmit the data table structure information and stored data content of the normal data shard to the target abnormal data shard through a network connection channel, wherein the normal data shard is determined in the abnormal primary data shard and the abnormal backup data shard; According to the data table structure information of the normal data shard, a new data table structure is constructed on the target abnormal data shard; The stored data content of the normal data slice is imported into the new data table structure so as to perform data recovery processing on the target abnormal data slice.

5. The method according to claim 4, characterized in that The step of performing data recovery processing on the target abnormal data slice by using the data table structure information of the normal data slice corresponding to the target abnormal data slice further includes: When performing data recovery processing on the target abnormal data slice, storing the data slice identification information corresponding to the target abnormal data slice in an abnormal record file; When data recovery of the target abnormal data shard fails, adding a recovery abnormality identifier to the data shard identifier information corresponding to the target abnormal data shard in the abnormal record file; When the data recovery process for the target abnormal data slice is completed, the data slice identification information corresponding to the target abnormal data slice in the abnormal record file is deleted.

6. The method according to claim 1, characterized in that The data slice also stores data content update time information; The method further comprises: For the primary data shard and the backup data shard with the same data shard identification information, when the data table structure information of the primary data shard is consistent with the data table structure information of the backup data shard, compare the data content update time information of the primary data shard with the data content update time information of the backup data shard to obtain a time comparison result; In the case where the time comparison results are inconsistent, determining the target update time information from the data content update time information of the primary data shard and the data content update time information of the backup data shard; Using the data slice corresponding to the target update time information of the data content as a new primary data slice; A new backup data shard is constructed based on the new primary data shard.

7. The method according to claim 1, characterized in that The step of determining the abnormal data fragment corresponding to the same data fragment identification information according to the traversal result further includes: Determine the target data table structure based on the traversal results; For the primary data shard and the backup data shard having the same data shard identification information, when the data table structure information of the primary data shard is consistent with the data table structure information of the backup data shard, the data table structure information of the primary data shard and the data table structure information of the backup data shard are compared with the target data table structure to obtain a data table structure comparison result; When the comparison results of the data table structures are inconsistent, the primary data shard and the backup data shard are determined as the abnormal data shards.

8. The method according to claim 1, characterized in that The method further comprises: Before traversing the data table structure information of each data slice of each storage node, opening a control lock component; After completing data recovery of the target abnormal data slice, close the control lock component.

9. A data recovery device, applied to a data storage cluster, wherein the data storage cluster includes a plurality of storage nodes, each of which includes a plurality of data shards, characterized in that: The device comprises: A traversal module, configured to traverse the data table structure information of each data shard of each storage node in response to a data query request for the data storage cluster, based on the same data shard identification information, wherein the data content stored on each data shard of each storage node is different, and the two data shards with the same data shard identification information are respectively arranged on different storage nodes, and the two data shards with the same data shard identification information include a primary data shard and a backup data shard, and the data content stored in the primary data shard and the backup data shard is the same; A first determination module is used to determine, according to the traversal result, an abnormal data shard corresponding to the same data shard identification information, wherein the abnormal data shard includes an abnormal primary data shard and an abnormal backup data shard, the data structure information of the abnormal primary data shard is inconsistent with the data structure information of the abnormal backup data shard, and the storage node corresponding to the abnormal data shard is an abnormal storage node; A voting module, configured to perform voting processing on the abnormal data shards by using the data shards on the normal storage nodes in the data storage cluster, and determine a target abnormal data shard from the abnormal primary data shards and the abnormal backup data shards; and The recovery module is used to call the data recovery component to perform data recovery processing on the target abnormal data slice.

10. An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

Images