Data backup method and device, electronic equipment and storage medium

Abstract

The invention provides a data backup method and device, electronic equipment and a storage medium, and relates to the technical field of data storage. The method comprises the following steps: in response to a data writing request for target data, acquiring node information corresponding to each node set in a data backup system; each node set comprises a first node for data sharing between a working main node and a plurality of standby main nodes in the data backup system, and a second node for determining a new working main node from the plurality of standby main nodes when the working main node fails; based on the information of each node, screening out two target slave node sets meeting a first node quantity requirement and a second node quantity requirement from the plurality of candidate slave node sets; the target data are backed up through the two target slave node sets, and storage positions corresponding to the multiple pieces of backup data are recorded. Therefore, the data management cost required by data backup is reduced while complete data backup is realized.

Description

Technical Field

[0001] The present application relates to the technical field of data storage, and in particular to a data backup method, device, electronic device and storage medium. Background Art

[0002] In recent years, with the rapid development of information technology, huge amounts of data have been generated in many fields (such as finance, medical care, etc.), and the amount of data is still increasing at a relatively fast rate. Data security and availability have become an indispensable part of the operation of various enterprises. In order to ensure that data can be retrieved in time when it is damaged or lost, data backup is usually used to protect the data. The main data is backed up to generate backup data. When the main data is lost or damaged, the backup data can be used to restore the main data.

[0003] However, although the current data backup method can improve data security by backing up data in multiple computer rooms and / or multiple clusters, it will also increase data management costs. For example, hardware costs and operation and maintenance costs. In addition, incomplete data backup may still occur. Therefore, how to achieve complete data backup while reducing the data management costs required for backup data is an urgent problem to be solved. Summary of the invention

[0004] The embodiments of the present application provide a data backup method, device, electronic device and storage medium, which are used to reduce the data management cost required for backing up data while achieving complete backup of data.

[0005] In a first aspect, an embodiment of the present application provides a data backup method, which is applied to a data backup system, and the method includes:

[0006] In response to a data write request for target data, node information corresponding to each node set in the data backup system is obtained; wherein each node set includes a first node and a second node, the first node is used for data sharing between a working master node and a plurality of backup master nodes in the data backup system, and the second node is used for determining a new working master node from a plurality of backup master nodes when a working master node fails;

[0007] Based on the obtained information of each node, two target slave node sets that meet the first node quantity requirement and the second node quantity requirement are screened out from the multiple candidate slave node sets included in each node set; wherein the multiple slave nodes included in each candidate slave node set are used to back up and store the data written to the working master node;

[0008] The target data is backed up through two target slave node sets, and storage locations corresponding to a plurality of backup data are recorded; wherein each storage location is used to indicate a slave node storing the corresponding backup data.

[0009] In an optional embodiment, based on the obtained information of each node, two target slave node sets that meet the first node quantity requirement and the second node quantity requirement are screened out from a plurality of candidate slave node sets included in each node set, including:

[0010] Based on the first node quantity and the second node quantity corresponding to each node set respectively determined by the multiple node information, three node sets including the local node set where the working master node is located are selected from each node set; the total number of first nodes and the total number of second nodes of any two node sets in the three node sets are respectively greater than half of the total number of first nodes and half of the total number of second nodes of the three node sets;

[0011] Two target slave node sets are screened out from at least two candidate slave node sets included in the three node sets.

[0012] In an optional embodiment, two target slave node sets are screened out from at least two candidate slave node sets included in the three node sets, including:

[0013] If the local node set includes slave nodes, the candidate slave node set included in the local node set is used as the first target slave node set;

[0014] And, a candidate slave node set included in any one of the three node sets except the local node set is used as the second target slave node set.

[0015] In an optional embodiment, two target slave node sets are screened out from at least two candidate slave node sets included in the three node sets, including:

[0016] If the local node set does not include slave nodes, candidate slave node sets respectively included in two node sets of the three node sets except the local node set are used as two target slave node sets.

[0017] In an optional embodiment, backing up the target data through two target slave node sets includes:

[0018] For the two target slave node sets, perform the following operations respectively:

[0019] Obtaining the remaining amount of resources of each of the plurality of slave nodes included in a target slave node set, and based on the obtained plurality of remaining amounts of resources, performing data backup priority sorting on the plurality of slave nodes to obtain a corresponding priority sorting result;

[0020] Based on the priority sorting result, select slave nodes that meet the preset number from multiple slave nodes;

[0021] The target data is backed up separately through a preset number of slave nodes.

[0022] In an optional embodiment, after recording the storage locations corresponding to the plurality of backup data respectively, the method further includes:

[0023] In response to a data read request for target data, obtaining an operating status of a local node set;

[0024] If the local node set includes a target slave node set, and the operation status indicates that the local node set is still in a normal state, the backup data is read from the corresponding slave node based on the storage location of the backup data associated with the target slave node set.

[0025] In an optional embodiment, after obtaining the running status of the local node set, the method further includes:

[0026] If the local node set includes a target slave node set, but the operating status indicates that the local node set is in an abnormal state, the backup data is read from the corresponding slave node based on the storage location of the backup data associated with the other target slave node set of the two target slave node sets.

[0027] In a second aspect, an embodiment of the present application further provides a data backup device, which is applied to a data backup system, and the device includes:

[0028] An information acquisition module, for obtaining node information corresponding to each node set in the data backup system in response to a data write request for target data; wherein each node set includes a first node and a second node, the first node is used for data sharing between a working master node and a plurality of backup master nodes in the data backup system, and the second node is used for determining a new working master node from a plurality of backup master nodes when a working master node fails;

[0029] A set screening module, used to screen out two target slave node sets that meet the first node quantity requirement and the second node quantity requirement from the multiple candidate slave node sets included in each node set based on the obtained information of each node; wherein the multiple slave nodes included in each candidate slave node set are used to back up and store the data written to the working master node;

[0030] The data backup module is used to back up the target data through two target slave node sets and record storage locations corresponding to multiple backup data respectively; wherein each storage location is used to indicate the slave node storing the corresponding backup data.

[0031] In an optional embodiment, when two target slave node sets that meet the first node quantity requirement and the second node quantity requirement are screened out from a plurality of candidate slave node sets included in each node set based on the obtained information of each node, the set screening module is specifically used to:

[0032] Based on the first node quantity and the second node quantity corresponding to each node set respectively determined by the multiple node information, three node sets including the local node set where the working master node is located are selected from each node set; the total number of first nodes and the total number of second nodes of any two node sets in the three node sets are respectively greater than half of the total number of first nodes and half of the total number of second nodes of the three node sets;

[0033] Two target slave node sets are screened out from at least two candidate slave node sets included in the three node sets.

[0034] In an optional embodiment, when two target slave node sets are screened out from at least two candidate slave node sets included in the three node sets, the set screening module is specifically configured to:

[0035] If the local node set includes slave nodes, the candidate slave node set included in the local node set is used as the first target slave node set;

[0036] And, a candidate slave node set included in any one of the three node sets except the local node set is used as the second target slave node set.

[0037] In an optional embodiment, when two target slave node sets are screened out from at least two candidate slave node sets included in the three node sets, the set screening module is specifically configured to:

[0038] If the local node set does not include slave nodes, candidate slave node sets respectively included in two node sets of the three node sets except the local node set are used as two target slave node sets.

[0039] In an optional embodiment, when backing up the target data through two target slave node sets, the data backup module is specifically used to:

[0040] For the two target slave node sets, perform the following operations respectively:

[0041] Obtaining the remaining amount of resources of each of the plurality of slave nodes included in a target slave node set, and based on the obtained plurality of remaining amounts of resources, performing data backup priority sorting on the plurality of slave nodes to obtain a corresponding priority sorting result;

[0042] Based on the priority sorting result, select slave nodes that meet the preset number from multiple slave nodes;

[0043] The target data is backed up separately through a preset number of slave nodes.

[0044] In an optional embodiment, after recording the storage locations corresponding to the plurality of backup data respectively, the device further includes a data reading module, the data reading module being specifically used for:

[0045] In response to a data read request for target data, obtaining an operating status of a local node set;

[0046] If the local node set includes a target slave node set, and the operation status indicates that the local node set is still in a normal state, the backup data is read from the corresponding slave node based on the storage location of the backup data associated with the target slave node set.

[0047] In an optional embodiment, after acquiring the running status of the local node set, the data reading module is further used to:

[0048] If the local node set includes a target slave node set, but the operating status indicates that the local node set is in an abnormal state, the backup data is read from the corresponding slave node based on the storage location of the backup data associated with the other target slave node set of the two target slave node sets.

[0049] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0050] Processor; and

[0051] Memory for storing programs,

[0052] The program includes instructions, and when the instructions are executed by the processor, the processor executes the data backup method as described in the first aspect.

[0053] In a fourth aspect, an embodiment of the present application further provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the data backup method as described in the first aspect.

[0054] In a fifth aspect, the present application provides a computer program product, which, when called by a computer, enables the computer to execute the steps of the data backup method as described in the first aspect.

[0055] The beneficial effects of this application are as follows:

[0056] In the data backup method provided in the embodiment of the present application, only by satisfying the first node quantity requirement and the second node quantity requirement for two target slave node sets, it is possible to complete the backup storage of the target data while writing the target data. By backing up the target data with a minimum set of slave nodes, the hardware cost and operation and maintenance cost can be reduced while achieving complete disaster recovery of the data, that is, the data management cost required for backing up the data is reduced.

[0057] Furthermore, since the node set includes a first node for data sharing between a working master node and a plurality of backup master nodes in the data backup system, and a second node for determining a new working master node from a plurality of backup master nodes when a working master node fails, the availability and integrity of target data are ensured while minimizing the impact of data loss due to hardware failures, natural disasters, and the like.

[0058] In addition, other features and advantages of the present application will be described in the subsequent description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described here are used to provide a further understanding of the present application, constitute a part of the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0060] Figure 1 A schematic diagram of the system architecture of a data backup system applicable to an embodiment of the present application;

[0061] Figure 2 A schematic diagram of node composition of a node set provided in an embodiment of the present application;

[0062] Figure 3 A schematic diagram of the structure of an HDFS cluster deployment provided in an embodiment of the present application;

[0063] Figure 4 A schematic diagram of an implementation flow of a data backup method provided in an embodiment of the present application;

[0064] Figure 5 A logical schematic diagram of a screening target slave node set provided in an embodiment of the present application;

[0065] Fig. 6A A schematic diagram of a specific scenario of an AZ failure provided in an embodiment of the present application;

[0066] Figure 6B A schematic diagram of a specific scenario of another AZ failure provided in an embodiment of the present application;

[0067] Figure 6C A schematic diagram of a specific scenario of another AZ failure provided in an embodiment of the present application;

[0068] Figure 7 A logical schematic diagram of backing up target data provided in an embodiment of the present application;

[0069] Figure 8 A logical schematic diagram of reading backup data provided in an embodiment of the present application;

[0070] Fig. 9 A schematic diagram of the structure of a data backup device provided in an embodiment of the present application;

[0071] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0072] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0073] It should be understood that the various steps described in the method implementation of the present application can be performed in different orders and / or performed in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0074] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0075] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0076] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0077] Some of the terms used in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

[0078] (1) Distributed copy (DistCP): is a tool for data replication between or within large clusters. MapReduce can be used to implement data distribution, error handling, recovery, and reporting during the replication process. MapReduce is a parallel programming model for processing massive amounts of data and is used for parallel computing of large data sets.

[0079] (2) Availability zone (AZ): It is a collection of one or more physical data centers, usually with independent power supply facilities. The computing, network, storage and other resources are logically divided into multiple clusters within the AZ. In addition, multiple AZs in a region are connected by high-speed optical fibers to meet the needs of users to build high-availability systems across AZs. Users can choose "single AZ" or "multiple AZs" according to their needs.

[0080] (3) Multi-cluster: It can be in the same computer room or in different computer rooms. In actual applications, the latter is often used, that is, distributed in different computer rooms. In this case, if the clusters come from different cloud service providers or clouds of different natures, it is called "multi-cloud".

[0081] (4) Hadoop Distributed File System (HDFS): refers to a distributed file system (DFS) designed to run on general-purpose hardware. HDFS has the characteristics of high fault tolerance and can be deployed on low-cost hardware. In addition, HDFS can also provide high throughput to access application data, so it is also very suitable for applications with very large data sets.

[0082] Based on the above-mentioned nouns and related terminology explanations, the design concept of the embodiments of the present application is briefly introduced below:

[0083] In the current data-driven era, data security and availability are an integral part of business operations. Faced with potential risks such as hardware failures and natural disasters, how to ensure the continued availability and integrity of data has become an urgent problem to be solved. Currently, the above problems can be solved through data backup.

[0084] Exemplarily, existing data backup methods mainly include: asynchronous data backup of multiple computer rooms and multiple clusters, dual-write data backup of multiple computer rooms and multiple clusters, and data backup based on federated deployment of multiple computer rooms.

[0085] Among them, multi-computer room-multi-cluster data asynchronous backup usually refers to synchronizing the data of one cluster (such as the main cluster) to multiple other clusters (such as backup clusters). In this way, when the main cluster fails, any backup cluster can quickly take over. Optionally, the aforementioned data synchronization can be performed using tools such as DistCP.

[0086] Multi-computer room-multi-cluster data double-write backup refers to the transformation of the nodes in the computer room or cluster, and the double-write of data when writing data to achieve synchronous data backup. In this way, when a cluster fails, the backup cluster can quickly take over.

[0087] Data backup based on multi-computer room federation deployment means that each computer room starts one or more master nodes (such as the Name Node in HDFS), and different computer rooms store different data. This method can ensure that when a single computer room fails, only the data of that computer room is lost, and the data of other computer rooms are intact.

[0088] Although the above data backup methods can improve data security, they also increase the cost of data management. All of the above solutions deploy multiple clusters in multiple computer rooms, which means more storage space to store backup data. Adding the bandwidth cost of communication between computer rooms, the hardware cost will increase several times. In addition, maintaining and managing multiple clusters also requires more resources and manpower. In other words, the industry's existing traditional disaster recovery solutions or data backup methods usually rely on high hardware costs, a large amount of bandwidth resources and complex operation and maintenance processes.

[0089] Moreover, in the above-mentioned asynchronous data backup of multiple computer rooms and multiple clusters, data backup is performed asynchronously. If a computer room fails suddenly, the data in the backup computer room can be used, but the part that has not been copied in the future will still be missing, that is, there is a delay in data backup between different computer rooms; in the above-mentioned dual-write data backup of multiple computer rooms and multiple clusters, all nodes need to be deeply modified to ensure that data can be written in duplicate during write operations. Once the coverage is not comprehensive, it will cause data loss in the backup computer room; in the above-mentioned data backup of the multi-computer room federation deployment, it is impossible to ensure the integrity of all data when the computer room fails, and data is still lost. That is, the data backup is still incomplete.

[0090] In view of this, in order to solve or improve the above-mentioned problems, an embodiment of the present application proposes a data backup method, which may specifically include: in response to a data write request for target data, obtaining node information corresponding to each node set in the data backup system; wherein each node set includes a first node and a second node, the first node is used for data sharing between a working master node and multiple backup master nodes in the data backup system, and the second node is used to determine a new working master node from multiple backup master nodes when a working master node fails; based on the obtained information of each node, two target slave node sets that meet the first node quantity requirement and the second node quantity requirement are screened out from multiple candidate slave node sets included in each node set; wherein the multiple slave nodes included in each candidate slave node set are used to back up and store data written to the working master node; the target data is backed up through two target slave node sets, and the storage locations corresponding to the multiple backup data are recorded; wherein each storage location is used to indicate the slave node storing the corresponding backup data.

[0091] In this way, only two target slave node sets that meet the first node quantity requirement and the second node quantity requirement can be used to complete the backup storage of the target data while writing the target data. The target data can be backed up using the minimum slave node set. Under the premise of achieving complete disaster recovery of data, the hardware cost and operation and maintenance cost can be reduced, that is, the data management cost required for backup data is reduced. In addition, since the node set includes a first node for data sharing between a working master node and multiple backup master nodes in a data backup system, and a second node for determining a new working master node from multiple backup master nodes when a working master node fails, in this way, not only the availability and integrity of the target data are ensured, but also the impact of data loss due to hardware failures, natural disasters, etc. is minimized.

[0092] In particular, the preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application, and the embodiments of the present application and the features in the embodiments may be combined with each other if there is no conflict.

[0093] See also Figure 1 As shown, it is a schematic diagram of the system architecture of a data backup system applicable to an embodiment of the present application, and the data backup system may include: node cluster 101, node cluster 102 and node cluster 103. Nodes in different node clusters or different nodes in the same node cluster may exchange information through a communication network, wherein the communication mode adopted by the aforementioned communication network may include: wireless communication mode and wired communication mode.

[0094] Exemplarily, the node cluster 101 can access the network through cellular mobile communication technology to communicate with the node cluster 102. The aforementioned cellular mobile communication technology, for example, includes the fifth generation mobile communication (5th generation mobile networks, 5G) technology or the next generation mobile communication technology.

[0095] Optionally, the node cluster 101 may access the network through a short-range wireless communication method to communicate with the node cluster 102. The short-range wireless communication method, for example, includes wireless fidelity (Wi-Fi) technology.

[0096] The embodiment of the present application does not impose any restriction on the number of communication devices involved in the above system architecture. For example, the above system architecture may include more node clusters, or may include fewer node clusters. Figure 1 As shown, only node cluster 101, node cluster 102 and node cluster 103 are described as examples, and the node clusters are briefly introduced below.

[0097] See also Figure 2 As shown, each node set (i.e., node cluster 101, node cluster 102, or node cluster 103) may include a first node and a second node. Among them, the first node can be used for data sharing between a working master node and multiple backup master nodes in a data backup system, and the second node can be used to determine a new working master node from multiple backup master nodes when a working master node fails. In this way, it can be ensured that the data backup system can still perform data backup when a failure or anomaly occurs. It should be noted that in the embodiment of the present application, there is no specific limitation on the number of first nodes and second nodes included in each node set, and the number of first nodes and second nodes included in different node sets may be the same or different. As Figure 2As shown, node cluster 101 and node cluster 102 each include two first nodes and two second nodes, while node cluster 103 includes only one first node and one second node.

[0098] Optionally, the above node set may also include at least one master node and at least one slave node, such as Figure 2 Of course, the above node set may also not include the master node and the slave node, such as Figure 2 In the node cluster 103. In addition, the master node can also be divided into a working master node and a standby master node. When a problem or failure occurs in the working master node, the standby master node needs to be switched to the working master node to provide services for the client or user.

[0099] Take the node set in the data backup system as the node set in AZ (or computer room) in HDFS as an example. The first node mentioned above can be the Journal Node in HDFS, the second node mentioned above can be the Zookeeper Node, the working master node mentioned above can be the Active Name Node (or the master Name Node), the standby master node mentioned above can be the Standby Name Node (or the standby Name Node), and the slave node mentioned above can be a data node (Data Node).

[0100] Among them, the Journal Node is used to synchronize metadata information between multiple Name Nodes to ensure that the metadata of each Name Node is consistent. When there are two or more Name Nodes in the system, the Journal Node will be responsible for recording metadata changes (such as file creation, deletion, modification, etc.) in one or more persistent storage systems (such as local file systems, HDFS, or network file systems). These change records are called "edit logs". Then, other Name Nodes can read these edit logs from the Journal Node and apply these edit log changes to their own metadata storage, thereby ensuring that the metadata of all Name Nodes is synchronized. Zookeeper Node can generate a leader, namely the Active Name Node, from multiple Name nodes based on a preset working master node selection algorithm (such as the Fast Paxos algorithm) to ensure the consistency and reliability of the data backup system (i.e., HDFS).

[0101] Name Node is mainly responsible for managing HDFS, for example, it can include Name Space (that is, directory structure) management and block (Block) management (which can include the correspondence between Block and Filename (that is, the name of the replica data or backup data) and Data Node List). Data Node is mainly used to store data or files that need to be written. For example, HDFS divides a file into several Blocks and stores them on Data Node. In addition, Data Node can also be responsible for reading and writing underlying files (that is, data reading and writing). Since the file is first divided into multiple Blocks for storage, if the client initiates a command to read the file on HDFS, then the Name Node will inform the client on which Data Nodes these Block data are stored. After that, the client will directly interact with the Data Node to read the data.

[0102] For example, see Figure 3 As shown, it is a structural diagram of an HDFS cluster deployment provided in an embodiment of the present application. In the HDFS cluster, a Name Service contains three Name Nodes, which are distributed in two AZs (i.e. computer rooms) with a distribution ratio of AZ1:AZ2=2:1, and the working mode is 1Active+2Standby. The survival of any Name Node can ensure the provision of external services.

[0103] 6 Data Nodes can be distributed in 2 AZs, with a distribution ratio of AZ1:AZ2=1:1. The number of Data Nodes is determined by the amount of stored data. For example, if a block stores 4 copies (i.e. 4 backup data), 2 Data Nodes in each AZ can store 2 copies, and the storage ratio of the two AZs can be 1:1 to ensure storage balance.

[0104] The Zookeeper cluster consists of 5 Zookeeper Nodes, which are distributed in 3 AZs in a distribution ratio of AZ1:AZ2:AZ3=2:2:1. Deployment across 3 AZs ensures that when a single AZ fails, most ZookeeperNodes survive and complete the master election of the HDFS cluster, thereby ensuring the integrity of data backup.

[0105] The Journal Node cluster consists of 5 Journal Nodes, which are distributed in 3 AZs with a distribution ratio of AZ1:AZ2:AZ3=2:2:1. Deployment across 3 AZs ensures that when a single AZ fails, most Journal Nodes will still survive and complete the master election of the HDFS cluster, thereby ensuring the integrity of data backup.

[0106] It should be noted that the data backup system in the embodiment of the present application can also be understood as a distributed file system or a distributed storage system, and the specific type of the data backup system is not specifically limited.

[0107] The data backup method provided by the exemplary embodiment of the present application is described below in combination with the above-mentioned system architecture and with reference to the accompanying drawings. It should be noted that the above-mentioned system architecture is only shown to facilitate understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this regard.

[0108] See also Figure 4 As shown, it is a schematic diagram of an implementation process of a data backup method provided in an embodiment of the present application. The execution subject takes a data backup system as an example. The specific implementation process of the method is as follows:

[0109] S401: In response to a data write request for target data, node information corresponding to each node set in the data backup system is obtained.

[0110] Specifically, when executing step S401, the user can send a data write request for the target data to the working master node in the data backup system (for example, HDFS) through a terminal device or a client. At this time, the data backup system can obtain the node information corresponding to each node set in the aforementioned data backup system. Exemplarily, each node set corresponds to a computer room or an AZ in the area involved in the aforementioned data backup system. Each node information may include information such as the remaining storage space of the slave nodes in the corresponding node set and the number of slave nodes that can be used for data backup.

[0111] Taking the data backup system as HDFS as an example, assuming that HDFS includes 5 AZs, and the 5 AZs correspond to 5 node sets respectively. For example, the nodes included in each of the above 5 node sets are shown in Table 1:

[0112] Table 1 Examples of nodes included in different node sets

[0113] Node Collection Node.set1 Node.set2 Node.set3 Node.set4 Node.set5 AZ AZ1 AZ2 AZ3 AZ4 AZ5 Number of Name Nodes 2 1 0 3 2 Number of Journal Nodes 2 2 1 3 2 Number of Zookeeper Nodes 2 2 1 3 2 Number of Data Nodes 3 3 0 4 3

[0114] Based on the node conditions of each of the five node sets recorded in the above table, after receiving a data write request for the target data, HDFS can obtain the relevant information of the Name Node corresponding to the five AZs, that is, the remaining storage space and whether it can be used for backup storage of the target data.

[0115] Exemplarily, the three Name Nodes in AZ1 are Name Node_1, Name Node_2 and Name Node_3, and the remaining storage spaces corresponding to the three Name Nodes are 2.7GB, 986MB and 1.37GB respectively. NameNode_1 can be used for backup storage of target data, Name Node_2 cannot be used for backup storage of target data, and Name Node_3 can be used for backup storage of target data.

[0116] Optionally, whether the above-mentioned Name Node can be used to back up and store the target data can be determined according to the data type of the data to be backed up and stored, that is, different Name Nodes can be used to back up and store different types of data. Of course, whether the above-mentioned Name Node can be used to back up and store the target data can also be determined according to whether the remaining storage space corresponding to the Name Node is greater than a set remaining storage space threshold (e.g., the storage space required for the data to be backed up and stored). The embodiments of the present application do not specifically limit this.

[0117] S402: Based on the obtained information of each node, two target slave node sets that meet the first node quantity requirement and the second node quantity requirement are screened out from a plurality of candidate slave node sets included in each node set.

[0118] The multiple slave nodes included in each candidate slave node set can be used to back up and store data written to the working master node (or written to the data backup system).

[0119] In an alternative implementation, see Figure 5As shown, the data backup system can select three node sets (such as node set 1, node set 2 and node set n) including the local node set where the working master node is located from each node set based on the first node number and the second node number corresponding to each node set (such as n node sets) determined by multiple node information; then, two target slave node sets can be screened out from at least two candidate slave node sets included in the three node sets. Among them, the total number of first nodes and the total number of second nodes of any two node sets in the aforementioned three node sets are respectively greater than half of the total number of first nodes and half of the total number of second nodes of the three node sets. In this way, it can be ensured that when the node set where the working master node in the data backup system is located fails or anomalies occur, more than half of the first nodes and second nodes in the data backup system are still alive (that is, they can work normally), thereby ensuring that the data backup system can still normally support data backup and reading of backup data.

[0120] Take the three node sets as AZ1, AZ2 and AZ3 in Table 1 as an example. It can be seen that the sum of the number of first nodes in any two AZs among AZ1, AZ2 and AZ3 is greater than half of the total number of first nodes in these three AZs, and the sum of the number of second nodes in any two AZs is greater than half of the total number of second nodes in these three AZs.

[0121] For example, the nodes in the data backup system are still taken as nodes in HDFS. Fig. 6A As shown in the figure, assuming that the Active Name Node is the Name Node in AZ1, and AZ1 fails, since more than half of the nodes in the Zookeeper cluster and the JournalNode cluster are alive, the integrity and availability of the data backup system can be guaranteed, and a new Active Name Node can be reselected. For example, the surviving Name Node in AZ2 is used as the new Active Name Node. In this way, HDFS can still provide services to the outside world normally. In addition, since there are two copies of the written data (i.e., backup data) in AZ2, for HDFS, the backup data of the written data is not completely lost.

[0122] It should be noted that Fig. 6A Replica 1 to Replica 4 are four backup data of the same data (eg, data A), and Replica 1' to Replica 4' are four replica data of another data (eg, data B).

[0123] For another example, see Figure 6BAs shown in the figure, assuming that the Active Name Node is the Name Node in AZ2, and AZ2 fails, since more than half of the nodes in the Zookeeper cluster and the Journal Node cluster are alive, the integrity and availability of the data backup system can be guaranteed, and a new Active Name Node can be reselected. For example, a surviving Name Node in AZ1 is used as the new Active Name Node, and the other Name Node is used as the Standby Name Node. In this way, HDFS can still provide services to the outside world normally. In addition, since there are two copies of the written data (i.e., backup data) in AZ1, for HDFS, the backup data of the written data is not completely lost.

[0124] See Figure 6C As shown in the figure, if AZ3 fails, since more than half of the nodes in the Zookeeper cluster and the Journal Node cluster survive, the data backup system can be guaranteed to be complete and available. In addition, since the three Name Nodes in AZ1 and AZ2 are all alive (i.e., 1 Active Name Node + 2 Standby Name Nodes), there is no need to reselect a new Active Name Node. In addition, since there are two copies of the written data (i.e., backup data) in both AZ1 and AZ2, there is no loss of backup data of the written data for HDFS.

[0125] In order to improve the efficiency of selecting the target slave node set and reduce the resource overhead of the data backup system when backing up the target data, if the local node set includes slave nodes (i.e., the candidate slave node set), the data backup system can use the candidate slave node set included in the local node set as the first target slave node set; and use the candidate slave node set included in any one of the above three node sets except the local node set as the second target slave node set.

[0126] In this way, the candidate slave node set in the local node set where the working master node providing services to the client or user is located is directly used as a candidate slave node set for backing up the target data. This not only reduces the traffic bandwidth cost brought by cross-AZ, but also improves the efficiency of data operations (such as backup or reading).

[0127] Optionally, if the local node set does not include slave nodes, the data backup system may use candidate slave node sets respectively included in two node sets of the three node sets except the local node set as two target slave node sets for backing up target data.

[0128] Based on the above method, dual AZ data storage ensures that when any backup data fails (i.e., a disaster) or anomaly occurs, there is still backup data in the data backup system, thereby further improving the reliability and availability of data.

[0129] S403: Back up the target data through two target slave node sets, and record storage locations corresponding to the multiple backup data.

[0130] Each storage location may be used to indicate a slave node storing corresponding backup data. Exemplarily, the aforementioned storage location may be a physical memory address of the backup data in the corresponding slave node.

[0131] In an optional implementation, when executing step S403, the data backup system can perform the following operations for any one of the above two target slave node sets, such as the first target slave node set: obtain the remaining resource amount of each of the multiple slave nodes included in the first target slave node set, and based on the obtained multiple remaining resource amounts, prioritize the data backup of the multiple slave nodes to obtain corresponding priority sorting results; then, based on the aforementioned priority sorting results, select slave nodes that meet the preset number from the multiple slave nodes; finally, back up the target data respectively through the preset number of slave nodes. In this way, when backing up the target data, the preset number of slave nodes are selected in combination with the remaining resource amount (i.e., the remaining storage space) of each slave node in the target slave node set, and the slave nodes with more remaining resource amounts can be preferentially selected to back up the target data, thereby achieving balanced use of the slave nodes.

[0132] Exemplarily, assume that the first target slave node set includes four slave nodes (e.g., Slave Node1, Slave Node2, Slave Node3, and Slave Node4), and the resource surplus corresponding to the above four slave nodes may be: 2.28GB, 4.23GB, 1.54GB, and 3.75GB, respectively. The data backup system may prioritize the data backup of the above four slave nodes in such a way that the larger the resource surplus, the higher the priority ranking. Then the priority ranking results of the above four slave nodes are, from high to low, Slave Node2, Slave Node4, Slave Node1, and Slave Node3.

[0133] Further, if the above preset number is 2, after obtaining the above priority sorting result, the data backup system can select Slave Node2 and SlaveNode4 from the above 4 slave nodes according to the above priority sorting result, so as to back up the target data through Slave Node2 and Slave Node4 respectively.

[0134] In another optional implementation, the data backup system may also prioritize the data backup of the plurality of slave nodes included in the first target slave node set according to the resource utilization rates (i.e., storage space utilization rates) corresponding to the plurality of slave nodes respectively included in the first target slave node set, and obtain corresponding priority sorting results; then, based on the aforementioned priority sorting results, select at least one slave node with a lower resource utilization rate and meeting the aforementioned preset number from the plurality of slave nodes; finally, back up the target data through the aforementioned at least one slave node, thereby improving the resource utilization rate of the slave nodes.

[0135] Still taking the 4 slave nodes included in the above-mentioned first target slave node set as an example. Assume that the resource utilization rates corresponding to Slave Node1, SlaveNode2, Slave Node3 and Slave Node4 are respectively 25.8%, 42.5%, 37.1% and 51.7%. The data backup system can prioritize the data backup of the above-mentioned 4 slave nodes in a way that the lower the resource utilization rate, the higher the priority ranking. Then the priority ranking results of the above-mentioned 4 slave nodes are from high to low: Slave Node1, Slave Node3, Slave Node2 and Slave Node4. Therefore, the data backup system can use SlaveNode1 and Slave Node3 as slave nodes for data backup of the target data.

[0136] Based on the data backup method described in steps S401 to S403 above, refer to Figure 7As shown, it is a logical schematic diagram of data backup for target data provided by an embodiment of the present application. After client 1 sends a data write request for Block A (i.e., target data 1) to the Active NameNode in HDFS, the Active Name Node can obtain the Data Node information in AZ1 and AZ2, and select Data Node1 and Data Node2 in AZ1, and Data Node4 and Data Node6 in AZ2 as Data Nodes for backing up Block A. Among them, Data Node1 and Data Node2 are Data Nodes in the AZ (i.e., AZ1) where client 1 is located. Similarly, after client 2 sends a data write request for Block B (i.e., target data 2) to the Active NameNode in HDFS, the Active Name Node can obtain the Data Node information in AZ1 and AZ2, and select Data Node2 and Data Node3 in AZ1, and Data Node4 and Data Node6 in AZ2 as Data Nodes for backing up Block B. Among them, Data Node4 and Data Node6 are Data Nodes in the AZ (i.e., AZ2) where client 2 is located.

[0137] It can be seen that when a client initiates a data write request, HDFS first identifies the AZ (i.e., computer room or data center) where the client is located. Then, based on the computer room where the client is located, HDFS can select one or more AZs to store the replica data according to the replica placement strategy (i.e., data backup strategy). Exemplarily, this decision-making process ensures that 2 replicas of data are placed or stored in the AZ where the client is located, so that clients in the AZ can directly and efficiently read the replica data. Therefore, after selecting one or more AZs, HDFS can distribute the replica data to the Data Nodes of these AZs, and create the required number of replica data according to the aforementioned replica placement strategy. Furthermore, after the target data is written successfully (i.e., the target data backup is completed), HDFS will update the metadata and record the location information of the replica data so that subsequent data read requests can accurately locate the corresponding replica data.

[0138] The above replica placement strategy fully considers the geographical location and network topology of AZ. When the target data is written to HDFS, it ensures that each AZ (or computer room) stores an average of 2 copies of the data. This design is intended to provide higher data reliability and reading efficiency, especially in the face of local failures or network partitions. Even if an AZ fails, the replica data in other AZs can still ensure data availability and service continuity. At the same time, since each AZ stores at least one copy of the data, the client can directly obtain it from the Data Node of the local AZ when reading the replica data, reducing the data transmission delay across AZs and improving the reading efficiency.

[0139] The above network topology can be a three-layer structure of <computer room, rack, machine> to achieve the reading and writing of target data. Among them, each computer room has an independent network infrastructure and power supply, and the computer rooms are connected by high-speed network to ensure the stability and speed of data transmission. The rack is a physical device unit in the computer room, usually containing multiple servers. The servers inside the rack are connected to the high-speed network through switches. The machine is the actual physical device that stores and processes data. Each machine is connected to the switch in the rack and communicates with other machines and computer rooms through the switch.

[0140] In an optional implementation, after executing step S403, the data backup system can also obtain the operating status of the local node set in response to a data read request for the target data. The aforementioned operating status may include a normal state and an abnormal state (i.e., a failure occurs). Optionally, if the aforementioned local node set includes a target slave node set, and the operating status indicates that the local node set is still in a normal state, the data backup system can read the backup data from the corresponding slave node based on the storage location of the backup data associated with a target slave node set. In this way, when the client or user reads the target data, they can read the backup data of the target data without crossing AZ.

[0141] If the above-mentioned local node set includes a target slave node set, but the operating status indicates that the local node set is in an abnormal state, the data backup system can read the backup data from the corresponding slave node based on the storage location of the backup data associated with the other target slave node set in the two target slave node sets.

[0142] Of course, if the above local node set does not include the target slave node set, the data backup system can read the backup data of the target data from the slave nodes in other node sets in a cross-AZ manner.

[0143] See also Figure 8As shown, it is a logical schematic diagram of reading backup data provided by an embodiment of the present application. In HDFS, when multiple clients need to read target data (such as Block A or Block B), HDFS will give priority to guiding the client to read the replica data stored in the local node cluster (ie, the local AZ). In this way, while optimizing network traffic, a large amount of cross-AZ read traffic is avoided, thereby significantly improving read performance and reducing network latency. In addition, by storing replica data in multiple AZs, HDFS can ensure that when the local AZ fails, the client can still read replica data from other AZs, but the read performance in this case is usually lower than reading replica data from the local AZ. Therefore, giving priority to reading replica data in the local AZ not only improves the user experience, but also effectively reduces network costs and maintenance complexity.

[0144] In summary, in the data backup method provided in the embodiment of the present application, for the target data, through the deployment of a three-node cluster and the backup storage of two slave node sets, a highly reliable, low-cost, and efficient data backup is achieved, the availability and integrity of the target data are ensured, and the impact of data loss due to hardware failures, natural disasters, etc. is minimized. Due to the existence of the first node and the second node, it can be ensured that the data backup system can provide services continuously and stably, that is, when any node set (i.e., a computer room or AZ) fails, the target data can still be backed up and read in the other two node sets. The data storage of the dual slave node set can ensure that when part of the backup data still survives a disaster, the reliability and availability of the data are further improved. Moreover, in order to reduce the traffic across the node set and improve the read and write efficiency of the data, the read and write operations are preferentially performed from the local node set, which reduces the traffic bandwidth cost across the node cluster and improves the read and write efficiency of the data.

[0145] Further, based on the same technical concept, the embodiment of the present application provides a data backup device, which can be applied to the above data backup system, and the data backup device is used to implement the above method flow of the embodiment of the present application. Fig. 9 As shown, the data backup device 900 may include: an information acquisition module 901, a collection screening module 902, a data backup module 903 and a data reading module 904, wherein:

[0146] The information acquisition module 901 is used to obtain the node information corresponding to each node set in the data backup system in response to the data write request for the target data; wherein each node set includes a first node and a second node, the first node is used for data sharing between the working master node and multiple backup master nodes in the data backup system, and the second node is used for determining a new working master node from multiple backup master nodes when the working master node fails;

[0147] A set screening module 902 is used to screen out two target slave node sets that meet the first node quantity requirement and the second node quantity requirement from multiple candidate slave node sets included in each node set based on the obtained information of each node; wherein the multiple slave nodes included in each candidate slave node set are used to back up and store the data written to the working master node;

[0148] The data backup module 903 is used to back up the target data through two target slave node sets, and record storage locations corresponding to multiple backup data respectively; wherein each storage location is used to indicate a slave node storing the corresponding backup data.

[0149] In an optional embodiment, when two target slave node sets that meet the first node quantity requirement and the second node quantity requirement are screened out from a plurality of candidate slave node sets included in each node set based on the obtained information of each node, the set screening module 902 is specifically used to:

[0150] Based on the first node quantity and the second node quantity corresponding to each node set respectively determined by the multiple node information, three node sets including the local node set where the working master node is located are selected from each node set; the total number of first nodes and the total number of second nodes of any two node sets in the three node sets are respectively greater than half of the total number of first nodes and half of the total number of second nodes of the three node sets;

[0151] Two target slave node sets are screened out from at least two candidate slave node sets included in the three node sets.

[0152] In an optional embodiment, when two target slave node sets are screened out from at least two candidate slave node sets included in the three node sets, the set screening module 902 is specifically configured to:

[0153] If the local node set includes slave nodes, the candidate slave node set included in the local node set is used as the first target slave node set;

[0154] And, a candidate slave node set included in any one of the three node sets except the local node set is used as the second target slave node set.

[0155] In an optional embodiment, when two target slave node sets are screened out from at least two candidate slave node sets included in the three node sets, the set screening module 902 is specifically configured to:

[0156] If the local node set does not include slave nodes, candidate slave node sets respectively included in two node sets of the three node sets except the local node set are used as two target slave node sets.

[0157] In an optional embodiment, when backing up the target data through two target slave node sets, the data backup module 903 is specifically used to:

[0158] For the two target slave node sets, perform the following operations respectively:

[0159] Obtaining the remaining amount of resources of each of the plurality of slave nodes included in a target slave node set, and based on the obtained plurality of remaining amounts of resources, performing data backup priority sorting on the plurality of slave nodes to obtain a corresponding priority sorting result;

[0160] Based on the priority sorting result, select slave nodes that meet the preset number from multiple slave nodes;

[0161] The target data is backed up separately through a preset number of slave nodes.

[0162] In an optional embodiment, after recording the storage locations corresponding to the plurality of backup data, the data backup device 900 further includes a data reading module 904, which is specifically used to:

[0163] In response to a data read request for target data, obtaining an operating status of a local node set;

[0164] If the local node set includes a target slave node set, and the operation status indicates that the local node set is still in a normal state, the backup data is read from the corresponding slave node based on the storage location of the backup data associated with the target slave node set.

[0165] In an optional embodiment, after acquiring the running status of the local node set, the data reading module 904 is further used to:

[0166] If the local node set includes a target slave node set, but the operating status indicates that the local node set is in an abnormal state, the backup data is read from the corresponding slave node based on the storage location of the backup data associated with the other target slave node set of the two target slave node sets.

[0167] Based on the description of the above method embodiment and device embodiment, the exemplary embodiment of the present invention further provides an electronic device, including: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program that can be executed by the at least one processor, and the computer program is used to enable the electronic device to perform the method according to the embodiment of the present invention when executed by the at least one processor.

[0168] An embodiment of the present application also provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute a method according to an embodiment of the present application.

[0169] An embodiment of the present application also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute a method according to an embodiment of the present application.

[0170] See also Fig.10 As shown, the structured block diagram of the electronic device 1000 that can be used as the server or client of the present application will now be described, which is an example of the hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent the computer device of various forms of digital electronics, such as, laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only used as examples, and are not intended to limit the implementation of the present application described herein and / or required.

[0171] like Fig.10 As shown, the electronic device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 to a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the device 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0172] A plurality of components in the electronic device 1000 are connected to the I / O interface 1005, including: an input unit 1006, an output unit 1007, a storage unit 1008, and a communication unit 1009. The input unit 1006 may be any type of device capable of inputting information to the electronic device 1000, and the input unit 1006 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 1007 may be any type of device capable of presenting information, and may include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1008 may include but is not limited to a disk, an optical disk. The communication unit 1009 allows the electronic device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a worldwide interoperability for microwave access (WiMax) device, a cellular communication device, and / or the like.

[0173] The computing unit 1001 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSP), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1001 performs the various methods and processes described above. For example, in some embodiments, the above-mentioned data backup method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1000 via the ROM 1002 and / or the communication unit 1009. In some embodiments, the computing unit 1001 may be configured to perform the above-mentioned data backup method by any other appropriate means (e.g., by means of firmware).

[0174] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0175] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM) or a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0176] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0177] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tub (CRT) or a liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0178] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0179] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0180] Furthermore, it should be understood that what is disclosed above is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope covered by the present application.

Claims

1. A data backup method, characterized in that: Applied to data backup systems, including: In response to a data write request for target data, node information corresponding to each node set in the data backup system is obtained; wherein each node set includes a first node and a second node, the first node is used for data sharing between a working master node and a plurality of backup master nodes in the data backup system, and the second node is used for determining a new working master node from the plurality of backup master nodes when a failure occurs in the working master node; Based on the obtained information of each node, two target slave node sets that meet the first node quantity requirement and the second node quantity requirement are screened out from the multiple candidate slave node sets included in the each node set; wherein the multiple slave nodes included in each candidate slave node set are used to back up and store the data written to the working master node; The target data is backed up by the two target slave node sets, and storage locations corresponding to a plurality of backup data are recorded; wherein each storage location is used to indicate a slave node storing corresponding backup data.

2. The method according to claim 1, characterized in that The method of selecting two target slave node sets that meet the first node quantity requirement and the second node quantity requirement from a plurality of candidate slave node sets included in each node set based on the obtained information of each node comprises: Based on the first node quantity and the second node quantity respectively corresponding to each node set determined by the multiple node information, three node sets including the local node set where the working master node is located are selected from the various node sets; the total number of first nodes and the total number of second nodes of any two node sets among the three node sets are respectively greater than half of the total number of first nodes and half of the total number of second nodes of the three node sets; The two target slave node sets are screened out from at least two candidate slave node sets included in the three node sets.

3. The method according to claim 2, characterized in that The step of selecting the two target slave node sets from at least two candidate slave node sets included in the three node sets includes: If the local node set includes slave nodes, taking the candidate slave node set included in the local node set as the first target slave node set; And, a candidate slave node set included in any one of the three node sets except the local node set is used as the second target slave node set.

4. The method according to claim 2, characterized in that The step of selecting the two target slave node sets from at least two candidate slave node sets included in the three node sets includes: If the local node set does not include slave nodes, candidate slave node sets respectively included in two node sets among the three node sets except the local node set are used as the two target slave node sets.

5. The method according to claim 1, characterized in that The backing up the target data by the two target slave node sets includes: For the two target slave node sets, the following operations are performed respectively: Obtaining the remaining amount of resources of each of the plurality of slave nodes included in a target slave node set, and based on the obtained plurality of remaining amounts of resources, performing data backup priority sorting on the plurality of slave nodes to obtain a corresponding priority sorting result; Based on the priority sorting result, selecting slave nodes that meet a preset number from the plurality of slave nodes; The target data is backed up respectively by the preset number of slave nodes.

6. The method according to any one of claims 1 to 5, characterized in that After recording the storage locations corresponding to the plurality of backup data respectively, the method further includes: In response to a data read request for the target data, acquiring an operating status of the local node set; If the local node set includes a target slave node set, and the operating status indicates that the local node set is still in a normal state, the backup data is read from the corresponding slave node based on the storage location of the backup data associated with the target slave node set.

7. The method according to claim 6, characterized in that After obtaining the running status of the local node set, the method further includes: If the local node set includes the target slave node set, but the operating status indicates that the local node set is in an abnormal state, the backup data is read from the corresponding slave node based on the storage location of the backup data associated with the other target slave node set of the two target slave node sets.

8. A data backup device, characterized in that: Applied to data backup systems, including: An information acquisition module, configured to obtain node information corresponding to each node set in the data backup system in response to a data write request for target data; wherein each node set includes a first node and a second node, the first node is used for data sharing between a working master node and a plurality of backup master nodes in the data backup system, and the second node is used for determining a new working master node from the plurality of backup master nodes when a failure occurs in the working master node; A set screening module, used for screening out two target slave node sets that meet the first node quantity requirement and the second node quantity requirement from the multiple candidate slave node sets included in the respective node sets based on the obtained information of the respective nodes; wherein the multiple slave nodes included in each candidate slave node set are used for backing up and storing the data written to the working master node; The data backup module is used to back up the target data through the two target slave node sets and record storage locations corresponding to multiple backup data respectively; wherein each storage location is used to indicate a slave node storing corresponding backup data.

9. An electronic device, comprising: processor; as well as Memory for storing programs, The program includes instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

Images