Processing method for data backup in cluster and cluster

Abstract

The invention discloses a processing method for data backup in a cluster and the cluster. The method comprises the steps that each edge node in the cluster sends node data to a first node in a node pool to which the edge node belongs; each first node in the cluster sends node backup data to a central node in the cluster; wherein edge nodes of the cluster are divided into a plurality of node pools, the first node is deployed in each node pool, and node backup data sent by the first node comprises node data of each edge node in the node pool to which the first node belongs; and the center node stores the node backup data.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method for processing data backup in a cluster and a cluster. Background Art

[0002] In a network architecture with a central node and multiple edge nodes, each edge node usually backs up node data to the central node. However, this will put great pressure on the network bandwidth of the central node. Summary of the invention

[0003] In view of this, the present application provides a method for processing data backup in a cluster and a cluster, as follows:

[0004] A method for processing data backup in a cluster, the method comprising:

[0005] Each edge node in the cluster sends node data to a first node in a node pool to which the edge node belongs;

[0006] Each of the first nodes in the cluster sends node backup data to a central node in the cluster;

[0007] The edge nodes of the cluster are divided into a plurality of node pools, each of which is deployed with the first node, and the node backup data sent by the first node includes node data of each edge node in the node pool to which the first node belongs;

[0008] The central node stores the node backup data.

[0009] In the above method, preferably, the edge node sends the node data to the first node according to a first period, and the first node sends the node backup data to the central node according to a second period;

[0010] The method further comprises:

[0011] The first node sends the node backup data to a second node in the node pool to which the first node belongs according to a third period;

[0012] The second period is greater than the third period, and the third period is greater than the first period.

[0013] In the above method, preferably, the first node is a node that satisfies the first condition among the backup nodes in the node pool to which it belongs; and the second node is a node that satisfies the second condition among the backup nodes in the node pool to which it belongs;

[0014] The first condition is different from the second condition.

[0015] In the above method, preferably, the first condition includes: the first score value of the backup node with respect to the deployed data processor is sorted in the first place according to the first sorting rule;

[0016] The second condition includes: the second score value of the backup node regarding the system operation resources is sorted first according to the second sorting rule.

[0017] In the above method, preferably, the first score value is at least related to the number of the data processors and the operating environment resource information of the data processors;

[0018] The second scoring value is at least related to system operation resource information of the backup node and the data processor deployed therein.

[0019] In the above method, preferably, the central node receives the node resource information sent by each backup node through a data connection with each backup node and determines the first node and the second node in each node pool according to the node resource information;

[0020] The node resource information includes: quantity resource information of the data processors, operating environment resource information of the data processors, and system operating resource information of the backup node and the data processors deployed thereon.

[0021] In the above method, preferably, a first plug-in is deployed in the central node, a second plug-in is deployed in the backup node, and the data connection is established through the first plug-in and the second plug-in.

[0022] The above method, preferably, further comprises:

[0023] In the event that the edge node fails, switching the processing service of the failed edge node to a first node in the node pool to which the failed edge node belongs, and the first node in the node pool to which the failed edge node belongs serves as the edge node to perform the processing service;

[0024] A node that meets the first condition is selected from backup nodes in the node pool to which the failed edge node belongs as a new first node.

[0025] In the above method, preferably, the edge nodes are divided into corresponding node pools according to the networks in which they are located;

[0026] Among them, edge nodes in the same network or geographically close networks are divided into the same node pool.

[0027] A cluster comprises: a plurality of edge nodes and a central node; the plurality of edge nodes are divided into a plurality of node pools, and a first node is deployed in each of the node pools;

[0028] Among them, each of the edge nodes sends node data to the first node in the node pool to which the edge node belongs; each of the first nodes sends node backup data to the central node, and the node backup data sent by the first node includes the node data of each edge node in the node pool to which the first node belongs; the central node stores the node backup data.

[0029] It can be seen from the above technical solution that in a data backup processing method and cluster provided by the present application, the edge nodes of the cluster can be divided into multiple node pools, each node pool deploys a first node, and the first node sends the node data of each edge node in the node pool to the central node in the cluster. By using the first node as a data transfer node, the number of nodes that interact with the central node can be reduced, thereby reducing the burden on the network bandwidth of the central node. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0031] Figure 1 An interactive flow chart of a method for processing data backup in a cluster provided in an embodiment of the present application;

[0032] Figure 2 This is a schematic diagram of the architecture of the cluster in the embodiment of the present application;

[0033] Figure 3 This is an example diagram of transmitting node backup data to a central node through a first node in an embodiment of the present application;

[0034] Figure 4 This is an example diagram of backing up node data of an edge node to a second node through a first node in an embodiment of the present application;

[0035] Figure 5 This is an example diagram for selecting a first node and a second node in an embodiment of the present application;

[0036] Figure 6 A schematic diagram of the interaction between the backup node and the central node in an embodiment of the present application;

[0037] Figure 7 A schematic diagram of the architecture of a cluster provided in an embodiment of the present application;

[0038] Figure 8Another schematic diagram of the architecture of a cluster provided in an embodiment of the present application;

[0039] Fig. 9 This is a schematic diagram of the structure of an edge node in an embodiment of the present application;

[0040] Fig.10 This is a schematic diagram of the interaction between nodes in the cluster in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0042] refer to Figure 1 As shown, it is an interactive flow chart of a method for processing data backup in a cluster provided by an embodiment of the present application. The method can be applied to a cluster including multiple nodes, wherein the cluster includes multiple edge nodes and a central node. The edge node can be an electronic device capable of providing data computing functions. One or more data processors DPU (Data Processing Unit) can be deployed in the edge node. The central node is an electronic device capable of managing the edge node. Figure 2 As shown in , multiple edge nodes form a cluster, and there is also a central node in the cluster. The central node can at least store the edge backup data of each edge node, and each edge node provides data computing functions through one or more DPUs. The technical solution in this embodiment is mainly used in the process of storing edge backup data in the central node, which causes great pressure on the network bandwidth of the central node.

[0043] Specifically, the method in this embodiment may include the following process:

[0044] Step 101: Each edge node in the cluster sends node data to the first node in the node pool to which the edge node belongs.

[0045] Among them, the edge nodes of the cluster are divided into multiple node pools, such as Figure 3 The node pool a and the node pool b in the cluster are configured as follows. A first node is deployed in each node pool. The edge node in this embodiment refers to a working node in the cluster for executing and processing a large number of services. The cluster is also configured with a node as a backup node.

[0046] In a specific implementation, any node in the node pool, such as the edge node and the first node, is deployed with a plug-in, such as the BKclient plug-in. Based on this, each edge node can send its own node data to the first node in the node pool to which it belongs through the deployed plug-in.

[0047] For example, Figure 3 As shown in , the cluster at least includes: node pool a and node pool b. Node pool a includes at least edge node 1, edge node 2, edge node 3 and first node 4. Node pool b includes at least edge node 5, edge node 6 and first node 7. Plug-ins are deployed in edge node 1, edge node 2, edge node 3 and first node 4 respectively. The plug-ins in edge node 1, edge node 2 and edge node 3 respectively establish data transmission channels c with the plug-in in first node 4. Edge node 1 sends its own node data to first node 4 based on data transmission channel c1 between edge node 1 and first node 4. Edge node 2 sends its own node data to first node 4 based on data transmission channel c2 between edge node 2 and first node 4. Edge node 3 sends its own node data to first node 4 based on data transmission channel c3 between edge node 3 and first node 4. The same is true for node pool b. Plug-ins are deployed in edge node 5, edge node 6 and first node 7 respectively. The plug-ins in edge node 5 and edge node 6 respectively establish data transmission channels d with the plug-in in first node 7. Edge node 5 sends its own node data to first node 7 based on data transmission channel d1 between edge node 5 and first node 7. Edge node 6 sends its own node data to first node 7 based on data transmission channel d2 between edge node 5 and first node 7.

[0048] Step 102: Each first node in the cluster sends node backup data to the central node in the cluster.

[0049] The node backup data sent by the first node includes: node data of each edge node in the node pool to which the first node belongs.

[0050] For example, Figure 3 As shown in , the first node 4 can receive the node data of the edge node 1, the edge node 2 and the edge node 3, based on which, the first node 4 sends the node backup data composed of the node data of the edge node 1, the edge node 2 and the edge node 3 to the central node 8; the first node 7 can receive the node data of the edge node 5 and the edge node 6, based on which, the first node 7 sends the node backup data composed of the node data of the edge node 5 and the edge node 6 to the central node 8. Thus, the central node 8 can receive the node data of each edge node in the cluster through the first node of each node pool, thereby realizing data collection.

[0051] It should be noted that the node data in the node backup data can be used for data recovery after data loss due to node failure or other failures.

[0052] In a specific implementation, a plug-in is deployed on the central node in the cluster. Based on this, the first node of each node pool can send node backup data consisting of node data of each edge node in the node pool collected by itself to the central node through the deployed plug-in.

[0053] For example, Figure 3 As shown in the figure, the respective plug-ins of the first node 7 and the first node 4 establish data transmission channels e with the plug-in in the central node 8 respectively. The first node 7 sends the node backup data collected by itself to the central node 8 based on the data transmission channel e1 between the first node 7 and the central node 8, and the first node 4 sends the node backup data collected by itself to the central node 8 based on the data transmission channel e2 between the first node 7 and the central node 8.

[0054] Step 103: The central node stores the node backup data.

[0055] For example, Figure 3 As shown in , the central node 8 saves the node backup data sent by the first node 4 and the first node 7 respectively, so as to realize the backup of the node data of each edge node. In this process, the nodes that interact with the central node 8 for data are only the first nodes in each node pool, such as the first node 4 and the first node 7, and the central node 8 does not need to interact with each edge node for data respectively. Therefore, the number of nodes that interact with the central node 8 for data is reduced from 5 to 2. It can be seen that the number of nodes that interact with the central node for data can be significantly reduced in this embodiment, so that the burden of the network bandwidth on the central node 8 is reduced.

[0056] It can be seen from the above scheme that in a method for processing data backup in a cluster provided by an embodiment of the present application, the edge nodes of the cluster can be divided into multiple node pools, and a first node is deployed in each node pool. The first node sends the node data of each edge node in the node pool to which it belongs to the central node in the cluster. By using the first node as a data transfer node, the number of nodes that interact with the central node can be reduced, thereby reducing the burden on the network bandwidth of the central node.

[0057] In one implementation, edge nodes may be divided into corresponding node pools according to the networks they are located in. Edge nodes in the same network or networks in close geographical locations are divided into the same node pool.

[0058] The geographical proximity can be understood as: the geographical location is less than or equal to the first threshold. For example, the geographically close network can be a metropolitan area network. The same network can be understood as: a network corresponding to the same gateway.

[0059] Specifically, in this embodiment, the edge nodes in the cluster can be divided into node pools according to the regions where the edge nodes are located. For example, the edge nodes belonging to the same gateway are divided into the same node pool. If the number of edge nodes in the node pool is less than or equal to the second threshold, then the edge nodes corresponding to the gateways whose geographical locations are less than or equal to the first threshold can be further divided into the same node pool to avoid too many node pools resulting in too many nodes of the first nodes that interact with the central node for data, thereby avoiding a heavy burden on the network bandwidth of the central node; if the number of edge nodes in the node pool is greater than or equal to the third threshold, then the node pool can be divided into two or more node pools to avoid too many edge nodes in the node pool resulting in a heavy burden on the network bandwidth of the first nodes in the node pool.

[0060] For example, Figure 3 As shown in , edge node 1, edge node 2 and edge node 3 are divided into node pool a because they are in the same local area network, while edge node 5 and edge node 6 are divided into node pool b because they are in the same metropolitan area network.

[0061] In one implementation, the edge node may send node data to a first node in a node pool to which the edge node belongs according to a first period, and the first node may send node backup data to a central node in the cluster according to a second period.

[0062] Based on this, the first node in each node pool can also send node backup data to the second node in the node pool to which the first node belongs according to the third period. The second node can be understood as: a node that backs up the first node. The second node can be selected from the backup nodes in the node pool, and the backup nodes in the node pool are pre-deployed nodes in the cluster.

[0063] The second period is greater than the third period, and the third period is greater than the first period.

[0064] That is to say, each edge node sends node data to the first node in the node pool to which the edge node belongs according to the shortest first cycle T1, i.e., the maximum frequency, and the first node collects the node data of each edge node in the node pool, i.e., the node backup data; and the first node in each node pool sends the node backup data collected by the first node to the second node in the node pool according to the longer third cycle T3, i.e., the lower frequency, and the second node backs up the node data of all edge nodes in the node pool to which the first node belongs; at the same time, the first node in each node pool sends the node backup data collected by the first node to the central node in the cluster according to the longest second cycle T2, i.e., the minimum frequency, and the central node backs up the node data of all edge nodes in the cluster. In this way, while realizing node data backup, the interaction frequency between the central node and the first node can be reduced, thereby further reducing the burden on the network bandwidth of the central node.

[0065] For example, Figure 4 As shown in , edge nodes 1, 2, and 3 send their own node data to the first node 4 according to T1, and edge nodes 5 and 6 send their own node data to the first node 7 according to T1. The first node 4 corresponds to the second node 9, and the first node 7 corresponds to the second node 10. The first node 4 sends the collected node backup data to the second node 9 according to T3, and the second node 9 backs up the node data of all edge nodes in the node pool to which the first node 4 belongs. The first node 7 sends the collected node backup data to the second node 10 according to T3, and the second node 10 backs up the node data of all edge nodes in the node pool to which the first node 7 belongs. At the same time, the first node 4 and the first node 7 send their own node backup data to the central node 8 according to T2, and the central node 8 backs up its node data for all edge nodes. As a result, while realizing node data backup, the interaction frequency between the central node 8 and the first node 4 and the first node 7 can be reduced, thereby further reducing the burden on the network bandwidth of the central node 8.

[0066] Based on the above implementation scheme, the first node is: a node that meets the first condition among the backup nodes in the node pool to which it belongs, and the second node is: a node that meets the second condition among the backup nodes in the node pool to which it belongs.

[0067] The first condition is different from the second condition.

[0068] It should be noted that all nodes in each node pool have node role information role to indicate whether the node is an edge node or a backup node.

[0069] That is to say, in this embodiment, the first node can be selected from the backup nodes deployed in each node pool according to the first condition, and the second node can be selected according to the second condition.

[0070] Specifically, since the first node and the second node implement different functions in the node pool, the conditions used for selecting the nodes are different.

[0071] Based on the above implementation, the first node acts as a backup node for the edge node. When an edge node fails, the first node needs to perform processing services for the failed edge node. Therefore, the first node is mainly used to achieve rapid migration of processing services, that is, the processing services are migrated from the failed edge node to the first node; the second node acts as a backup for the first node, and is mainly used to store the node data of each edge node collected by the first node. Therefore, the first condition for selecting the first node mainly considers the state of the data processor that can perform processing services; and the second condition for selecting the second node mainly considers the state of the system operation resources that can achieve data backup.

[0072] For example, the first condition may include: the backup node is ranked first according to the first sorting rule in the first scoring system of the deployed data processor. The first sorting rule may be a sorting rule from large to small.

[0073] In a specific implementation, in this embodiment, the first score value of each backup node in the node pool regarding the deployed data processor can be obtained first, and then the first score value corresponding to each backup node is sorted from large to small, and the backup node with the first score value ranked first is determined as the first node.

[0074] The first score value may be at least related to the number of data processors in the backup node and the operating environment resource information of the data processors. Further, the first score value may also be related to the system operating resource information of the backup node and the data processors deployed therein.

[0075] For example, in this embodiment, a first weight value, a second weight value, and a third weight value corresponding to the first score value may be pre-set. The first weight value is: a weight value corresponding to the system operation resource information of the backup node and the data processor deployed thereon; the second weight value is: a weight value corresponding to the number of data processors in the backup node; and the third weight value is a weight value corresponding to the operating environment resource information of the data processor. Among them, the second weight value and the third weight value corresponding to the first score value are respectively greater than the first weight value.

[0076] For another example, the second condition may include: the second score value of the backup node regarding the second score value of the system operation resource is sorted first according to the second sorting rule. The second sorting rule may be a sorting rule from large to small.

[0077] In the specific implementation, in this embodiment, the second score value of each backup node in the node pool regarding the system operation resources can be obtained first, and then the second score value corresponding to each backup node can be sorted from large to small, and the backup node with the first second score value can be determined as the second node.

[0078] The second score value is at least related to the system operation resource information of the backup node and the data processor deployed therein. Furthermore, the second score value may also be related to the number of data processors in the backup node and the operation environment resource information of the data processors.

[0079] For example, in this embodiment, the first weight value, the second weight value, and the third weight value corresponding to the second score value may be preset. The first weight value is: the weight value corresponding to the system operation resource information of the backup node and the data processor deployed therein; the second weight value is: the weight value corresponding to the number of data processors in the backup node; the third weight value is the weight value corresponding to the operating environment resource information of the data processor. Among them, the first weight value corresponding to the second score value is greater than the second weight value and the third weight value.

[0080] Specifically, in this embodiment, the first scoring value and the second scoring value can be obtained according to formula (1):

[0081] P = α × S1 + β × S2 + γ × S3 (1)

[0082] At this time, P is the first scoring value or the second scoring value, α is the first weight value, S1 is the scoring value corresponding to the system operation resource information of the backup node and its deployed data processor; β is the second weight value, S2 is the scoring value corresponding to the number of data processors; γ is the third weight value, S3 is the scoring value corresponding to the operating environment resource information of the data processor. α, β, and γ are adjustable weight values, the sum of which is 1. They can be adjusted based on the specific environment and the expected scoring effect.

[0083] It should be noted that when the first score value is obtained according to formula (1), α in formula (1) is smaller than β in formula (1), and α in formula (1) is smaller than γ in formula (1).

[0084] When the second score value is obtained according to formula (1), α in formula (1) is greater than β in formula (1), and α in formula (1) is greater than γ in formula (1).

[0085] The following is an explanation of Notice (1):

[0086] First, S1 introduces the logarithmic effect of the backup node on P, ensuring that the benefits of increasing the first or second score value decrease. S1 can be obtained by formula (2):

[0087] S1=log(1+Score node ) (2)

[0088] Among them, S2 can bring a nonlinear effect of exponential growth to the number of DPUs, which means that the increase in the number of DPUs will accelerate the increase of the first score value or the second score value. S2 can be obtained by formula (3):

[0089]

[0090] Where D is the number of DPUs in the backup node.

[0091] Among them, S3 can be combined with the number of DPUs, the average score of all DPUs and the sum of the scores of all DPUs, and consider the impact of the combination on the first score or the second score. The logarithmic term log is used to ensure the gradual stability of the growth of the first score or the second score. S3 can be obtained by formula (4):

[0092] S3=D·Score avg ·log(1+Score total ) (4)

[0093] Score node It refers to the score value of the backup node itself, which can be obtained by formula (5):

[0094]

[0095] Among them, BW size BW is the maximum capacity of the network bandwidth of the backup node that can be used for data transmission. used BW is the bandwidth currently used by the backup node; min BW is the minimum threshold of the network bandwidth of the backup node. max The maximum bandwidth level supported by the network bandwidth of the backup node; CPU total The total number of cores of the central processing unit (CPU) in the backup node; CPU used The number of CPU cores currently used by the backup node; Mem total Mem is the total memory capacity of the backup node; free The free memory capacity of the backup node; Disk total The total disk capacity of the backup node; Disk free is the free disk capacity of the backup node. w1, w2, w3, w4, and w5 are weight coefficients, which are adjustable parameters with a sum of 1. They can be adjusted based on the specific environment and the expected scoring effect.

[0096] Among them, Score total It refers to the sum of the scores of all DPUs in the backup node, which can be obtained by formula (6);

[0097]

[0098] Among them, Score avg It refers to the average score of all DPUs in the backup node, which can be obtained by formula (7):

[0099]

[0100] in, is the score value of the i-th DPU.

[0101] Based on the above implementation scheme, the central node can receive the node resource information sent by the backup node through the data connection with each backup node, and determine the first node and the second node in each node pool according to the node resource information.

[0102] The node resource information of the edge node may include: the number resource information of the data processors deployed in the backup node such as S2, the operating environment resource information of the data processors deployed in the backup node such as S3, and the system operating resource information of the backup node and its deployed data processors such as S1.

[0103] Based on this, the central node scores and calculates the node resource information of each received backup node through formula (1) to formula (7) to obtain the first score value and the second score value corresponding to each backup node, and then for each node pool, selects the backup node with the largest first score value as the first node in the node pool to which it belongs, and selects the backup node with the largest second score value as the second node in the node pool to which it belongs.

[0104] For example, Figure 5 As shown in , there are multiple backup nodes in node pool a and multiple backup nodes in node pool b. The central node 8 performs scoring calculations on the node resource information of each backup node in each node pool received by formula (1) to formula (7) to obtain the first scoring value and the second scoring value corresponding to each backup node in node pool a, and the first scoring value and the second scoring value corresponding to each backup node in node pool b. Then, for node pool a, the backup node with the largest first scoring value is selected as the first node 4 in node pool a, and the backup node with the largest second scoring value is selected as the second node 9 in node pool a; for node pool b, the backup node with the largest first scoring value is selected as the first node 7 in node pool b, and the backup node with the largest second scoring value is selected as the second node 10 in node pool b.

[0105] Based on the above implementation, the data connection between the central node and the backup node can be realized through a plug-in. For example, a first plug-in is deployed in the central node, such as a BK manager plug-in (BookKeeperTaskManager) in a certain operating system; a second plug-in is deployed in the backup node, such as a BK client plug-in in a certain operating system. The data connection can be established through the first plug-in and the second plug-in.

[0106] For example, Figure 6 As shown in , there is a BK manager plug-in in the central node and a BK client plug-in in the backup node. A data connection is established between the central node and each backup node through the BK manager plug-in and the BK client plug-in. Through the data connection, the central node receives the node resource information of each backup node in each node pool, and determines the first node and the second node in each node pool accordingly.

[0107] In one implementation, in this embodiment, when the cluster is initially created, after the node pools are divided, the central node selects a first node as an edge node backup and a second node for storing node backup data for each node pool;

[0108] In another implementation, in this embodiment, when an edge node fails and the first node is required to serve as a new edge node to continue processing services for the failed edge node, the central node selects a new first node for the node pool to which the failed edge node belongs.

[0109] In a specific implementation, in this embodiment, each edge node can be continuously monitored for failure. In the case of a failure of an edge node, the processing service of the failed edge node can be switched to (i.e., migrated to) the first node in the node pool to which the failed edge node belongs, so that the first node in the node pool to which the failed edge node belongs serves as an edge node to execute the processing service of the failed edge node. Then, from the backup nodes in the node pool to which the failed edge node belongs, a backup node that meets the first condition is selected as the new first node.

[0110] That is to say, in this embodiment, if an edge node fails, the first node in the node pool can continue to execute the processing business of the failed edge node as a backup, and it is necessary to re-select a backup node as the first node. At this time, the first node can be re-selected from the backup nodes according to formulas (1) to (7) in the previous text. The re-selected first node continues to receive the node data sent by each edge node in the node pool to which it belongs, and sends the obtained node backup data to the second node backup in the node pool to which it belongs, and also to the central node backup in the cluster.

[0111] It should be noted that when the first node switched to, i.e., the new edge node, performs the processing business of the failed edge node, it first obtains the node data of the failed edge node from the second node in the node pool to which it belongs (because the second node has the node backup data sent by the first node), and then performs the switched processing business according to the obtained node data. Therefore, in this embodiment, for the failed edge node, the node data can be provided by the second node in the node pool to facilitate the new edge node to continue to perform the corresponding processing business, and the processing business will not be unable to be executed or need to be restarted due to the failure of the edge node, thereby improving the reliability of the processing business execution.

[0112] Furthermore, if the first node fails, the business function of the first node can be switched to the second node. Since the second node has node backup data backed up by the first node, the second node can directly take over the business function of the first node, i.e., collect the node data of each edge node and send the composed node backup data to the central node, and reselect a backup node as the second node. Therefore, in this embodiment, the backup of node data can continue to be guaranteed, thereby improving the reliability of backup.

[0113] refer to Figure 7 , is a schematic diagram of the architecture of a cluster provided in an embodiment of the present application, the cluster at least comprising: a plurality of edge nodes 701 and a central node 702; the plurality of edge nodes 701 are divided into a plurality of node pools 703, and a first node 704 is deployed in each of the node pools 703;

[0114] Among them, each of the edge nodes 701 sends node data to the first node 704 in the node pool 703 to which the edge node 701 belongs; each of the first nodes 704 sends node backup data to the central node 702, and the node backup data sent by the first node 704 includes the node data of each edge node in the node pool 703 to which the first node 704 belongs; the central node 702 stores the node backup data.

[0115] It can be seen from the above scheme that in a cluster provided by an embodiment of the present application, the edge nodes of the cluster can be divided into multiple node pools, and a first node is deployed in each node pool. The first node sends the node data of each edge node in the node pool to which it belongs to the central node in the cluster. By using the first node as a data transfer node, the number of nodes that interact with the central node can be reduced, thereby reducing the burden on the network bandwidth of the central node.

[0116] In one implementation, each of the node pools 703 is further deployed with a second node 705, such as Figure 8 As shown in;

[0117] The edge node 701 sends the node data to the first node 704 according to the first cycle, and the first node 704 sends the node backup data to the central node 702 according to the second cycle; the first node 704 sends the node backup data to the second node 705 in the node pool 703 to which the first node 704 belongs according to the third cycle; wherein the second cycle is greater than the third cycle, and the third cycle is greater than the first cycle.

[0118] In one implementation, the first node 704 is a node that satisfies a first condition among the backup nodes in the node pool 703 to which it belongs; the second node 705 is a node that satisfies a second condition among the backup nodes in the node pool 703 to which it belongs; wherein the first condition is different from the second condition.

[0119] Among them, the first condition includes: the first score value of the backup node regarding the deployed data processor is sorted first according to the first sorting rule; the second condition includes: the second score value of the backup node regarding the system running resources is sorted first according to the second sorting rule.

[0120] In one implementation, the first scoring value is at least related to the number of the data processors and the operating environment resource information of the data processors; the second scoring value is at least related to the system operating resource information of the backup node and the data processors deployed thereon.

[0121] Among them, the central node 702 receives the node resource information sent by the backup node through the data connection between the central node and each backup node, and determines the first node 704 and the second node 705 in each node pool 703 according to the node resource information; the node resource information includes: the number resource information of the data processors, the operating environment resource information of the data processors, and the system operating resource information of the backup nodes and the data processors deployed thereon.

[0122] In one implementation, a first plug-in is deployed in the central node 702, a second plug-in is deployed in the backup node, and the data connection is established through the first plug-in and the second plug-in.

[0123] In one implementation, when the edge node 701 fails, the processing business of the failed edge node 701 is switched to the first node 704 in the node pool 703 to which the failed edge node 701 belongs, and the first node 704 in the node pool 703 to which the failed edge node 701 belongs serves as the edge node 701 to execute the processing business; and a node that meets the first condition is selected from the backup nodes of the node pool 703 to which the failed edge node 701 belongs as the new first node 704.

[0124] In one implementation, the edge nodes 701 are divided into corresponding node pools 703 according to the networks in which they are located; edge nodes 701 in the same network or networks in a close geographical area are divided into the same node pool 703 .

[0125] It should be noted that the specific implementation method of the node in this embodiment can refer to the corresponding content in the previous text and will not be described in detail here.

[0126] Taking the cluster in the cloud network as an example, the technical solution of this application is described in detail below:

[0127] Against the backdrop of the rapid growth of cloud network applications, emerging business operation scenarios are increasing, which has prompted the widespread adoption of data processing unit (DPU) hardware. As more and more business processes are offloaded to the DPU for processing, this host and DPU integrated environment has put forward new requirements for data backup. In particular, the backup of critical data (such as business data or files) may need to follow different management strategies in different business scenarios. In particular, in a network architecture with a central node and multiple edge nodes, it is necessary to effectively back up the data of the edge nodes to the central node, and directly synchronizing each edge node to the central node will put great pressure on the network bandwidth of the center, and multiple data replications may be required during the backup process.

[0128] In response to the above problems, the present application proposes an optimized data backup management solution, which is suitable for the environment of node hosts and node pools (also referred to as edge node pools) integrated with DPU. The technical solution of the present application introduces new node roles such as backup master nodes and backup slave nodes in the node pool. Specifically, by electing a backup master node (i.e., the first node in the previous text), other edge nodes synchronize the data to this backup master node. Subsequently, the backup master node is responsible for periodically synchronizing the data to the central node. Through this solution, the number of connections and network bandwidth required for each edge node to directly synchronize data to the central node is greatly reduced, thereby optimizing resource consumption and backing up the node pool as a whole. In addition, the technical solution of the present application can also be used as a selection criterion for electing a backup master node based on the performance of hardware resources such as DPU to ensure the efficiency and reliability of the backup process.

[0129] In summary, this application provides an efficient data backup management solution that aims to cope with the growing business needs and complexity in cloud network environments, and is particularly suitable for scenarios where data needs to be backed up from edge nodes to central nodes. By optimizing the backup path and election mechanism, the technical solution of this application can improve the efficiency of data synchronization, reduce the burden on central nodes, and ensure the security and integrity of critical data.

[0130] The following are the key points of the technical solution of this application:

[0131] 1. Node pool and role design: This application divides geographically dispersed edge nodes into different node pools, and defines two key roles in each node pool: working nodes (i.e. edge nodes, also known as WORKER) and backup nodes (BACKUP). In these node pools, backup master nodes and backup slave nodes (i.e. the second node in the previous text) are elected from the backup nodes through an election mechanism, and each of them assumes different data backup responsibilities.

[0132] 2. Backup client plug-in, namely BK client plug-in: In order to effectively manage the backup process, the technical solution of this application deploys a backup client plug-in on each node (including edge nodes and backup nodes). The plug-in is responsible for collecting and reporting the node resource information of the node, including: role information and its physical resource status, such as CPU, memory, IO speed, disk capacity, and special attention to the number and performance information of DPUs. This detailed resource information will be sent to the backup manager of the central node to facilitate the selection of the most suitable backup master node and slave node from the node pool.

[0133] 3. Hierarchical backup synchronization mechanism: The technical solution of the present application designs a hierarchical data backup synchronization mechanism to minimize network bandwidth pressure and optimize the data synchronization process. Specifically, all working nodes synchronize their backup data (i.e., the node data in the previous text) to the backup master node within the time period T1; the backup master node synchronizes the node backup data to the backup slave node within the time period T2; and within the longer time period T3, the backup master node synchronizes the node backup data to the central node. By setting T1 to be less than T2, and T2 to be less than T3, the technical solution of the present application ensures the efficiency and flexibility of data backup.

[0134] 4. Scoring algorithm for backup master node election: In order to ensure that the elected backup master node can process large amounts of data efficiently and reliably, the technical solution of this application has developed a novel scoring algorithm. This algorithm not only takes into account traditional performance indicators such as the host's CPU, memory, disk capacity, IO rate, etc., but also specifically evaluates the number and performance of DPUs. This comprehensive evaluation mechanism aims to select nodes that have both sufficient hardware resources and high-performance DPUs as backup master nodes.

[0135] The technical solution of this application is described in detail below:

[0136] (1) Deploy the BK client plug-in on each node to obtain the node role (such as the edge node WORKER or backup node BACKUP that executes business processing) and communicate with the DPU agent running inside the DPU to obtain the number of DPUs on the current node and the resource information inside each DPU. Fig. 9 As shown in the figure, BK client plug-in and DPU1 and DPU2 are deployed on node 1. BK client plug-in can obtain node role. DPU1 and DPU2 send resource information to BKclient plug-in through agent. DPU is used for data plane processing of business. Configuration plane data is stored in the relevant storage dataStorage on the node. BK client plug-in running on the host can access data storage for subsequent backup data synchronization to the backup master node.

[0137] (2) For edge nodes in the same region, a node pool object is created to uniformly manage all nodes in a certain area. The central node runs independently outside the node pool and connects to the BK client plug-in running on all nodes under each node pool through the BK manager plug-in. After each node pool is connected to the central node, the BK client plug-in of all nodes will report all the information collected about the node to the BK manager. The central node selects the backup master node and backup slave node for the node with the role of BACKUP.

[0138] (3) The BK manager plug-in selects the backup BACKUP node as the elector based on the node role, and elects the backup master and slave nodes from the BACKUP node. If the node with the BACKUP role does not exist, an election is held among the remaining nodes with other roles. In other words, a portion of the working nodes are selected as backup nodes, and then the backup master node is elected from the backup nodes. During the election, the resources of the node host HOST, the number of DPUs, and the resources of the DPU internal environment are scored in turn. There are differences in the election rules for the backup master node and the backup slave node, as follows:

[0139] ① Backup master node: Prioritize nodes with high DPU quantity and resource score (i.e., first score value) in DPU. The purpose is to quickly migrate the services of the failed node to the master node when performing master-slave switching. In the HOST combined with DPU fusion scenario, the services usually run on the DPU, so the fusion environment with better DPU resources will be more suitable for restoring the services. The backup master node meets the DPU hardware requirements for switching, which will reduce the copying of backup data restored from the central node, and can restore the services more quickly and reduce the service interruption time.

[0140] ② Backup slave node: Prioritize nodes with higher HOST resource scores (i.e., the second score value). The design idea of ​​the backup slave node is to store backup data inside the node pool. When the backup master node fails, it can quickly take over the role of backup manager. Since there is a copy of the backup data, the backup management business can be quickly restored. Therefore, the backup slave node is more used to store backup data. For environments with better HOST resources, data can be written more quickly, and the resource requirements for the DPU are not high.

[0141] Therefore, the election of backup master nodes and backup slave nodes can be differentiated by the weights of the scoring algorithm.

[0142] (4) After the central node selects a backup master node and a backup slave node for each node pool, the node with the role of WORKER in the node pool synchronizes the local backup data (i.e., the node data in the previous text) to the backup master node BK master through the BK client plug-in according to the dataStorage cycle T1, and then the backup master node BK master synchronizes the node backup data to the backup slave node BK slave according to the cycle T2. The backup master node BK master also synchronizes the node backup data to the BK manager plug-in of the central node according to the cycle T3, which is used for data recovery after data loss caused by node pool failure or other failures. Here, T1 is less than T2, and T2 is less than T3. Reduce the frequent preemption of the central node network bandwidth, single point failures in the node pool, the node pool can be quickly restored, and the short cycle of backup data is also conducive to the timeliness of data.

[0143] like Fig.10As shown in the figure, it is an example diagram of data transmission between nodes in the cluster through plug-ins. There are node pools 1 and 2 in the cluster. Node pool 1 has nodes 1, 2, 3, 4 and 5. Node 1 and 2 are nodes with the role of WORKER (i.e., edge nodes in the previous text), and nodes 3, 4 and 5 are nodes with the role of BACKUP (i.e., backup nodes in the previous text). In addition to the storage area data Storage, nodes 1, 2, 3, 4 and 5 also have BK client plug-ins. Node 3 is the backup master node BK master, and node 4 is the backup slave node BK slave. In addition to the storage area data Storage, the central node also has a BK manager plug-in. Node 1 and Node 2 use the BK client plug-in to synchronize the local backup data (i.e. the node data mentioned above) to the backup master node BK master, i.e. Node 3, according to the cycle (T1) of the data storage. Then, Node 3 synchronizes the node backup data to the backup slave node BK slave, i.e. Node 4, according to the cycle (T2). Node 3 also synchronizes the node backup data to the BK manager plug-in of the central node according to the cycle (T3). The same applies to Node Pool 2.

[0144] pass Fig.10 As shown, compared with the method in which each node transmits data to the BK manager plug-in of the central node through the BK client plug-in, in this embodiment, the backup master node BK master collects the backup data of the node with the role of WORKER, and then the backup master node BK master synchronizes the backup data to the BK manager plug-in of the central node through the BK client plug-in. This can reduce the number of nodes that directly interact with the central node, effectively reduce the demand for network bandwidth, and reduce the pressure on the central node to handle a large number of concurrent data synchronization requests, thereby optimizing the resource consumption of the entire network.

[0145] Among them, each node transmits information such as node role, the number of DPUs on the current node, and resource information inside each DPU to the BK manager plug-in of the central node through the BK client plug-in, so that the central node can elect backup master nodes and backup slave nodes for nodes with the role of BACUP. This transmission process does not need to be performed frequently, so it will not cause a large burden on the network bandwidth.

[0146] It should be noted that the system operation resource score can include the score estimation of the system on the design host and the embedded system in the DPU. They are all common hardware and system operation environments, and the same scoring rules can be used. For details, please refer to the line of sight method of obtaining the first score value and the second score value in formula (1) to formula (7) in the previous text.

[0147] In summary, the technical solution of this application has the following advantages:

[0148] 1. Optimization of network bandwidth and resource consumption: By adopting a hierarchical data synchronization mechanism, that is, all edge nodes first back up node data to the backup master node, and then the backup master node is responsible for synchronizing to the central node, the amount of data transmission in the network is significantly reduced. This strategy not only effectively reduces the demand for network bandwidth, but also reduces the pressure on the central node to handle a large number of concurrent data synchronization requests, thereby optimizing the resource consumption of the entire network.

[0149] 2. Improved flexibility and reliability of the backup process: The introduction of backup client plug-ins and backup master node election scoring algorithms can dynamically select the most suitable backup master node based on the real-time performance indicators and resource conditions of the node. This method not only improves the flexibility of the backup process and copes with the dynamic changes in the network environment and node performance, but also further ensures the reliability and efficiency of the data backup process by selecting the best node as the backup master node.

[0150] 3. Efficient use of DPU resources: The technical solution of this application pays special attention to the use of DPU resources, and comprehensively considers the number and performance of DPUs through a scoring algorithm to optimize backup nodes. This approach effectively improves the efficiency of DPU resource use, especially in data-intensive application scenarios, and can give full play to the advantages of DPU in data processing and acceleration, and improve the processing capacity and speed of the entire backup process.

[0151] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0152] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0153] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0154] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for processing data backup in a cluster, the method comprising: Each edge node in the cluster sends node data to a first node in a node pool to which the edge node belongs; Each of the first nodes in the cluster sends node backup data to a central node in the cluster; The edge nodes of the cluster are divided into a plurality of node pools, each of which is deployed with the first node, and the node backup data sent by the first node includes node data of each edge node in the node pool to which the first node belongs; The central node stores the node backup data.

2. According to the method of claim 1, the edge node sends the node data to the first node according to a first period, and the first node sends the node backup data to the central node according to a second period; The method further comprises: The first node sends the node backup data to a second node in the node pool to which the first node belongs according to a third period; The second period is greater than the third period, and the third period is greater than the first period.

3. The method according to claim 2, wherein the first node is a node that satisfies a first condition among backup nodes in the node pool to which the first node belongs; and the second node is a node that satisfies a second condition among backup nodes in the node pool to which the second node belongs; in, The first condition is different from the second condition.

4. The method according to claim 3, wherein the first condition comprises: The first score value of the backup node for the deployed data processor is sorted in the first place according to the first sorting rule; The second condition includes: the second score value of the backup node regarding the system operation resources is sorted first according to the second sorting rule.

5. The method according to claim 4, wherein the first score value is at least related to the number of the data processors and the operating environment resource information of the data processors; The second scoring value is at least related to system operation resource information of the backup node and the data processor deployed therein.

6. The method according to claim 5, wherein the central node receives the node resource information sent by each backup node through a data connection with each backup node and determines the first node and the second node in each node pool according to the node resource information; in, The node resource information includes: quantity resource information of the data processors, operating environment resource information of the data processors, and system operating resource information of the backup node and the data processors deployed thereon.

7. According to the method of claim 6, a first plug-in is deployed in the central node, a second plug-in is deployed in the backup node, and the data connection is established through the first plug-in and the second plug-in.

8. The method according to claim 3, further comprising: In the event that the edge node fails, switching the processing service of the failed edge node to a first node in the node pool to which the failed edge node belongs, and the first node in the node pool to which the failed edge node belongs serves as the edge node to perform the processing service; A node that meets the first condition is selected from backup nodes in the node pool to which the failed edge node belongs as a new first node.

9. According to the method of claim 1 or 2, the edge nodes are divided into corresponding node pools according to the networks in which they are located; in, Edge nodes in the same network or geographically close networks are divided into the same node pool.

10. A cluster comprising: A plurality of edge nodes and a central node; the plurality of edge nodes are divided into a plurality of node pools, and a first node is deployed in each of the node pools; Among them, each of the edge nodes sends node data to the first node in the node pool to which the edge node belongs; each of the first nodes sends node backup data to the central node, and the node backup data sent by the first node includes the node data of each edge node in the node pool to which the first node belongs; the central node stores the node backup data.

Images