Business data block backup method and system, electronic equipment and storage medium

By predicting the failure probability of storage nodes and performing group backups, the problem of low data security and high cost caused by improper selection of storage nodes in existing technologies is solved, achieving more efficient and secure data storage and processing.

CN119025333APending Publication Date: 2024-11-26CHINA TELECOM CORP LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411018390.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In existing technologies, business data backup methods suffer from problems such as low data security and high cost due to improper selection of storage nodes. In particular, when multiple storage nodes fail simultaneously, data security is low and each storage node needs to store a complete backup of the business data, resulting in high costs and inconvenience in data reading and writing.

Method used

By acquiring basic information and operational performance data of storage nodes, a simultaneous fault prediction model is trained to predict the failure probability of storage nodes in the next service cycle. Based on the probability values, multiple storage node groups are formed, ensuring that the failure probability of any two storage nodes in the same group is less than or equal to the probability threshold, and then block backup is performed.

Benefits of technology

It improves the security of business data storage, reduces the cost of backup and disaster recovery, increases the parallelism of data writing and reading, and enhances the efficiency of data processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119025333A_ABST
    Figure CN119025333A_ABST
Patent Text Reader

Abstract

The invention discloses a service data block backup method and system, electronic equipment and a storage medium. The method comprises the steps of obtaining basic information of a plurality of storage nodes and operation performance data and fault information in a preset historical period; training a node simultaneous failure prediction model according to the basic information, the operation performance data and the failure information, and predicting a first probability value of simultaneous failure of the two storage nodes in the next service period according to the node simultaneous failure prediction model; grouping the storage nodes according to the first probability values and a preset probability threshold to obtain a plurality of storage node groups, so that the first probability values corresponding to any two storage nodes in the same storage node group are smaller than or equal to the probability threshold; and performing block backup on the to-be-backed-up service data according to the plurality of storage node groups. According to the invention, the security of service data storage is improved, the cost of service data backup disaster recovery is reduced, and the method can be widely applied to the technical field of data processing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method, system, electronic device and storage medium for backing up business data blocks. Background Art

[0002] With the rapid development of information technologies such as cloud computing, big data, the Internet of Things, and artificial intelligence, and the digital transformation of traditional industries, Internet data is showing a huge growth trend. Internet data must have dedicated facilities for storage, transmission, and use. In order to ensure that stable data services can be provided to users, data storage security is very important. However, due to factors such as hardware equipment, software networks, and external attacks, data stored at a single point is prone to data loss and service failure, causing immeasurable losses to service providers and users. Therefore, currently, multiple pairs of data are backed up and stored in multiple storage nodes, that is, data is copied and stored in multiple storage nodes, so that in the event of a catastrophic event, data and systems can be quickly restored to ensure the normal operation of the business.

[0003] However, the current conventional backup method is to copy the entire business data into multiple copies and place them in different storage nodes. This method has the following problems: 1) Although the business data is backed up, there may be inappropriate storage nodes (for example, different storage nodes provided by the same service provider) or multiple storage nodes may fail at the same time, resulting in low data security; 2) Each storage node must store a complete business data backup, which is costly and inconvenient to read and write data.

[0004] Terminology explanation:

[0005] Backup and disaster recovery: refers to a series of measures and technical means to back up data, achieve disaster recovery, and ensure system availability in order to ensure data security and business continuity.

[0006] Storage node: In a computer network, a device or system used to store data. A storage node can be a computer, server, or data center. Summary of the invention

[0007] The purpose of the present invention is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0008] Therefore, an object of an embodiment of the present invention is to provide a method for backing up business data in blocks, which improves the security of business data storage and reduces the cost of business data backup and disaster recovery.

[0009] Another object of an embodiment of the present invention is to provide a business data block backup system.

[0010] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present invention include:

[0011] On the one hand, an embodiment of the present invention provides a method for backing up business data in blocks, comprising the following steps:

[0012] Obtain basic information of multiple storage nodes as well as operating performance data and fault information in a preset historical period;

[0013] Training a node simultaneous failure prediction model according to the basic information, the operating performance data, and the failure information, and predicting a first probability value of simultaneous failure of two storage nodes within the next service cycle according to the node simultaneous failure prediction model;

[0014] The storage nodes are grouped according to the first probability value and a preset probability threshold to obtain a plurality of storage node groups, so that the first probability values ​​corresponding to any two storage nodes in the same storage node group are less than or equal to the probability threshold;

[0015] The service data to be backed up is backed up in blocks according to the plurality of storage node groups.

[0016] Further, in one embodiment of the present invention, the obtaining of basic information of multiple storage nodes and operating performance data and fault information in a preset historical period specifically includes:

[0017] Acquire the basic information of each storage node, the basic information including at least one of a node name, a node location, a network topology, a hardware configuration, a software environment, and a service provider;

[0018] Obtain the operating performance data and the fault information of each storage node in the preset historical period, the operating performance data including at least one of the CPU utilization, memory utilization, disk IO data, network delay, packet loss rate and bandwidth utilization of the storage node, and the fault information including the time when the fault of the storage node occurred.

[0019] Further, in one embodiment of the present invention, the training of a node simultaneous fault prediction model according to the basic information, the operating performance data and the fault information specifically includes:

[0020] Segmenting the preset historical period according to the service cycle to obtain a plurality of sample service cycles, and determining the operating performance data and the fault information of the storage node in each of the sample service cycles;

[0021] Randomly select two of the storage nodes as sample storage nodes, construct a training sample according to the basic information of the sample storage nodes and the operating performance data in the current sample service cycle, and determine the label information of the training sample according to the fault information of the sample storage nodes in the next sample service cycle, wherein the label information is used to characterize whether the sample storage nodes fail at the same time;

[0022] Inputting the training samples into a pre-built long short-term memory network to obtain a node simultaneous failure prediction result;

[0023] A loss value is determined according to the node simultaneous fault prediction result and the label information, and the long short-term memory network is updated according to the loss value to obtain a trained node simultaneous fault prediction model.

[0024] Further, in one embodiment of the present invention, the predicting, according to the node simultaneous failure prediction model, a first probability value of simultaneous failure of two storage nodes within the next service cycle specifically includes:

[0025] Acquire the operation performance data of the two storage nodes in the current service cycle, and generate input data according to the basic information of the two storage nodes and the operation performance data in the current service cycle;

[0026] The input data is input into the node simultaneous failure prediction model to obtain the first probability value that the two storage nodes will fail simultaneously in the next service cycle.

[0027] Further, in an embodiment of the present invention, grouping the storage nodes according to the first probability value and a preset probability threshold to obtain a plurality of storage node groups specifically includes:

[0028] When the first probability value is greater than the probability threshold, determining that the corresponding two storage nodes are mutually exclusive storage nodes, and dividing the mutually exclusive storage nodes into different storage node groups;

[0029] When all the mutually exclusive storage nodes are divided, the remaining storage nodes are added to the storage node groups, so that each storage node group includes at least two storage nodes.

[0030] Furthermore, in one embodiment of the present invention, the business data block backup method further includes a step of updating the probability threshold, which specifically includes:

[0031] Obtaining an initial value and an update granularity value of the probability threshold;

[0032] Grouping the storage nodes according to the first probability value and the probability threshold, and determining whether the grouping is successful;

[0033] When the grouping is successful, the probability threshold of the round of cycles is subtracted from the update granularity value to obtain the updated probability threshold, and the storage nodes are grouped according to the first probability value and the probability threshold;

[0034] When the grouping is unsuccessful, the probability threshold of the previous cycle is used as the final value of the probability threshold.

[0035] Further, in an embodiment of the present invention, the block-by-block backing up of the service data to be backed up according to the plurality of storage node groups specifically includes:

[0036] Splitting the service data to be backed up into multiple data blocks, and determining the storage node group corresponding to each data block;

[0037] The data block is copied to obtain a plurality of backup data blocks, and the plurality of backup data blocks are then respectively stored in a plurality of the storage nodes in the corresponding storage node group.

[0038] On the other hand, an embodiment of the present invention provides a business data block backup system, including:

[0039] A data acquisition module is used to obtain basic information of multiple storage nodes and operating performance data and fault information in a preset historical period;

[0040] A prediction module, configured to train a node simultaneous failure prediction model according to the basic information, the operation performance data and the fault information, and predict a first probability value of simultaneous failure of two storage nodes within a next service cycle according to the node simultaneous failure prediction model;

[0041] a node grouping module, configured to group the storage nodes according to the first probability value and a preset probability threshold to obtain a plurality of storage node groups, so that the first probability value corresponding to any two storage nodes in the same storage node group is less than or equal to the probability threshold;

[0042] The block backup module is used to perform block backup of the service data to be backed up according to the plurality of storage node groups.

[0043] On the other hand, an embodiment of the present invention provides an electronic device, which includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the business data block backup method as described above is realized.

[0044] On the other hand, an embodiment of the present invention further provides a storage medium, which is a computer-readable storage medium used for computer-readable storage, and the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the business data block backup method as described above.

[0045] The advantages and beneficial effects of the present invention will be partly given in the following description, partly become apparent from the following description, or be understood through the practice of the present invention:

[0046] The embodiment of the present invention obtains basic information of multiple storage nodes and operation performance data and fault information in a preset historical period, trains a node simultaneous fault prediction model based on the basic information, operation performance data and fault information, and predicts the first probability value of two storage nodes failing at the same time in the next service cycle based on the node simultaneous fault prediction model, groups the storage nodes according to the first probability value and a preset probability threshold to obtain multiple storage node groups, so that the first probability value corresponding to any two storage nodes in the same storage node group is less than or equal to the probability threshold, and performs block backup of the business data to be backed up according to the multiple storage node groups. The embodiment of the present invention groups the storage nodes so that the probability of any two storage nodes failing at the same time in each storage node group is less than or equal to the probability threshold, provides backup services by completing the grouping of storage nodes, and improves the security of business data storage; uses multiple storage node groups to store business data in blocks, without the need to store a complete business data backup on each storage node, which can improve the parallelism of data writing and data reading, reduce the cost of business data backup disaster recovery, and improve the efficiency of business data writing and reading. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solution in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solution of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 A flowchart of a method for backing up business data in blocks provided by an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of business data backup in the prior art;

[0050] Figure 3 A step flow chart of step S101 provided in an embodiment of the present invention;

[0051] Figure 4 A step flow chart of step S102 provided in an embodiment of the present invention;

[0052] Figure 5 A schematic diagram of the association of fault information of a storage node provided in an embodiment of the present invention;

[0053] Figure 6 Another step flow chart of step S102 provided in an embodiment of the present invention;

[0054] Figure 7 A schematic diagram of the prediction process of the node simultaneous failure prediction model provided by an embodiment of the present invention;

[0055] Figure 8 A step flow chart of step S103 provided in an embodiment of the present invention;

[0056] Fig. 9 A flowchart of the steps of updating the probability threshold provided by an embodiment of the present invention;

[0057] Fig.10 A schematic diagram of an instruction for updating a probability threshold provided by an embodiment of the present invention;

[0058] Fig.11 A step flow chart of step S104 provided in an embodiment of the present invention;

[0059] Fig.12 A schematic diagram of performing block-by-block backup of business data provided by an embodiment of the present invention;

[0060] Fig.13 A schematic diagram of the structure of a business data block backup system provided by an embodiment of the present invention;

[0061] Fig.14 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention;

[0062] Fig.15 A schematic diagram of the structure of a storage medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0063] Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limitations on the present application. It should be noted that, although the functional module division is performed in the system schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order from the module division in the system schematic diagram or the flow chart. For the step numbers in the following embodiments, they are only set for the convenience of explanation, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0064] In the description of the present invention, the meaning of "a plurality" is two or more. If there is a description of the first or the second, it is only for the purpose of distinguishing the technical features, and it cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features. In addition, unless otherwise defined, all technical and scientific terms used in this document have the same meaning as those commonly understood by technicians in the technical field of this application. The terms used in this document are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0065] The business data block backup method provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a set-top box, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the business data block backup method, etc., but is not limited to the above forms.

[0066] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0067] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0068] like Figure 2 FIG. 1 is a schematic diagram of business data backup in the prior art. In the prior art, for a piece of business data, in order to prevent a single storage node failure from causing service paralysis, the business data can be copied into multiple copies and stored in multiple different storage nodes, such as Figure 2 Storage nodes A, B, C, etc. are stored in the storage node. In this way, even if one or several (partial) storage nodes fail, the business data can be obtained from the normally operating storage nodes, which can ensure the normal operation of the service. However, this method has the following problems: 1) Although the business data is backed up, there may be inappropriate storage nodes (for example, different storage nodes provided by the same service provider are selected) and multiple storage nodes fail at the same time, resulting in low data security; 2) Each storage node must store a complete business data backup, which is costly and inconvenient to read and write data.

[0069] The present invention proposes a business data block backup method, in which multiple storage nodes are summarized and aggregated, and basic information of each storage node, operating performance data and fault information in a historical time period are queried and recorded. Then, based on the acquired information, a prediction model is trained to predict the probability of two different storage nodes failing at the same time in the next service cycle. According to the probability of two different storage nodes failing at the same time, the storage nodes are grouped so that the probability of any two storage nodes in each group failing at the same time is less than a probability threshold. Finally, backup services are provided through the storage nodes that have completed the grouping.

[0070] like Figure 1 FIG. 1 is a flowchart of a method for backing up business data blocks provided by an embodiment of the present invention. Figure 1 The embodiment of the present invention provides a method for backing up business data in blocks, which specifically includes the following steps:

[0071] S101. Obtain basic information of multiple storage nodes and operating performance data and fault information in a preset historical period.

[0072] Specifically, multiple storage nodes are summarized and aggregated, and basic information of each storage node, operating performance data and fault information within a historical time period are queried and recorded.

[0073] like Figure 3 FIG. 1 is a flowchart of step S101 provided in an embodiment of the present invention, referring to FIG. Figure 3 As an optional implementation, basic information of multiple storage nodes and operation performance data and fault information in a preset historical period are obtained, which specifically includes:

[0074] S1011. Obtain basic information of each storage node, where the basic information includes at least one of a node name, a node location, a network topology, a hardware configuration, a software environment, and a service provider;

[0075] S1012. Obtain the operating performance data and fault information of each storage node in a preset historical period, the operating performance data including at least one of the CPU utilization, memory utilization, disk IO data, network delay, packet loss rate and bandwidth utilization of the storage node, and the fault information including the time when the fault of the storage node occurred.

[0076] Specifically, in modern large-scale data storage application environments, there are usually multiple storage nodes distributed in different regions, carrying a large amount of data and services. In this solution, it is necessary to record and manage the basic information of each storage node, and query and record the operating performance data and fault information of the storage node.

[0077] Collect and organize basic information about storage nodes, including: the name, location, network topology, hardware configuration, software environment, service provider, etc. of the storage nodes. At the same time, obtain the operation performance data and fault information of the storage nodes through monitoring systems and log records. The operation performance data can include system performance indicators such as CPU utilization, memory utilization, disk IO, network latency, packet loss rate, network bandwidth utilization, etc. Operation fault information refers to when the storage node fails and cannot provide data services normally.

[0078] S102: training a node simultaneous failure prediction model based on basic information, operation performance data, and fault information, and predicting a first probability value of simultaneous failures of two storage nodes within a next service cycle based on the node simultaneous failure prediction model.

[0079] Specifically, based on the acquired information, a prediction model is trained to predict the probability of two different storage nodes failing at the same time in the next service cycle. After the prediction model is trained, the prediction model can be used to determine the probability of failures occurring at the same time between the storage nodes in the next future service cycle.

[0080] like Figure 4 FIG. 1 is a flow chart of step S102 provided in an embodiment of the present invention, referring to FIG. Figure 4 As an optional implementation, a node simultaneous fault prediction model is trained based on the basic information, the operation performance data and the fault information, which specifically includes:

[0081] S1021. Segment the preset historical period according to the service cycle to obtain multiple sample service cycles, and determine the operation performance data and fault information of the storage node in each sample service cycle;

[0082] S1022. Randomly select two storage nodes as sample storage nodes, construct training samples based on basic information of the sample storage nodes and operating performance data in the current sample service cycle, and determine label information of the training samples based on fault information of the sample storage nodes in the next sample service cycle. The label information is used to characterize whether the sample storage nodes fail at the same time.

[0083] S1023, inputting the training samples into a pre-built long short-term memory network to obtain a node simultaneous failure prediction result;

[0084] S1024. Determine a loss value according to the node simultaneous failure prediction result and label information, and update the long short-term memory network according to the loss value to obtain a trained node simultaneous failure prediction model.

[0085] Specifically, in an embodiment of the present invention, based on the basic information of the storage nodes and the operation status in the historical time period, the probability of two different storage nodes failing at the same time in a predetermined time period in the future is predicted. Here, the service cycle is first defined (for example, it can be one month, one year, etc.), and then the historical time period is segmented to construct training data. For example, assuming that there are two storage nodes A and storage node B, their basic information, operating performance data, and fault information can be obtained, and the historical time period can be divided according to the predetermined service cycle, such as each month as a service cycle. Then, based on the fault information, it is determined whether the two storage nodes have failed at the same time in each service cycle of the historical time period. Figure 5 FIG. 1 is a schematic diagram showing the association of fault information of a storage node provided by an embodiment of the present invention. Figure 5 The broken line data in represents the running performance data, and the arrow points to the time node when the failure occurs. When the basic information and running performance data of two storage nodes are close, the probability of both failing at the same time will be high. Therefore, based on the basic information of the two storage nodes and the running performance data in the previous service cycle, the probability of simultaneous failure in the next service cycle can be predicted. Figure 5 That is, the operating performance data of storage node A and storage node B in the T1 period can be used as the input data of the prediction model, and the result of whether storage node A and storage node B fail at the same time in the T2 period can be used as the corresponding label data. The label data can measure the accuracy of the prediction result and realize the iterative training of the node simultaneous failure prediction model.

[0086] It is understandable that the embodiment of the present invention does not limit the specific structure of the node simultaneous failure prediction model and the form of the result output, and it can be implemented with reference to the model commonly used in the prior art for predicting timing tasks.

[0087] like Figure 6 FIG. 1 is another step flow chart of step S102 provided in an embodiment of the present invention, referring to FIG. Figure 6 As an optional implementation, the first probability value of two storage nodes failing at the same time in the next service cycle is predicted according to the node simultaneous failure prediction model, which specifically includes:

[0088] S1025. Obtain the operation performance data of the two storage nodes in the current service cycle, and generate input data according to the basic information of the two storage nodes and the operation performance data in the current service cycle;

[0089] S1026. Input the input data into a node simultaneous failure prediction model to obtain a first probability value of two storage nodes failing simultaneously in the next service cycle.

[0090] Specifically, Figure 7 The figure shows a schematic diagram of the prediction process of the node simultaneous failure prediction model provided by an embodiment of the present invention. The input data includes basic information of the two storage nodes and operating performance data in the previous service cycle. Based on this information, the prediction model can evaluate the correlation and similarity of the two storage nodes, thereby predicting the probability of simultaneous failure of the two storage nodes in the next service cycle.

[0091] S103. Grouping the storage nodes according to the first probability value and a preset probability threshold to obtain a plurality of storage node groups, such that the first probability values ​​corresponding to any two storage nodes in the same storage node group are less than or equal to the probability threshold.

[0092] Specifically, the storage nodes are grouped according to the probability that two different storage nodes fail at the same time.

[0093] like Figure 8 FIG. 1 is a flow chart of step S103 provided in an embodiment of the present invention, referring to FIG. Figure 8 As an optional implementation, the storage nodes are grouped according to the first probability value and the preset probability threshold to obtain a plurality of storage node groups, which specifically include:

[0094] S1031. When the first probability value is greater than the probability threshold, determine that the corresponding two storage nodes are mutually exclusive storage nodes, and divide the mutually exclusive storage nodes into different storage node groups;

[0095] S1032: When all mutually exclusive storage nodes are divided, the remaining storage nodes are added to the storage node groups, so that each storage node group includes at least two storage nodes.

[0096] Specifically, in this scheme, after obtaining the probability of each storage node failing at the same time in the next service cycle in the future, the storage nodes are divided into multiple groups according to the probability. For example, assuming that there are 20 available storage nodes in total, they can be divided into 4 groups (the specific number of groups can be determined according to demand), and each group includes 5 storage nodes. When dividing, set the constraint condition: if the probability of two different storage nodes failing at the same time is greater than a predetermined probability threshold, they are divided into different groups. The probability threshold value here can be set according to demand, for example, it can be set to 5%, then when dividing, if the probability of storage node A and storage node B failing at the same time is 30%, they are divided into different groups; if the probability of storage node A and storage node B failing at the same time is 1%, they can be divided into the same group. When specifically dividing, the storage node combinations can be sorted according to the size of the probability of simultaneous failure, and then the combinations with the probability of simultaneous failure greater than 5% are divided first. After all these combinations are divided, the remaining storage nodes are filled into the groups that have not reached 5. Through the above method, after division, the probability of simultaneous failure of any two of the five storage nodes in each group is less than 5%. After a copy of the data is distributed to the storage nodes in the same group, the security of the data can be greatly improved because the probability of simultaneous failure of the storage nodes in the group is very low.

[0097] like Fig. 9 FIG. 1 is a flowchart of a step of updating a probability threshold provided by an embodiment of the present invention, referring to FIG. Fig. 9 As an optional implementation, the business data block backup method further includes a step of updating the probability threshold, which specifically includes:

[0098] S201, obtaining an initial value of a probability threshold and an update granularity value;

[0099] S203, grouping the storage nodes according to the first probability value and the probability threshold, and determining whether the grouping is successful;

[0100] S204: When the grouping is successful, the probability threshold of the round of cycles is subtracted from the updated granularity value to obtain an updated probability threshold, and the storage nodes are grouped according to the first probability value and the probability threshold.

[0101] S205: When the grouping is unsuccessful, the probability threshold of the previous cycle is used as the final value of the probability threshold.

[0102] Specifically, in the constraints defined above, the setting of the predetermined probability threshold is very important. If it is set too high, the effect of grouping will be greatly reduced, the probability of simultaneous failure of storage nodes in the group will be too high, and the practicality of the method will be affected. If it is set too low, it may lead to failure to complete the grouping normally. In this scheme, a preferred grouping method is also provided, which can not only realize grouping, but also constrain the probability of simultaneous failure of storage nodes in each group to be as small as possible.

[0103] like Fig.10 The figure is a schematic diagram of an instruction for updating a probability threshold provided by an embodiment of the present invention, and the specific process is as follows:

[0104] 1) Set the initial value m of the probability threshold and the update granularity value a, where the value of m can be relatively large to ensure that there is a solution for the first grouping, that is, there is a grouping method that satisfies the initial value m of the probability threshold. For example, it can be set to 5%; the value of a is the granularity of the update. The smaller it is, the more accurate the probability threshold is. For example, it can be set to 0.1%.

[0105] 2) Use the current probability threshold to divide the storage nodes and determine whether there is a solution, that is, whether the storage nodes can be grouped and combined while meeting the current probability threshold; if not (if there is no solution in the first division, it is necessary to reset the initial value m of a larger probability threshold), it means that the probability threshold determined in the previous cycle is already the smallest possible probability threshold. If it is smaller, there will be no feasible division combination. At this time, execute step 3). If so, execute step 4).

[0106] 3) The probability threshold in the previous cycle is used as the final probability threshold, and any one of the grouping solutions obtained in the previous round is selected to complete the grouping of storage nodes.

[0107] 4) Update the current probability threshold, subtract the update granularity value a, obtain the new current probability threshold, and return to step 2) to re-judge.

[0108] Through the above method, the probability threshold can be gradually reduced, which can not only realize grouping, but also constrain the probability of simultaneous failure of storage nodes in each group to be as small as possible. For example, the initial value m of the probability threshold is set to 5%, which can complete the grouping. After the first round of update, the probability threshold m becomes 4.9%, and the grouping can still be completed... After multiple iterations, when the probability threshold reaches 0.8%, it is found that there is no solution to the grouping task. This means that the current storage nodes can meet the grouping method in which the probability of two different storage nodes failing at the same time is less than or equal to 0.9% under extreme circumstances. In this way, the probability of simultaneous failure of the five storage nodes in each group is no more than 0.9%, which is greatly reduced compared to the previous 5%, and can further improve the security of data.

[0109] S104: Back up the service data to be backed up in blocks according to multiple storage node groups.

[0110] Specifically, after the storage nodes are grouped, the probability that any two storage nodes in each group fail at the same time is less than or equal to the probability threshold. When performing backup, multiple storage nodes in the same group can be used to perform data storage tasks. In this way, the probability that multiple storage nodes fail at the same time is extremely small, which can greatly improve data security.

[0111] like Fig.11 FIG. 1 is a flowchart of step S104 provided in an embodiment of the present invention, referring to FIG. Fig.11 As an optional implementation, the business data to be backed up is backed up in blocks according to multiple storage node groups, which specifically includes:

[0112] S1041, splitting the business data to be backed up into multiple data blocks, and determining a storage node group corresponding to each data block;

[0113] S1042: Duplicate the data block to obtain multiple backup data blocks, and then store the multiple backup data blocks in multiple storage nodes in the corresponding storage node group respectively.

[0114] Specifically, Fig.12 FIG. 1 is a schematic diagram of performing block backup of business data provided by an embodiment of the present invention. In the embodiment of the present invention, the storage nodes divided into multiple groups can provide a block storage function. For a piece of business data, it can be divided into several data blocks. Fig.12 For each data block, a group of storage nodes can be selected for backup service. For example, there may be 5 storage node groups, each with 6 storage nodes, and the number of storage node groups is S1-S5. The business data is divided into 3 data blocks, and 3 groups of storage nodes can be selected for backup. Fig.12The storage node group S1-S3 is selected. A group of storage nodes corresponding to data block 1 is S1, and the six storage nodes therein are S11-S16. Data block 1 can be copied and replicated, and then each copied data is stored in a storage node in the storage node group S1. Since each storage node group has six storage nodes, data block 1 can be specifically replicated 2-6 times. The processing process of other data blocks is similar and will not be repeated here. In an embodiment of the present invention, the number of storage node groups can support the number of divided blocks of business data, and the number of storage nodes in each group can support the number of backups of data blocks. Therefore, when grouping the storage nodes, it can be based on the number of divided blocks and the number of backups required by the business data.

[0115] pass Fig.12 The implementation method shown can perform secure backup of a business data on multiple storage nodes. On the one hand, since the probability of any two storage nodes in each group failing at the same time is less than the probability threshold, the possibility of multiple storage nodes failing at the same time is extremely low, which can improve data security. On the other hand, multiple storage node groups are used to store business data in blocks, and there is no need to store a complete business data backup on each storage node. This can improve the parallelism of data writing and data reading, which is beneficial to improving data processing efficiency while reducing costs.

[0116] The above describes the method flow and implementation process of the embodiment of the present invention. It can be understood that the embodiment of the present invention predicts the probability of two different storage nodes failing at the same time in the next service cycle, and groups the storage nodes according to the probability of two different storage nodes failing at the same time, so that the probability of any two storage nodes in each group failing at the same time is less than or equal to the probability threshold, and provides backup services by completing the grouped storage nodes, thereby improving the security of business data storage; optimizing the size of the probability threshold through a cyclic iteration method, thereby constraining the probability of storage nodes in each group failing at the same time to be as small as possible, further improving the security of business data storage; dividing business data into multiple data blocks, each data block uses a storage node group for backup storage, and there is no need to store a complete business data backup on each storage node, which can improve the parallelism of data writing and data reading, reduce the cost of business data backup disaster recovery, and improve the efficiency of business data writing and reading.

[0117] like Fig.13 FIG. 1 is a schematic diagram of the structure of a business data block backup system provided by an embodiment of the present invention, referring to FIG. Fig.13 The embodiment of the present invention provides a business data block backup system, including:

[0118] A data acquisition module is used to obtain basic information of multiple storage nodes and operating performance data and fault information in a preset historical period;

[0119] A prediction module, used to train a node simultaneous failure prediction model based on basic information, operating performance data, and fault information, and predict a first probability value of simultaneous failure of two storage nodes in the next service cycle based on the node simultaneous failure prediction model;

[0120] A node grouping module, used to group the storage nodes according to the first probability value and a preset probability threshold to obtain a plurality of storage node groups, so that the first probability values ​​corresponding to any two storage nodes in the same storage node group are less than or equal to the probability threshold;

[0121] The block backup module is used to perform block backup of the business data to be backed up according to multiple storage node groups.

[0122] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0123] The embodiment of the present invention also provides an electronic device, the electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and the program is executed by the processor to realize the above-mentioned business data block backup method. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0124] like Fig.14 FIG. 1 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention, referring to FIG. Fig.14 , an embodiment of the present invention provides an electronic device, including:

[0125] The processor 1401 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention;

[0126] The memory 1402 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1402 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1402, and the processor 1401 calls and executes the business data block backup method of the embodiment of the present invention;

[0127] Input / output interface 1403, used to implement information input and output;

[0128] The communication interface 1404 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0129] A bus 1405 that transmits information between the various components of the device (e.g., the processor 1401, the memory 1402, the input / output interface 1403, and the communication interface 1404);

[0130] The processor 1401 , the memory 1402 , the input / output interface 1403 and the communication interface 1404 are connected to each other in communication within the device via a bus 1405 .

[0131] like Fig.15 FIG. 1 is a schematic diagram of a storage medium according to an embodiment of the present invention. Fig.15 The embodiment of the present invention further provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs 1501, and the one or more programs 1501 can be executed by one or more processors to implement the above-mentioned business data block backup method.

[0132] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0133] The embodiment of the present invention also discloses a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 1 The method shown.

[0134] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0135] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified to the contrary, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0136] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the above methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0137] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0138] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the above-mentioned program is printed, since the above-mentioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or processing in other suitable ways as necessary, and then stored in a computer memory.

[0139] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0140] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0141] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0142] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for backing up business data in blocks, characterized in that: The following steps are involved: Obtain basic information of multiple storage nodes as well as operating performance data and fault information in a preset historical period; Training a node simultaneous failure prediction model according to the basic information, the operating performance data, and the failure information, and predicting a first probability value of simultaneous failure of two storage nodes within the next service cycle according to the node simultaneous failure prediction model; The storage nodes are grouped according to the first probability value and a preset probability threshold to obtain a plurality of storage node groups, so that the first probability values ​​corresponding to any two storage nodes in the same storage node group are less than or equal to the probability threshold; The service data to be backed up is backed up in blocks according to the plurality of storage node groups.

2. A method for backing up business data in blocks according to claim 1, characterized in that: The obtaining of basic information of multiple storage nodes and operating performance data and fault information in a preset historical period specifically includes: Acquire the basic information of each storage node, the basic information including at least one of a node name, a node location, a network topology, a hardware configuration, a software environment, and a service provider; Obtain the operating performance data and the fault information of each storage node in the preset historical period, the operating performance data including at least one of the CPU utilization, memory utilization, disk IO data, network delay, packet loss rate and bandwidth utilization of the storage node, and the fault information including the time when the fault of the storage node occurred.

3. A method for backing up business data in blocks according to claim 1, characterized in that: The training of a node simultaneous fault prediction model according to the basic information, the operating performance data and the fault information specifically includes: Segmenting the preset historical period according to the service cycle to obtain a plurality of sample service cycles, and determining the operating performance data and the fault information of the storage node in each of the sample service cycles; Randomly select two of the storage nodes as sample storage nodes, construct a training sample according to the basic information of the sample storage nodes and the operating performance data in the current sample service cycle, and determine the label information of the training sample according to the fault information of the sample storage nodes in the next sample service cycle, wherein the label information is used to characterize whether the sample storage nodes fail at the same time; Inputting the training samples into a pre-built long short-term memory network to obtain a node simultaneous failure prediction result; A loss value is determined according to the node simultaneous fault prediction result and the label information, and the long short-term memory network is updated according to the loss value to obtain a trained node simultaneous fault prediction model.

4. A method for backing up business data in blocks according to claim 3, characterized in that: The predicting, according to the node simultaneous failure prediction model, a first probability value of simultaneous failure of two storage nodes within the next service cycle specifically includes: Acquire the operation performance data of the two storage nodes in the current service cycle, and generate input data according to the basic information of the two storage nodes and the operation performance data in the current service cycle; The input data is input into the node simultaneous failure prediction model to obtain the first probability value that the two storage nodes will fail simultaneously in the next service cycle.

5. A method for backing up business data in blocks according to claim 1, characterized in that: The step of grouping the storage nodes according to the first probability value and a preset probability threshold to obtain a plurality of storage node groups specifically includes: When the first probability value is greater than the probability threshold, determining that the corresponding two storage nodes are mutually exclusive storage nodes, and dividing the mutually exclusive storage nodes into different storage node groups; When all the mutually exclusive storage nodes are divided, the remaining storage nodes are added to the storage node groups, so that each storage node group includes at least two storage nodes.

6. A method for backing up business data in blocks according to claim 5, characterized in that: The business data block backup method further includes a step of updating the probability threshold, which specifically includes: Obtaining an initial value and an update granularity value of the probability threshold; Grouping the storage nodes according to the first probability value and the probability threshold, and determining whether the grouping is successful; When the grouping is successful, the probability threshold of the round of cycles is subtracted from the update granularity value to obtain the updated probability threshold, and the storage nodes are grouped according to the first probability value and the probability threshold; When the grouping is unsuccessful, the probability threshold of the previous cycle is used as the final value of the probability threshold.

7. A method for backing up business data in blocks according to any one of claims 1 to 6, characterized in that: The performing block-by-block backing up of the service data to be backed up according to the plurality of storage node groups specifically includes: Splitting the service data to be backed up into multiple data blocks, and determining the storage node group corresponding to each data block; The data block is copied to obtain a plurality of backup data blocks, and the plurality of backup data blocks are then respectively stored in a plurality of the storage nodes in the corresponding storage node group.

8. A business data block backup system, characterized in that: include: A data acquisition module is used to obtain basic information of multiple storage nodes and operating performance data and fault information in a preset historical period; A prediction module, configured to train a node simultaneous failure prediction model according to the basic information, the operation performance data and the fault information, and predict a first probability value of simultaneous failure of two storage nodes within a next service cycle according to the node simultaneous failure prediction model; a node grouping module, configured to group the storage nodes according to the first probability value and a preset probability threshold to obtain a plurality of storage node groups, so that the first probability value corresponding to any two storage nodes in the same storage node group is less than or equal to the probability threshold; The block backup module is used to perform block backup of the service data to be backed up according to the plurality of storage node groups.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the business data block backup method as described in any one of claims 1 to 7 are realized.

10. A storage medium, the storage medium being a computer-readable storage medium, used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the business data block backup method as described in any one of claims 1 to 7.