Data backup method and device

Abstract

The invention discloses a data backup method and device, and relates to the field of data processing, and the method comprises the steps: obtaining node operation data of a to-be-backed-up node, and calculating a network quality score and a network stability score according to the node operation data; sampling data to be backed up by taking the data blocks as scales to obtain sampling data of the plurality of data blocks, and calculating a data feature score according to the sampling data; determining the backup data block capacity according to the network quality score, the network stability score and the data feature score; and according to the backup data block capacity, generating a backup strategy of the to-be-backed-up data, and backing up the to-be-backed-up data to the to-be-backed-up node according to the backup strategy. According to the method and the device, the more reasonable backup data block capacity is obtained by integrating the influence of the network factors and the data types, so that the more reasonable backup strategy is obtained, and the data backup efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a data backup method and device. Background Art

[0002] When backing up data in a large-scale distributed storage system, unstable factors such as fluctuations in network bandwidth will affect the performance, efficiency and reliability of data backup. Existing backup strategies are mainly based on fixed data block size backup strategies, which are difficult to adapt to complex and changeable network environments, resulting in insufficient performance and low efficiency of data backup. At the same time, existing backup strategies do not consider the degree of compatibility between different types of data and the data block size used for backup. When the data block is too large or the data type does not match the data block size, the efficiency of reading data will decrease, and the read-write consistency needs to be verified, which will lead to a decrease in data backup performance and efficiency. Therefore, how to comprehensively consider network factors and data types during data backup to improve data backup efficiency is still an urgent problem to be solved. Summary of the invention

[0003] The present application provides a data backup method and device to solve the technical problem that the prior art does not consider network factors and data types during backup, resulting in insufficient data backup efficiency.

[0004] According to a first aspect of the implementation manner of the present application, there is provided a data backup method, comprising:

[0005] Obtaining node operation data of the node to be backed up, and calculating a network quality score and a network stability score based on the node operation data;

[0006] Taking data blocks as a scale, sampling the data to be backed up is performed to obtain sampled data of multiple data blocks, and a data feature score is calculated based on the sampled data;

[0007] Determining the backup data block capacity according to the network quality score, the network stability score and the data feature score;

[0008] A backup strategy for the data to be backed up is generated according to the backup data block capacity, and the data to be backed up is backed up to the node to be backed up according to the backup strategy.

[0009] The present application first calculates the network quality score and the network stability score according to the node operation data of the node to be backed up, taking into account the network factors of the node to be backed up during the backup; then samples the backup data on a data block basis and calculates the data feature score, taking into account the data feature differences caused by the data type of the data to be backed up; then determines the backup data block capacity according to the network quality score, the network stability score and the data feature score, generates a backup strategy according to the backup data block capacity, and then performs backup according to the backup strategy, which can comprehensively consider the influence of network factors and data types, obtain a more reasonable backup data block capacity, and then obtain a more reasonable backup strategy, thereby improving the efficiency of data backup.

[0010] In certain embodiments of the present application, the calculating of the network quality score and the network stability score according to the node operation data specifically includes:

[0011] The node operation data includes network environment data of the network environment where the node to be backed up is located and historical network backup data of the node to be backed up;

[0012] Calculate the network quality score based on the network environment data;

[0013] Calculate the network stability score based on historical network backup data.

[0014] This application first calculates the network quality score based on the network environment data of the network environment where the node to be backed up is located, and then calculates the network stability score based on the historical network backup data of the node to be backed up, comprehensively considering the impact of the network environment and network stability on the backup, and then being able to obtain a more reasonable backup data block capacity, thereby obtaining a more reasonable backup strategy to improve the efficiency of data backup.

[0015] In certain embodiments of the present application, the calculating the network quality score according to the network environment data specifically includes:

[0016] The network environment data includes delay data, packet loss rate and bandwidth of the network environment at a preset first sampling number;

[0017] Calculating the average delay and average packet loss rate of the network environment according to the first sampling number, the delay data and the packet loss rate;

[0018] According to the first sampling number, taking the standard deviation of the delay data as the jitter of the network environment;

[0019] A network quality score is obtained by weighted calculation according to the average delay, the average packet loss rate, the bandwidth and the jitter.

[0020] The present application first calculates the average delay and the average packet loss rate according to the first sampling number, the delay data and the packet loss rate, and then uses the standard deviation of the delay data as the jitter, and then combines the average delay, the average packet loss rate, the jitter and the bandwidth to perform a weighted calculation to obtain a network quality score. By comprehensively scoring the network quality based on the delay, the packet loss rate, the jitter and the bandwidth, the comprehensiveness and rationality of the network quality score can be improved, and a more reasonable backup data block capacity can be obtained, thereby obtaining a more reasonable backup strategy to improve the efficiency of data backup.

[0021] In certain embodiments of the present application, the calculating of the network stability score according to the historical network backup data specifically includes:

[0022] Performing time series analysis on the historical network backup data based on a sliding window method to obtain a historical network change sequence; wherein the historical network change sequence includes a delay mean change sequence, a delay fluctuation change sequence, a packet loss rate mean change sequence, a packet loss rate fluctuation change sequence, a throughput mean change sequence, and a throughput fluctuation change sequence;

[0023] Based on a standard deviation detection algorithm, anomalies are eliminated from the historical network change sequence to obtain a first network change sequence;

[0024] Smoothing the first network change sequence based on an exponential smoothing method to obtain a second network change sequence;

[0025] Based on a periodic decomposition algorithm, a trend analysis is performed on the second network change sequence to obtain a network availability ratio;

[0026] A network stability score is calculated based on the historical network change sequence and the network availability ratio.

[0027] The present application first performs a time series analysis on the historical network backup data based on the sliding window method to obtain a historical network change sequence, and then performs anomaly elimination and smoothing based on the standard deviation detection algorithm and the exponential smoothing method to obtain a second network change sequence, which can improve the accuracy of the network change sequence, and performs a trend analysis on the second network change sequence based on the periodic decomposition algorithm to obtain a network availability ratio, and then combines the historical network change sequence and the network availability ratio to calculate a network stability score, which can improve the comprehensiveness and rationality of the network stability score, and then obtain a more reasonable backup data block capacity, thereby obtaining a more reasonable backup strategy to improve the efficiency of data backup.

[0028] In certain embodiments of the present application, the calculating of the data feature score according to the sampled data specifically includes:

[0029] Analyzing the sampled data to obtain compression ratios, repetitions, and access frequencies corresponding to the plurality of data blocks;

[0030] Clustering the sampled data according to a preset number of feature categories to obtain multiple feature clusters;

[0031] Corresponding to the multiple feature clusters, according to the compression rates, repetition rates and access frequencies of the multiple data blocks, a compression rate mean, a repetition rate mean and an access frequency mean of each feature cluster are calculated;

[0032] The data feature score of each feature cluster is calculated based on the mean compression rate, mean repetition rate and mean access frequency of each feature cluster.

[0033] The present application first analyzes the sampled data to obtain compression rate, repetition rate and access frequency, and then clusters the sampled data to obtain multiple feature clusters, which can classify multiple data blocks according to data features; then, corresponding to the multiple feature clusters, the compression rate mean, repetition rate mean and access frequency mean of each feature cluster are calculated according to the compression rate, repetition rate and access frequency, which can more reasonably reflect the data features of the multiple feature clusters, that is, the sets of data blocks of different categories, and then calculate the data feature score of each feature cluster, which can improve the comprehensiveness and rationality of the data feature score, and then obtain a more reasonable backup data block capacity, thereby obtaining a more reasonable backup strategy to improve the efficiency of data backup.

[0034] In certain embodiments of the present application, determining the backup data block capacity according to the network quality score, the network stability score, and the data feature score specifically includes:

[0035] Based on the hierarchical analysis method, weights are assigned to the network quality score, the network stability score, and the data feature scores of multiple feature clusters to obtain a score weight group corresponding to each feature cluster;

[0036] Corresponding to each feature cluster, obtaining a comprehensive score for each feature cluster according to the score weight group, the network quality score, the network stability score and the data feature score;

[0037] According to the comprehensive scores of each feature cluster, a plurality of initial data block capacities corresponding to the plurality of feature clusters are obtained;

[0038] The multiple initial data block capacities are modified according to the network stability scores to obtain multiple backup data block capacities.

[0039] The present application firstly distributes weights for the network quality score, the network stability score and the data feature scores of multiple feature clusters based on the hierarchical analysis method, and can obtain a more reasonable score weight group corresponding to each feature cluster, and then when calculating the comprehensive score of each feature cluster and determining the initial data block capacity, the rationality of the initial data block capacity can be improved, and then when the initial data block capacity is corrected according to the network stability score to obtain the backup data block capacity, the rationality of the backup data block capacity can be improved, thereby comprehensively considering the influence of network factors and data types on backup, and then obtaining a more reasonable backup strategy to improve the efficiency of data backup.

[0040] According to a second aspect of the implementation of the present application, there is provided a data backup device, including a first scoring module, a second scoring module, a capacity determination module and a data backup module;

[0041] The first scoring module is used to obtain node operation data of the node to be backed up, and calculate the network quality score and the network stability score according to the node operation data;

[0042] The second scoring module is used to sample the data to be backed up based on the data block as a scale, obtain sampled data of multiple data blocks, and calculate the data feature score based on the sampled data;

[0043] The capacity determination module is used to determine the backup data block capacity according to the network quality score, the network stability score and the data feature score;

[0044] The data backup module is used to generate a backup strategy for the data to be backed up according to the backup data block capacity, and back up the data to be backed up to the node to be backed up according to the backup strategy.

[0045] In certain embodiments of the present application, the first scoring module includes a network quality scoring submodule and a network stability scoring submodule;

[0046] The node operation data includes network environment data of the network environment where the node to be backed up is located and historical network backup data of the node to be backed up;

[0047] The network quality scoring submodule is used to calculate the network quality score according to the network environment data;

[0048] The network stability scoring submodule is used to calculate the network stability score based on historical network backup data.

[0049] In certain embodiments of the present application, the network quality scoring submodule includes a first calculation unit, a jitter calculation unit, and a quality scoring unit;

[0050] The network environment data includes delay data, packet loss rate and bandwidth of the network environment at a preset first sampling number;

[0051] The first calculation unit is used to calculate the average delay and the average packet loss rate of the network environment according to the first sampling number, the delay data and the packet loss rate;

[0052] The jitter calculation unit is used to use the standard deviation of the delay data as the jitter of the network environment according to the first sampling number;

[0053] The quality scoring unit is used to obtain a network quality score by weighted calculation according to the average delay, the average packet loss rate, the bandwidth and the jitter.

[0054] In certain embodiments of the present application, the network stability scoring submodule includes a sequence analysis unit, an anomaly elimination unit, a data smoothing unit, a trend analysis unit, and a stability scoring unit;

[0055] The sequence analysis unit is used to perform time series analysis on the historical network backup data based on a sliding window method to obtain a historical network change sequence; wherein the historical network change sequence includes a delay mean change sequence, a delay fluctuation change sequence, a packet loss rate mean change sequence, a packet loss rate fluctuation change sequence, a throughput mean change sequence, and a throughput fluctuation change sequence;

[0056] The anomaly elimination unit is used to eliminate anomalies from the historical network change sequence based on a standard deviation detection algorithm to obtain a first network change sequence;

[0057] The data smoothing unit is used to perform smoothing processing on the first network change sequence based on an exponential smoothing method to obtain a second network change sequence;

[0058] The trend analysis unit is used to perform trend analysis on the second network change sequence based on a periodic decomposition algorithm to obtain a network availability ratio;

[0059] The stability scoring unit is used to calculate a network stability score based on the historical network change sequence and the network availability ratio.

[0060] In certain embodiments of the present application, the second scoring module includes a sampling analysis submodule, a feature clustering submodule, a mean calculation submodule and a feature scoring submodule;

[0061] The sampling analysis submodule is used to analyze the sampled data to obtain compression ratio, repetition and access frequency corresponding to the multiple data blocks;

[0062] The feature clustering submodule is used to cluster the sampled data according to a preset number of feature categories to obtain multiple feature clusters;

[0063] The mean calculation submodule is used to calculate the compression rate mean, repetition rate mean and access frequency mean of each feature cluster according to the compression rate, repetition rate and access frequency of the multiple data blocks corresponding to the multiple feature clusters;

[0064] The feature scoring submodule is used to calculate the data feature score of each feature cluster according to the compression rate mean, repetition rate mean and access frequency mean of each feature cluster.

[0065] In certain embodiments of the present application, the capacity determination module includes a weight allocation submodule, a comprehensive scoring submodule, an initial capacity determination submodule, and a capacity correction submodule;

[0066] The weight allocation submodule is used to allocate weights to the network quality score, the network stability score and the data feature scores of multiple feature clusters based on the hierarchical analysis method to obtain a score weight group corresponding to each feature cluster;

[0067] The comprehensive scoring submodule is used to obtain a comprehensive score for each feature cluster according to the scoring weight group, the network quality score, the network stability score and the data feature score, corresponding to each feature cluster;

[0068] The initial capacity determination submodule is used to obtain a plurality of initial data block capacities corresponding to the plurality of feature clusters according to the respective comprehensive scores of each feature cluster;

[0069] The capacity correction submodule is used to correct the multiple initial data block capacities according to the network stability score to obtain multiple backup data block capacities.

[0070] The present application first calculates the network quality score and the network stability score according to the node operation data of the node to be backed up, taking into account the network factors of the node to be backed up during the backup; then samples the backup data on a data block basis and calculates the data feature score, taking into account the data feature differences caused by the data type of the data to be backed up; then determines the backup data block capacity according to the network quality score, the network stability score and the data feature score, generates a backup strategy according to the backup data block capacity, and then performs backup according to the backup strategy, which can comprehensively consider the influence of network factors and data types, obtain a more reasonable backup data block capacity, and then obtain a more reasonable backup strategy, thereby improving the efficiency of data backup. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1: A flow chart of a data backup method shown in certain embodiments of the present application;

[0072] Figure 2 : is a module structure diagram of a data backup device shown in certain embodiments of the present application. DETAILED DESCRIPTION

[0073] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below in conjunction with the accompanying drawings are exemplary and are only used to explain some embodiments of the present application, and should not be construed as limitations on the embodiments of the present application. Based on the embodiments shown in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0074] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, unless otherwise clearly and specifically defined, "multiple" and "several" mean two or more.

[0075] When backing up data in a large-scale distributed storage system, the existing backup method mainly uses a fixed data block size backup strategy. This backup method is easily affected by the unstable factors of the network environment during backup, resulting in insufficient data backup performance and low efficiency. At the same time, this backup strategy does not consider the degree of compatibility between the data type of the data to be backed up and the data block size used for backup, which easily leads to a decrease in data backup performance and efficiency. Therefore, how to comprehensively consider the network factors and data types during data backup to improve data backup efficiency is still an urgent problem to be solved.

[0076] Based on the above technical background, please refer to Figure 1 The present application embodiment provides a data backup method, including step S101 to step S104, each step is as follows:

[0077] Step S101: Obtain node operation data of the node to be backed up, and calculate a network quality score and a network stability score based on the node operation data.

[0078] In certain embodiments of the present application, the calculating of the network quality score and the network stability score according to the node operation data specifically includes:

[0079] The node operation data includes network environment data of the network environment where the node to be backed up is located and historical network backup data of the node to be backed up;

[0080] Calculate the network quality score based on the network environment data;

[0081] Calculate the network stability score based on historical network backup data.

[0082] In some optional implementations of the present application, exemplarily, the method for acquiring the network environment data of the network environment where the node to be backed up is located is specifically:

[0083] Within a preset time period, sending a first detection signaling to the backup node multiple times at a preset frequency and obtaining a first return signaling returned by the backup node in response to the first detection signaling;

[0084] The network environment data of the network environment where the node to be backed up is located is obtained according to the first return signaling.

[0085] Specifically, a network communication protocol such as ICMP and / or UDP may be used to send a first detection signaling to the node to be backed up to obtain a first return signaling returned by the node to be backed up, and network environment data of the corresponding network environment may be obtained according to the first return signaling.

[0086] More specifically, the preferred solution for the preset time period is 10 minutes, and the preferred solution for the preset frequency is 12 times / min.

[0087] This application first calculates the network quality score based on the network environment data of the network environment where the node to be backed up is located, and then calculates the network stability score based on the historical network backup data of the node to be backed up, comprehensively considering the impact of the network environment and network stability on the backup, and then being able to obtain a more reasonable backup data block capacity, thereby obtaining a more reasonable backup strategy to improve the efficiency of data backup.

[0088] In certain embodiments of the present application, the calculating the network quality score according to the network environment data specifically includes:

[0089] The network environment data includes delay data, packet loss rate and bandwidth of the network environment at a preset first sampling number;

[0090] Calculating the average delay and average packet loss rate of the network environment according to the first sampling number, the delay data and the packet loss rate;

[0091] According to the first sampling number, taking the standard deviation of the delay data as the jitter of the network environment;

[0092] A network quality score is obtained by weighted calculation according to the average delay, the average packet loss rate, the bandwidth and the jitter.

[0093] In some optional implementations of the present application, the preferred solution for the first sampling times is 120 times.

[0094] In some optional implementations of the present application, the preferred weight distribution schemes of the average delay, the average packet loss rate, the bandwidth and the jitter are 0.3, 0.3, 0.2 and 0.2.

[0095] In some optional implementations of the present application, the network quality score is obtained by weighted calculation according to the average delay, the average packet loss rate, the bandwidth and the jitter, specifically:

[0096]

[0097] Among them, S q Score network quality, delay avg , delay max-t is the average delay and the maximum tolerable delay, loss avg is the average packet loss rate, bandwidth, bandwidth max For bandwidth and maximum bandwidth, shake, shake max is the jitter and maximum jitter.

[0098] In some optional embodiments of the present application, delay max-t , bandwidth max , shake max The preferred solution is delay max-t =50ms, bandwidth max =1Mbps, shake max =10ms.

[0099] The present application first calculates the average delay and the average packet loss rate according to the first sampling number, the delay data and the packet loss rate, and then uses the standard deviation of the delay data as the jitter, and then combines the average delay, the average packet loss rate, the jitter and the bandwidth to perform a weighted calculation to obtain a network quality score. By comprehensively scoring the network quality based on the delay, the packet loss rate, the jitter and the bandwidth, the comprehensiveness and rationality of the network quality score can be improved, and a more reasonable backup data block capacity can be obtained, thereby obtaining a more reasonable backup strategy to improve the efficiency of data backup.

[0100] In certain embodiments of the present application, the calculating of the network stability score according to the historical network backup data specifically includes:

[0101] Performing time series analysis on the historical network backup data based on a sliding window method to obtain a historical network change sequence; wherein the historical network change sequence includes a delay mean change sequence, a delay fluctuation change sequence, a packet loss rate mean change sequence, a packet loss rate fluctuation change sequence, a throughput mean change sequence, and a throughput fluctuation change sequence;

[0102] Based on a standard deviation detection algorithm, anomalies are eliminated from the historical network change sequence to obtain a first network change sequence;

[0103] Smoothing the first network change sequence based on an exponential smoothing method to obtain a second network change sequence;

[0104] Based on a periodic decomposition algorithm, a trend analysis is performed on the second network change sequence to obtain a network availability ratio;

[0105] A network stability score is calculated based on the historical network change sequence and the network availability ratio.

[0106] In certain optional embodiments of the present application, the preferred embodiment of the sliding window of the sliding window method is 60 minutes.

[0107] In some optional implementations of the present application, the standard deviation detection algorithm is used to remove anomalies from the historical network change sequence to obtain a first network change sequence, which is specifically:

[0108] The standard deviation of each sequence in the historical network change sequence is calculated, and each sequence is tested. During the test, data points exceeding 3 times the standard deviation of the sequence are removed to obtain a first network change sequence.

[0109] In certain optional embodiments of the present application, the preferred solution for the smoothing factor of the exponential smoothing method is 1.

[0110] In certain optional implementations of the present application, the preferred solution of the periodic decomposition algorithm is the STL (Seasonal-Trend decomposition using LOESS) algorithm.

[0111] In some optional implementations of the present application, the network stability score is calculated based on the historical network change sequence and the network availability ratio, specifically:

[0112] S s =w1delay avg +w2dekay v +w3loss avg +w3loss v +w5cap avg +w6capv +w7ratio ava ;

[0113] Among them, S s The network stability score is w1, w2, w3, w4, w5, w6, and w7 are the delay mean weight, delay fluctuation weight, packet loss rate mean weight, packet loss rate fluctuation weight, throughput mean weight, throughput fluctuation weight, and network availability weight, respectively. avg , delay v is the delay mean and delay volatility, loss avg , loss v is the mean value of packet loss rate and the volatility of packet loss rate, cap avg , cap v is the throughput mean and throughput volatility, ratio ava is the network availability ratio.

[0114] In certain optional embodiments of the present application, the preferred scheme of w1, w2, w3, w4, w5, w6, and w7 is w1=0.2, w2=0.1, w3=0.2, w4=0.1, w5=0.2, w6=0.1, and w7=0.1.

[0115] The present application first performs a time series analysis on the historical network backup data based on the sliding window method to obtain a historical network change sequence, and then performs anomaly elimination and smoothing based on the standard deviation detection algorithm and the exponential smoothing method to obtain a second network change sequence, which can improve the accuracy of the network change sequence, and performs a trend analysis on the second network change sequence based on the periodic decomposition algorithm to obtain a network availability ratio, and then combines the historical network change sequence and the network availability ratio to calculate a network stability score, which can improve the comprehensiveness and rationality of the network stability score, and then obtain a more reasonable backup data block capacity, thereby obtaining a more reasonable backup strategy to improve the efficiency of data backup.

[0116] Step S102: sampling the data to be backed up based on the data block as a scale to obtain sampled data of a plurality of data blocks, and calculating a data feature score based on the sampled data.

[0117] In certain embodiments of the present application, the calculating of the data feature score according to the sampled data specifically includes:

[0118] Analyzing the sampled data to obtain compression ratios, repetitions, and access frequencies corresponding to the plurality of data blocks;

[0119] Clustering the sampled data according to a preset number of feature categories to obtain multiple feature clusters;

[0120] Corresponding to the multiple feature clusters, according to the compression rates, repetition rates and access frequencies of the multiple data blocks, a compression rate mean, a repetition rate mean and an access frequency mean of each feature cluster are calculated;

[0121] The data feature score of each feature cluster is calculated based on the mean compression rate, mean repetition rate and mean access frequency of each feature cluster.

[0122] In certain optional embodiments of the present application, the preferred schemes for clustering the sampled data include but are not limited to K-means clustering algorithm, hierarchical clustering algorithm, DBSCAN clustering algorithm and spectral clustering algorithm.

[0123] In some optional implementations of the present application, the data feature score of each feature cluster is calculated according to the compression rate mean, repetition rate mean and access frequency mean of each feature cluster, specifically:

[0124]

[0125] in, is the data feature score of the i-th feature cluster, t1, t2, t3 are the compression ratio weight, duplicate value weight and access frequency weight, ratio i , repeat i ,freq i is the mean compression ratio, mean repetition ratio and mean access frequency of the i-th feature cluster, ratio max , repeat max ,freq max The maximum compression ratio, maximum duplication and maximum access frequency.

[0126] In some optional embodiments of the present application, the preferred scheme of t1, t2, and t3 is t1=0.4, t2=0.3, t3=0.3, ratio max , repeat max ,freq max The preferred solution is ratio max =90%, repeat max =0.3%,freq max =120 qh.

[0127] The present application first analyzes the sampled data to obtain compression rate, repetition rate and access frequency, and then clusters the sampled data to obtain multiple feature clusters, which can classify multiple data blocks according to data features; then, corresponding to the multiple feature clusters, the compression rate mean, repetition rate mean and access frequency mean of each feature cluster are calculated according to the compression rate, repetition rate and access frequency, which can more reasonably reflect the data features of the multiple feature clusters, that is, the sets of data blocks of different categories, and then calculate the data feature score of each feature cluster, which can improve the comprehensiveness and rationality of the data feature score, and then obtain a more reasonable backup data block capacity, thereby obtaining a more reasonable backup strategy to improve the efficiency of data backup.

[0128] Step S103: determining the backup data block capacity according to the network quality score, the network stability score and the data feature score.

[0129] In certain embodiments of the present application, determining the backup data block capacity according to the network quality score, the network stability score, and the data feature score specifically includes:

[0130] Based on the hierarchical analysis method, weights are assigned to the network quality score, the network stability score, and the data feature scores of multiple feature clusters to obtain a score weight group corresponding to each feature cluster;

[0131] Corresponding to each feature cluster, obtaining a comprehensive score for each feature cluster according to the score weight group, the network quality score, the network stability score and the data feature score;

[0132] According to the comprehensive scores of each feature cluster, a plurality of initial data block capacities corresponding to the plurality of feature clusters are obtained;

[0133] The multiple initial data block capacities are modified according to the network stability scores to obtain multiple backup data block capacities.

[0134] In certain embodiments of the present application, the multiple initial data block capacities corresponding to the multiple feature clusters are obtained according to the comprehensive scores of each feature cluster, specifically:

[0135] According to the comprehensive score of each feature cluster, a plurality of initial data block capacities of each feature cluster are obtained by searching in a preset score capacity mapping table.

[0136] In certain implementations of the present application, the multiple initial data block capacities are modified according to the network stability score to obtain multiple backup data block capacities, specifically:

[0137] According to the network stability score, a correction coefficient is found in a preset stability coefficient mapping table, and the multiple initial data block capacities are corrected according to the correction coefficient to obtain multiple backup data block capacities.

[0138] The present application firstly distributes weights for the network quality score, the network stability score and the data feature scores of multiple feature clusters based on the hierarchical analysis method, and can obtain a more reasonable score weight group corresponding to each feature cluster, and then when calculating the comprehensive score of each feature cluster and determining the initial data block capacity, the rationality of the initial data block capacity can be improved, and then when the initial data block capacity is corrected according to the network stability score to obtain the backup data block capacity, the rationality of the backup data block capacity can be improved, thereby comprehensively considering the influence of network factors and data types on backup, and then obtaining a more reasonable backup strategy to improve the efficiency of data backup.

[0139] Step S104: generating a backup strategy for the data to be backed up according to the backup data block capacity, and backing up the data to be backed up to the node to be backed up according to the backup strategy.

[0140] In some optional implementations of the present application, generating a backup strategy for the data to be backed up according to the backup data block capacity, and backing up the data to be backed up to the node to be backed up according to the backup strategy, specifically:

[0141] Calculate the number of backup data blocks required for each feature cluster according to the backup data block capacity of each feature cluster and the data volume component of the data to be backed up in each feature cluster;

[0142] Dividing the data to be backed up according to multiple feature clusters to obtain multiple backup data groups;

[0143] The multiple backup data groups are backed up to the to-be-backed-up node according to the number of backup data blocks required by each feature cluster.

[0144] Compared with the prior art, the present application first calculates the network quality score and the network stability score according to the node operation data of the node to be backed up, taking into account the network factors of the node to be backed up during backup; then samples the backup data based on the data block and calculates the data feature score, taking into account the data feature differences caused by the data type of the data to be backed up; then determines the backup data block capacity according to the network quality score, the network stability score and the data feature score, generates a backup strategy according to the backup data block capacity, and then performs backup according to the backup strategy, which can comprehensively consider the influence of network factors and data types, obtain a more reasonable backup data block capacity, and then obtain a more reasonable backup strategy, thereby improving the efficiency of data backup.

[0145] Corresponding to the previous method, see Figure 2 , the embodiment of the present application provides a data backup device, including a first scoring module 210, a second scoring module 220, a capacity determination module 230 and a data backup module 240;

[0146] The first scoring module 210 is used to obtain node operation data of the node to be backed up, and calculate the network quality score and the network stability score according to the node operation data;

[0147] The second scoring module 220 is used to sample the data to be backed up based on the data block as a scale, obtain sampled data of multiple data blocks, and calculate the data feature score based on the sampled data;

[0148] The capacity determination module 230 is used to determine the backup data block capacity according to the network quality score, the network stability score and the data feature score;

[0149] The data backup module 240 is used to generate a backup strategy for the data to be backed up according to the backup data block capacity, and back up the data to be backed up to the node to be backed up according to the backup strategy.

[0150] In certain embodiments of the present application, the first scoring module 210 includes a network quality scoring submodule 211 and a network stability scoring submodule 212;

[0151] The node operation data includes network environment data of the network environment where the node to be backed up is located and historical network backup data of the node to be backed up;

[0152] The network quality scoring submodule 211 is used to calculate the network quality score according to the network environment data;

[0153] The network stability scoring submodule 212 is used to calculate a network stability score based on historical network backup data.

[0154] In certain embodiments of the present application, the network quality scoring submodule 211 includes a first calculation unit 2111, a jitter calculation unit 2112, and a quality scoring unit 2113;

[0155] The network environment data includes delay data, packet loss rate and bandwidth of the network environment at a preset first sampling number;

[0156] The first calculation unit 2111 is used to calculate the average delay and the average packet loss rate of the network environment according to the first sampling number, the delay data and the packet loss rate;

[0157] The jitter calculation unit 2112 is used to use the standard deviation of the delay data as the jitter of the network environment according to the first sampling number;

[0158] The quality scoring unit 2113 is used to obtain a network quality score by weighted calculation according to the average delay, the average packet loss rate, the bandwidth and the jitter.

[0159] In certain embodiments of the present application, the network stability scoring submodule 212 includes a sequence analysis unit 2121, an abnormality elimination unit 2122, a data smoothing unit 2123, a trend analysis unit 2124, and a stability scoring unit 2125;

[0160] The sequence analysis unit 2121 is used to perform time series analysis on the historical network backup data based on a sliding window method to obtain a historical network change sequence; wherein the historical network change sequence includes a delay mean change sequence, a delay fluctuation change sequence, a packet loss rate mean change sequence, a packet loss rate fluctuation change sequence, a throughput mean change sequence, and a throughput fluctuation change sequence;

[0161] The anomaly elimination unit 2122 is used to eliminate anomalies from the historical network change sequence based on a standard deviation detection algorithm to obtain a first network change sequence;

[0162] The data smoothing unit 2123 is used to smooth the first network change sequence based on an exponential smoothing method to obtain a second network change sequence;

[0163] The trend analysis unit 2124 is used to perform trend analysis on the second network change sequence based on a periodic decomposition algorithm to obtain a network availability ratio;

[0164] The stability scoring unit 2125 is used to calculate a network stability score according to the historical network change sequence and the network availability ratio.

[0165] In certain embodiments of the present application, the second scoring module 220 includes a sampling analysis submodule 221, a feature clustering submodule 222, a mean calculation submodule 223, and a feature scoring submodule 224;

[0166] The sampling analysis submodule 221 is used to analyze the sampled data to obtain the compression ratio, repetition and access frequency corresponding to the multiple data blocks;

[0167] The feature clustering submodule 222 is used to cluster the sampled data according to a preset number of feature categories to obtain multiple feature clusters;

[0168] The mean calculation submodule 223 is used to calculate the compression rate mean, repetition rate mean and access frequency mean of each feature cluster according to the compression rate, repetition rate and access frequency of the multiple data blocks corresponding to the multiple feature clusters;

[0169] The feature scoring submodule 224 is used to calculate the data feature score of each feature cluster according to the mean compression rate, the mean repetition rate and the mean access frequency of each feature cluster.

[0170] In certain embodiments of the present application, the capacity determination module 230 includes a weight allocation submodule 231, a comprehensive scoring submodule 232, an initial capacity determination submodule 233, and a capacity correction submodule 234;

[0171] The weight allocation submodule 231 is used to allocate weights to the network quality score, the network stability score and the data feature scores of multiple feature clusters based on the hierarchical analysis method to obtain a score weight group corresponding to each feature cluster;

[0172] The comprehensive scoring submodule 232 is used to obtain a comprehensive score for each feature cluster according to the scoring weight group, the network quality score, the network stability score and the data feature score.

[0173] The initial capacity determination submodule 233 is used to obtain a plurality of initial data block capacities corresponding to the plurality of feature clusters according to the respective comprehensive scores of each feature cluster;

[0174] The capacity correction submodule 234 is used to correct the multiple initial data block capacities according to the network stability score to obtain multiple backup data block capacities.

[0175] The present application first calculates the network quality score and the network stability score according to the node operation data of the node to be backed up, taking into account the network factors of the node to be backed up during the backup; then samples the backup data on a data block basis and calculates the data feature score, taking into account the data feature differences caused by the data type of the data to be backed up; then determines the backup data block capacity according to the network quality score, the network stability score and the data feature score, generates a backup strategy according to the backup data block capacity, and then performs backup according to the backup strategy, which can comprehensively consider the influence of network factors and data types, obtain a more reasonable backup data block capacity, and then obtain a more reasonable backup strategy, thereby improving the efficiency of data backup.

[0176] Adaptively, the embodiments of the present application also provide a computer device and a computer-readable storage medium.

[0177] The computer device comprises: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor;

[0178] Wherein, when the processor executes the computer program, a data backup method of the present application is implemented.

[0179] The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute a data backup method of the present application.

[0180] The above are some implementation methods of the present application, and the purpose, technical scheme and beneficial effects of the present application are further described in detail. It should be clear that the above-mentioned partial implementation methods of the present application cannot be understood as limiting the present application. It is particularly pointed out that for those skilled in the art, any changes, modifications, equivalent substitutions and variations made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A data backup method, characterized in that: include: Obtaining node operation data of the node to be backed up, and calculating a network quality score and a network stability score based on the node operation data; Taking data blocks as a scale, sampling the data to be backed up is performed to obtain sampled data of multiple data blocks, and a data feature score is calculated based on the sampled data; Determining the backup data block capacity according to the network quality score, the network stability score and the data feature score; A backup strategy for the data to be backed up is generated according to the backup data block capacity, and the data to be backed up is backed up to the node to be backed up according to the backup strategy.

2. A data backup method according to claim 1, characterized in that: The calculating of the network quality score and the network stability score according to the node operation data specifically includes: The node operation data includes network environment data of the network environment where the node to be backed up is located and historical network backup data of the node to be backed up; Calculate the network quality score based on the network environment data; Calculate the network stability score based on historical network backup data.

3. A data backup method according to claim 2, characterized in that: The calculating the network quality score according to the network environment data specifically includes: The network environment data includes delay data, packet loss rate and bandwidth of the network environment at a preset first sampling number; Calculating the average delay and average packet loss rate of the network environment according to the first sampling number, the delay data and the packet loss rate; According to the first sampling number, taking the standard deviation of the delay data as the jitter of the network environment; A network quality score is obtained by weighted calculation according to the average delay, the average packet loss rate, the bandwidth and the jitter.

4. A data backup method according to claim 2, characterized in that: The calculating the network stability score according to the historical network backup data specifically includes: Performing time series analysis on the historical network backup data based on a sliding window method to obtain a historical network change sequence; wherein the historical network change sequence includes a delay mean change sequence, a delay fluctuation change sequence, a packet loss rate mean change sequence, a packet loss rate fluctuation change sequence, a throughput mean change sequence, and a throughput fluctuation change sequence; Based on a standard deviation detection algorithm, anomalies are eliminated from the historical network change sequence to obtain a first network change sequence; Smoothing the first network change sequence based on an exponential smoothing method to obtain a second network change sequence; Based on a periodic decomposition algorithm, a trend analysis is performed on the second network change sequence to obtain a network availability ratio; A network stability score is calculated based on the historical network change sequence and the network availability ratio.

5. A data backup method according to claim 1, characterized in that: The calculating of the data feature score according to the sampled data specifically includes: Analyzing the sampled data to obtain compression ratios, repetitions, and access frequencies corresponding to the plurality of data blocks; Clustering the sampled data according to a preset number of feature categories to obtain multiple feature clusters; Corresponding to the multiple feature clusters, according to the compression rates, repetition rates and access frequencies of the multiple data blocks, a compression rate mean, a repetition rate mean and an access frequency mean of each feature cluster are calculated; The data feature score of each feature cluster is calculated based on the mean compression rate, mean repetition rate and mean access frequency of each feature cluster.

6. A data backup method according to claim 5, characterized in that: The determining of the backup data block capacity according to the network quality score, the network stability score and the data feature score specifically includes: Based on the hierarchical analysis method, weights are assigned to the network quality score, the network stability score, and the data feature scores of multiple feature clusters to obtain a score weight group corresponding to each feature cluster; Corresponding to each feature cluster, obtaining a comprehensive score for each feature cluster according to the score weight group, the network quality score, the network stability score and the data feature score; According to the comprehensive scores of each feature cluster, a plurality of initial data block capacities corresponding to the plurality of feature clusters are obtained; The multiple initial data block capacities are modified according to the network stability scores to obtain multiple backup data block capacities.

7. A data backup device, characterized in that: It includes a first scoring module, a second scoring module, a capacity determination module and a data backup module; The first scoring module is used to obtain node operation data of the node to be backed up, and calculate the network quality score and the network stability score according to the node operation data; The second scoring module is used to sample the data to be backed up based on the data block as a scale, obtain sampled data of multiple data blocks, and calculate the data feature score based on the sampled data; The capacity determination module is used to determine the backup data block capacity according to the network quality score, the network stability score and the data feature score; The data backup module is used to generate a backup strategy for the data to be backed up according to the backup data block capacity, and back up the data to be backed up to the node to be backed up according to the backup strategy.

8. A data backup device according to claim 7, characterized in that: The first scoring module includes a network quality scoring submodule and a network stability scoring submodule; The node operation data includes network environment data of the network environment where the node to be backed up is located and historical network backup data of the node to be backed up; The network quality scoring submodule is used to calculate the network quality score according to the network environment data; The network stability scoring submodule is used to calculate the network stability score based on historical network backup data.

9. A data backup device according to claim 8, characterized in that: The network quality scoring submodule includes a first calculation unit, a jitter calculation unit and a quality scoring unit; The network environment data includes delay data, packet loss rate and bandwidth of the network environment at a preset first sampling number; The first calculation unit is used to calculate the average delay and the average packet loss rate of the network environment according to the first sampling number, the delay data and the packet loss rate; The jitter calculation unit is used to use the standard deviation of the delay data as the jitter of the network environment according to the first sampling number; The quality scoring unit is used to obtain a network quality score by weighted calculation according to the average delay, the average packet loss rate, the bandwidth and the jitter.

10. A data backup device according to claim 8, characterized in that: The network stability scoring submodule includes a sequence analysis unit, an abnormality elimination unit, a data smoothing unit, a trend analysis unit and a stability scoring unit; The sequence analysis unit is used to perform time series analysis on the historical network backup data based on a sliding window method to obtain a historical network change sequence; wherein the historical network change sequence includes a delay mean change sequence, a delay fluctuation change sequence, a packet loss rate mean change sequence, a packet loss rate fluctuation change sequence, a throughput mean change sequence, and a throughput fluctuation change sequence; The anomaly elimination unit is used to eliminate anomalies from the historical network change sequence based on a standard deviation detection algorithm to obtain a first network change sequence; The data smoothing unit is used to perform smoothing processing on the first network change sequence based on an exponential smoothing method to obtain a second network change sequence; The trend analysis unit is used to perform trend analysis on the second network change sequence based on a periodic decomposition algorithm to obtain a network availability ratio; The stability scoring unit is used to calculate a network stability score based on the historical network change sequence and the network availability ratio.

11. A data backup device according to claim 7, characterized in that: The second scoring module includes a sampling analysis submodule, a feature clustering submodule, a mean calculation submodule and a feature scoring submodule; The sampling analysis submodule is used to analyze the sampled data to obtain compression ratio, repetition and access frequency corresponding to the multiple data blocks; The feature clustering submodule is used to cluster the sampled data according to a preset number of feature categories to obtain multiple feature clusters; The mean calculation submodule is used to calculate the compression rate mean, repetition rate mean and access frequency mean of each feature cluster according to the compression rate, repetition rate and access frequency of the multiple data blocks corresponding to the multiple feature clusters; The feature scoring submodule is used to calculate the data feature score of each feature cluster according to the compression rate mean, repetition rate mean and access frequency mean of each feature cluster.

12. A data backup device according to claim 11, characterized in that: The capacity determination module includes a weight allocation submodule, a comprehensive scoring submodule, an initial capacity determination submodule and a capacity correction submodule; The weight allocation submodule is used to allocate weights to the network quality score, the network stability score and the data feature scores of multiple feature clusters based on the hierarchical analysis method to obtain a score weight group corresponding to each feature cluster; The comprehensive scoring submodule is used to obtain a comprehensive score for each feature cluster according to the scoring weight group, the network quality score, the network stability score and the data feature score, corresponding to each feature cluster; The initial capacity determination submodule is used to obtain a plurality of initial data block capacities corresponding to the plurality of feature clusters according to the respective comprehensive scores of each feature cluster; The capacity correction submodule is used to correct the multiple initial data block capacities according to the network stability score to obtain multiple backup data block capacities.

Images