Cloud-based data backup system and method

Abstract

The invention relates to the technical field of computer data backup, in particular to a cloud-based data backup system and method, and the system comprises a data load monitoring and partitioning module which constructs a feature set of computer backup data based on storage nodes in a cloud environment, recognizes a data set with similar features, and carries out data backup according to the difference degree between feature parameters; and judging whether the feature set of the computer backup data needs to be re-divided or not to obtain partition data of the computer backup data. According to the invention, through real-time monitoring and clustering analysis of the data features, the backup process is more intelligent and efficient. The dynamic adjustment of the feature set enables data with different access frequencies and importance to automatically obtain corresponding priorities. According to the method, the backup of the key data can be preferentially completed in the peak period, the risk that the key data is delayed to be processed in the case of resource shortage is reduced, meanwhile, the idle period is fully utilized to process the low-priority data, and the resource utilization efficiency is optimized.

Description

Technical Field

[0001] The present invention relates to the technical field of computer data backup, and in particular to a cloud-based data backup system and method. Background Art

[0002] Computer data backup refers to the technology of copying data from the primary storage device to other storage media (such as disks, tapes, cloud storage, etc.) in order to restore the data when it is lost, damaged or suffers an accident. It is widely used by individual users, enterprises and institutions to ensure the security and recoverability of data. Data backup technology usually covers a variety of backup methods, such as full backup, incremental backup and differential backup, to meet different recovery time requirements and storage capacity limitations.

[0003] The data backup system is used to automatically perform data backup operations. It can replicate data according to a predetermined strategy and schedule, and provide monitoring, management, and recovery functions to simplify the backup process and improve efficiency. This protects critical data and enables rapid recovery after accidental damage or deletion, ensuring data integrity and availability.

[0004] It is difficult for existing technologies to dynamically adjust partitions according to changes in data characteristics, and it is impossible to flexibly assign priorities to data with different characteristics during the backup process. For example, when certain data grows rapidly or is accessed frequently, it is difficult to adjust its backup priority in a timely manner, causing critical data to compete with general data for resources during peak hours, increasing the risk of delayed backup or data loss. In addition, existing technologies usually use a full backup method, which is unable to back up only the updated part when there are slight changes in the data block. Therefore, even if there are only a few changes, the entire data set will be backed up again, resulting in excessive consumption of storage space and bandwidth resources. The bandwidth management of existing systems is difficult to cope with fluctuations in bandwidth demand in backup tasks, which may result in insufficient bandwidth during peak hours, affecting the completion speed of backup, or idle bandwidth during off-peak hours, wasting network resources. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a cloud-based data backup system and method.

[0006] In order to achieve the above object, the present invention adopts the following technical solution: A cloud-based data backup system includes:

[0007] The data load monitoring and partitioning module builds a feature set of computer backup data based on storage nodes in the cloud environment, identifies data sets with similar features, and determines whether the feature set of computer backup data needs to be re-partitioned according to the difference between feature parameters to obtain computer backup data partition data;

[0008] The priority partition backup scheduling module prioritizes the partitions based on the computer backup data partition data, builds a partition priority list, determines the backup task and frequency of each partition according to the partition priority list, and obtains a partition status adjustment record;

[0009] The incremental hash chain difference detection module creates an initial incremental hash chain based on the partition status adjustment record, and updates the incremental hash chain. By comparing the hash value of the chain tail with the previous version, the change status of the backup data block is determined to obtain the incremental backup result.

[0010] The bandwidth prediction and dynamic optimization module obtains the bandwidth consumption data of each period in the backup process in real time according to the execution process of the incremental backup in the incremental backup result, and predicts the bandwidth demand before the next backup task is executed, and dynamically adjusts the bandwidth allocation according to the prediction result to obtain the computer backup data backup optimization configuration result.

[0011] As a further solution of the present invention, the step of obtaining the data set having similar features is specifically as follows:

[0012] Based on the storage nodes in the cloud environment, the backup data increment and access frequency data of the nodes are obtained by extracting log records, the growth trend of the backup data is analyzed, and the storage path information is extracted to generate a feature set of computer backup data;

[0013] According to the feature set of the computer backup data, the similarity of the growth rate and the access frequency is evaluated, the feature differences between the data records are compared, and similar data are divided into the same group to obtain a data set with similar features.

[0014] As a further solution of the present invention, the step of obtaining the partition data of the computer backup data is specifically as follows:

[0015] Based on the data set with similar characteristics, the formula is adopted:

[0016]

[0017] Calculate the comprehensive difference ΔF of the characteristic parameters between the current period and the previous period to obtain the difference information between the characteristic parameters;

[0018] Among them, f i,t represents the value of the i-th characteristic parameter in the current period t, f i,t-1 represents the i-th characteristic parameter value in the previous period t-1, and n represents the total number of characteristic parameters;

[0019] According to the difference information between the characteristic parameters, the difference between the current period and the previous period is evaluated, and by comparing the difference degree with a preset threshold, it is determined whether the characteristic set needs to be re-divided to obtain the computer backup data partition data.

[0020] As a further solution of the present invention, the step of obtaining the partition status adjustment record is specifically as follows:

[0021] According to the computer backup data partition data, the formula is used:

[0022]

[0023] Calculate the priority score P of each partition and integrate the data to get the partition priority list;

[0024] Among them, S is the storage capacity of the partition, T s is the total storage capacity, B is the partition backup frequency, T b is the average backup frequency, W s and W b are weight factors for storage occupancy ratio and backup frequency ratio respectively;

[0025] According to the partition priority list, the backup tasks of the partitions are sorted and allocated according to the priority scores, the backup status information of each partition is automatically updated, and the adjustment record of the partition status is generated.

[0026] As a further solution of the present invention, the acquisition step of updating the incremental hash chain is specifically as follows:

[0027] Based on the partition status adjustment record, filter the data blocks of the high priority partition, generate basic hash values ​​for the data blocks, and add the hash values ​​to the incremental hash chain in sequence to establish an initial incremental hash chain of the current data block state;

[0028] Based on the initial incremental hash chain of the current data block state, newly added or updated data blocks in the partition are identified, new hash values ​​of the data blocks are generated and appended to the end of the hash chain, and incremental backup update records are generated.

[0029] As a further solution of the present invention, the step of obtaining the incremental backup result is specifically as follows:

[0030] Based on the incremental backup update record, read the latest hash value at the end of the chain and compare it with the hash value at the end of the previous version of the chain to determine whether the contents are consistent and generate a hash chain comparison result;

[0031] According to the hash chain comparison result, a list of data blocks with marked changes is extracted in the next backup cycle, and a backup operation is performed on the data blocks in the list, and the incremental backup log is updated to obtain an incremental backup result.

[0032] As a further solution of the present invention, the steps of obtaining the computer backup data backup optimization configuration result are specifically:

[0033] Based on the execution process data of the incremental backup results, the bandwidth consumption of each period is obtained in real time, the bandwidth usage status is recorded and summarized regularly, and a bandwidth load status set is established;

[0034] According to the bandwidth load state set, the formula is adopted:

[0035]

[0036] Calculate the average change in bandwidth usage ΔC avg , get the bandwidth change trend analysis results in all time periods;

[0037] Where n′ is the total number of time periods, C i′ is the bandwidth usage in the i′th period, C i′-1 is the bandwidth usage in the previous period;

[0038] According to the bandwidth change trend analysis results in all the time periods, the formula is used:

[0039] Q next =D current +(L×ΔC avg )

[0040] Calculate the predicted bandwidth requirement Q for the next backup task next , dynamically adjust bandwidth allocation according to the prediction results, and obtain the optimal configuration result of computer backup data backup;

[0041] Among them, D current is the bandwidth usage of the current backup task, and L is the estimated backup task duration.

[0042] A cloud-based data backup method is provided, wherein the cloud-based data backup method is performed based on the cloud-based data backup system, and comprises the following steps:

[0043] S1: Based on the cloud storage nodes, the growth rate, access frequency and storage location information of the backup data are extracted, the feature set of the computer backup data is constructed, and the feature similarity is evaluated and then the grouping is screened to obtain the partition data of the computer backup data;

[0044] S2: based on the computer backup data partition data, extract access frequency and importance information, sort the partitions by importance to generate a priority list, assign backup tasks and frequencies, and obtain partition status adjustment records;

[0045] S3: Based on the partition status adjustment record, create an initial hash chain and generate a basic hash value, update the tail of the hash chain, and compare the update status of the data blocks recorded by the hash change record at the tail of the chain in sequence to obtain an incremental backup result;

[0046] S4: Based on the incremental backup result, the bandwidth consumption in each period is collected in real time, a bandwidth load state set is constructed, the bandwidth trend change is analyzed, and a bandwidth usage prediction sequence is obtained;

[0047] S5: Based on the bandwidth usage prediction sequence, dynamically adjust bandwidth allocation, optimize task bandwidth priority, allocate bandwidth to cover task requirements, and generate computer backup data backup optimization configuration results.

[0048] Compared with the prior art, the advantages and positive effects of the present invention are:

[0049] In the present invention, the backup process is made more intelligent and efficient through real-time monitoring and cluster analysis of data features. The dynamic adjustment of the feature set enables data of different access frequencies and importance to automatically obtain corresponding priorities. It ensures that the backup of key data can be completed in priority during peak hours, reduces the risk of delayed processing of key data when resources are tight, and makes full use of idle time to process low-priority data, thereby optimizing resource utilization efficiency. In addition, the incremental hash chain mechanism effectively tracks the changes of data blocks, and only performs incremental backup on the changed data blocks, which significantly reduces the repeated consumption of storage and bandwidth resources. By predicting bandwidth usage, bandwidth resource allocation can be optimized according to historical data, ensuring the smooth execution of backup tasks under different load conditions, and avoiding bandwidth congestion during peak hours and waste during idle time. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a system flow chart of the present invention;

[0051] Figure 2 A flow chart for identifying data sets with similar characteristics for the present invention;

[0052] Figure 3 A flowchart of obtaining computer backup data partition data for the present invention;

[0053] Figure 4 A flow chart for obtaining a partition state adjustment record for the present invention;

[0054] Figure 5 A flow chart of updating the incremental hash chain of the present invention;

[0055] Figure 6 A flowchart of the incremental backup result obtained by the present invention;

[0056] Figure 7The present invention obtains a flow chart of computer backup data backup optimization configuration results. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0058] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0059] See also Figure 1 , the present invention provides a technical solution: a cloud-based data backup system includes.

[0060] The data load monitoring and partitioning module collects the computer backup data growth rate, access frequency and data storage location information based on the storage nodes in the cloud environment, builds a feature set of the computer backup data, performs cluster analysis on the data feature set, identifies data sets with similar features, and manages data with different features. By comparing the feature set of the computer backup data collected in each period with the feature set of the computer backup data stored in the previous period, it is determined whether the feature set of the computer backup data needs to be re-divided according to the difference between the feature parameters, and the computer backup data partition data is obtained;

[0061] The priority partition backup scheduling module prioritizes the partitions based on the computer backup data partition data, according to the partition access frequency information and the importance of the current computer backup data, builds a partition priority list, determines the backup task and frequency of each partition according to the partition priority list, completes the backup task of the high-priority partition first during the peak period, performs the backup of the low-priority partition during the idle period, and obtains the partition status adjustment record;

[0062] The incremental hash chain difference detection module creates an initial incremental hash chain based on the partition state adjustment record, which is used to record the hash value of the computer backup data block of the high priority partition, generates a basic hash value for the computer backup data block of the high priority partition, and adds it to the incremental hash chain in sequence. After that, a new hash value is generated for the newly added or updated data block in the partition, and it is preferentially attached to the end of the incremental hash chain to update the incremental hash chain. By comparing the hash value of the chain end with the previous version, the change state of the backup data block is judged, the change of the computer backup data is recorded, and the changed backup data block is marked for incremental backup record to obtain the incremental backup result;

[0063] The bandwidth prediction and dynamic optimization module obtains the bandwidth consumption data of each period in the backup process in real time according to the execution process of incremental backup in the incremental backup result, records the bandwidth usage status, establishes a bandwidth load status set, obtains the bandwidth usage prediction sequence by calculating the bandwidth usage change trend of each period in the bandwidth load status set, calls the bandwidth usage prediction sequence, predicts the bandwidth demand before the next backup task is executed, dynamically adjusts the bandwidth allocation according to the prediction result, controls the bandwidth usage priority, and obtains the computer backup data backup optimization configuration result;

[0064] The computer backup data partition data includes the data partition number, characteristic parameters of each partition and the storage location of the data in the partition. The partition status adjustment record includes the partition priority level, the partition backup schedule and the backup frequency corresponding to each partition. The incremental backup results include the initial hash chain, the hash records of the newly added or updated data blocks of the high-priority partition and the change status identification of the data blocks. The computer backup data backup optimization configuration results include the bandwidth usage prediction sequence, bandwidth allocation priority setting and dynamic bandwidth allocation plan.

[0065] See also Figure 2 , the steps to identify data sets with similar characteristics are as follows:

[0066] Based on the storage nodes in the cloud environment, the backup data increment and access frequency data of the nodes are obtained by extracting log records, the growth trend of the backup data is analyzed, and the storage path information is extracted to generate a feature set of computer backup data;

[0067] First, the growth rate, access frequency and data storage location information of computer backup data are collected from the log system of each storage node. The log record is periodically extracted by calling the cloud storage system log interface (for example, using the REST API interface) to obtain the backup data increment of each node and analyze the growth trend of each backup data. The growth rate is extracted by comparing the total amount of data storage in adjacent time periods for incremental calculation, using hourly or daily units to calculate the growth rate after calculating the increment. The access frequency is obtained by reading the number of accesses in the access log, extracting the access timestamp of the backup data and accumulating it, and summarizing it on a daily basis to obtain frequency statistics. The data storage location information extracts the backup file path information from the directory structure of each storage node, and combines the physical or logical storage path of each node to record the accurate storage location information of the backup file. The above operations finally generate a feature set of computer backup data for subsequent analysis.

[0068] Based on the feature set of computer backup data, the similarity of growth rate and access frequency is evaluated, the feature differences between data records are compared, similar data are divided into the same group, and a data set with similar features is obtained;

[0069] When clustering the collected computer backup data feature set, the two key features of growth rate and access frequency are first directly called. By setting a specific similarity threshold, the similarity between the data is evaluated one by one, so that the data with similar features can be divided into the same group. In the specific operation process, the system gradually forms data groups using the hierarchical clustering method. First, the system compares the growth rate and access frequency characteristics of each data record with other records one by one, and determines whether it meets the similarity threshold by calculating the feature difference between the two records. If the difference is within the set similarity range, the two records are initially classified into one group. For example, suppose that the growth rate of a backup record A is 10GB / day and the access frequency is 50 times / day; the growth rate of another record B is 11GB / day and the access frequency is 52 times / day. The system determines that the feature difference between A and B is within the allowable range, and classifies A and B into the same group; if there is a record C, its growth rate is 20GB / day and the access frequency is 5 times / day, and the feature of C deviates significantly from A and B, so C is divided into another group separately. The system continues this process, matching features for each data record to ensure that each data point has highly similar features to the data in the same group. After forming the preliminary data groups, the system calculates the mean of the growth rate and access frequency of each group to further check the internal consistency of the data groups. If the features of some data deviate greatly from the mean within the group, they are reallocated to other groups or formed into new groups separately. This step refines the clustering results in an iterative manner, and finally obtains a clear grouping structure. Each data group is given an identification label to indicate the similarity of the data within the group.

[0070] See also Figure 3 , the specific steps for obtaining computer backup data partition data are:

[0071] Based on a data set with similar characteristics, the formula is used:

[0072]

[0073] Calculate the comprehensive difference ΔF of the characteristic parameters between the current period and the previous period to obtain the difference information between the characteristic parameters;

[0074] Among them, ΔF is used to reflect the overall degree of feature change, f i,t represents the value of the i-th characteristic parameter in the current period t, such as the growth rate or access frequency of backup data, f i,t-1 represents the i-th characteristic parameter value of the previous period t-1, which is compared with the corresponding characteristic parameter of the current period. n represents the total number of characteristic parameters, including all the characteristics that need to be analyzed. f i,t and f i,t-1 It is obtained by extracting the records of backup data from the log system of the storage node. For example, by analyzing the log data of a week, the daily growth rate and access frequency are calculated to form the characteristic parameters of the current and previous periods.

[0075] For example, in a certain period of time, the characteristic parameters recorded by the system are: the growth rate f in the current period 1,t =15GB / dayCurrent period access frequency f 2,t = 50 times / day. In the previous period, the characteristic parameters recorded are: the growth rate f in the previous period 1,t-1 = 12GB / day Access frequency in the previous period f 2,t-1 =45 times / day.

[0076] After substituting into the formula, the calculation process is as follows:

[0077]

[0078] The results show that the feature set of the current period has changed significantly compared with the previous period, with a difference of about 5.83.

[0079] According to the difference information between the characteristic parameters, the difference between the current period and the previous period is evaluated, and by comparing the difference degree with the preset threshold, it is determined whether the characteristic set needs to be re-divided to obtain the computer backup data partition data;

[0080] First, the difference ΔF obtained is about 5.83. Next, compare this difference with the preset threshold, for example, assuming that the threshold is set to 5.0. If ΔF exceeds this threshold, the system needs to re-divide the feature set. The specific analysis process is as follows: In the current period, the growth rate of records is 15GB / day, and the access frequency is 50 times / day; in the previous period, the growth rate is 12GB / day, and the access frequency is 45 times / day. Through calculation, it is found that ΔF shows a significant change in backup data management between the two periods, which means that the data usage of the storage node has fluctuated greatly. Since ΔF=5.83 is greater than the preset threshold of 5.0, the system recognizes this change and decides to execute the re-division process. In this process, the system will screen all feature data with large differences, first comparing the change trends of the feature parameters of the current period and the previous period, paying special attention to the changes in the growth rate and access frequency. For example, if the access frequency of a certain set of data increases significantly during the current period, but its growth rate does not increase accordingly, it may mean that the storage requirements of the data have changed, so it needs to be re-divided into different data groups to better reflect its characteristics and optimize resource allocation.

[0081] See also Figure 4 , the specific steps for obtaining partition status adjustment records are:

[0082] According to the computer backup data partition data, use the formula:

[0083]

[0084] Calculate the priority score P of each partition and integrate the data to get the partition priority list;

[0085] Among them, P reflects the priority of each partition in the backup scheduling. The higher the score, the higher the priority of the partition in the backup task. S is the storage capacity of the partition (in GB), which indicates the storage space occupied by the partition in the storage node. The storage capacity is obtained by monitoring the data capacity on the storage node. For example, the current storage occupancy of the partition can be directly queried by calling the cloud storage interface. T s It is the total storage capacity of the system (in GB). It is used to standardize the storage capacity of each partition to facilitate horizontal comparison of the occupancy of different partitions. This parameter is obtained by monitoring the total storage resources of the entire system, for example, directly obtaining the total storage capacity data from the cloud storage management system. is the storage occupancy ratio, which indicates the relative proportion of the partition storage in the total system storage. B is the partition backup frequency (in times / day), which indicates the average number of backups of the partition in a specific time period. The backup frequency can be counted through historical records. Log analysis tools are often used to identify the backup status of partitions over a period of time. bis the average backup frequency of the system (in times / day), which is used to standardize the backup frequency of each partition and reflect the overall backup activity level of the system. It is obtained by dividing the total number of backups of all partitions in a period of time by the number of partitions. s and W b They are weight factors for the storage occupancy ratio and the backup frequency ratio, respectively, and are used to adjust the influence of the two in the priority score. The weight factors are set according to the partition management strategy, for example, by analyzing the backup history data and setting them according to the actual situation of the partition occupying system resources.

[0086] For example, if the storage capacity of a partition is S = 200 GB, the total storage capacity of the system is T s = 1000GB, partition backup frequency B = 4 times / day, system average backup frequency T b = 2 times / day, and set the weight W s =0.6 and W b =0.4. The calculation process of the priority score P is as follows:

[0087]

[0088] P = 0.12 + 0.8 = 0.92

[0089] The results showed that the calculated priority score was P = 0.92.

[0090] According to the partition priority list, the backup tasks of the partitions are sorted and assigned according to the priority scores, the backup status information of each partition is automatically updated, and the adjustment record of the partition status is generated;

[0091] In the partition priority score, assume that the priority score of partition A is 0.92, while the score of partition B is 0.5. The system uses this score to determine the partition to be backed up first. For example, the storage capacity of partition A is 200GB, accounting for 20% of the total system storage, and it is backed up 4 times a day in the recent period, while the average backup frequency of the system is 2 times a day. This shows that the backup demand of partition A is high and it occupies a large storage space, so it gets a higher priority (0.92) in the score. This high score indicates that partition A has a high demand for system resources, so it will be prioritized to be backed up during peak hours during backup scheduling to ensure the security of important data. In contrast, the storage capacity of partition B is only 50GB, accounting for 5% of the system storage, and the backup frequency is 1 time a day, which is lower than the average backup frequency of the system. Therefore, the priority score of partition B is low (0.5), and the system will schedule it to be backed up during off-peak hours to reduce the occupation of system resources. This scoring and scheduling method ensures that partition A with high demand can complete the backup task in the most appropriate time period to ensure timely protection of data, while partition B with low demand completes the backup when system resources are idle, achieving reasonable allocation of resources.

[0092] See also Figure 5 , the specific steps for updating the incremental hash chain are:

[0093] Based on the partition status adjustment record, filter the data blocks of high-priority partitions, generate basic hash values ​​for the data blocks, and add the hash values ​​to the incremental hash chain in sequence to establish the initial incremental hash chain of the current data block status;

[0094] Based on identifying the data blocks of high-priority partitions, a basic hash value is generated for each data block. For example, suppose partition A contains three data blocks, namely Block1, Block2, and Block3. The system first generates a basic hash value (such as H1) for the content of Block1, and then generates hash values ​​H2 and H3 for Block2 and Block3 respectively. These hash values ​​are added to the initial incremental hash chain in sequence to form a chain structure H1->H2->H3, which represents the current data state of partition A. This initial incremental hash chain serves as the baseline state of partition A, which facilitates subsequent data change tracking.

[0095] Based on the initial incremental hash chain of the current data block status, identify the newly added or updated data blocks in the partition, generate a new hash value for the data block and append it to the end of the hash chain, and generate an incremental backup update record;

[0096] After the incremental hash chain is created, the system continues to incrementally update the newly added or updated data blocks in the partition. For example, in the next backup cycle, the system detects that Block2 of partition A has changed, and a new data block Block4 has been added. The system will regenerate a new hash value (assuming it is H2') for the updated Block2, and a hash value H4 for the newly added Block4, and then append H2' and H4 to the end of the incremental hash chain first, forming an updated chain H1->H2->H3->H2'->H4, and generating an incremental backup update record.

[0097] See also Figure 6 , the specific steps for obtaining incremental backup results are:

[0098] Based on the incremental backup update record, read the latest hash value at the end of the chain and compare it with the hash value at the end of the previous version of the chain to determine whether the content is consistent and generate a hash chain comparison result;

[0099] After the chain update is completed, the system will compare the latest hash value of the chain end (i.e. H4) with the hash value of the chain end of the previous version (i.e. H3). The specific process of the comparison operation is: the system first reads the chain end hash value H3 from the previous version of the incremental hash chain, and then reads the hash value H4 of the current chain end from the latest incremental hash chain, and directly compares the hash value contents of H3 and H4 to determine whether they are consistent. If the hash values ​​of H3 and H4 are different, it means that the data of partition A has been updated during this backup cycle. The system records these changes and marks the updated data block Block2 and the newly added data block Block4 for incremental backups that only contain changed or newly added data blocks in subsequent backup tasks, thereby improving backup efficiency.

[0100] According to the hash chain comparison result, a list of data blocks with marked changes is extracted in the next backup cycle, and a backup operation is performed on the data blocks in the list, and the incremental backup log is updated to obtain the incremental backup result;

[0101] Based on the generated incremental backup update record, the system performs incremental backup operations on the data blocks of the specified partition in the next backup cycle. Specifically, the system first extracts the marked changed Block2 and the newly added Block4 from the incremental backup record, and puts them in the list to be backed up. In the actual backup operation, the system only backs up the data blocks in the backup list and skips other unchanged data blocks. For example, the change information of Block2 and the new information of Block4 are loaded into the backup task, and the system will read the latest content of these two data blocks in turn and save them to the backup file. After the backup is completed, the system updates the incremental backup log to record the completion time of the backup and detailed information on the backup content, providing an updated backup reference for subsequent recovery and data management.

[0102] See also Figure 7 , the specific steps for obtaining the computer backup data backup optimization configuration results are:

[0103] Based on the execution process data of the incremental backup results, the bandwidth consumption of each period is obtained in real time, the bandwidth usage status is recorded and summarized regularly, and a bandwidth load status set is established;

[0104] Based on the execution process data in the incremental backup results, the system obtains the bandwidth consumption of each period of the backup process in real time. In specific operations, the system automatically records the bandwidth usage of the current period at fixed time intervals by calling the bandwidth monitoring tool, and stores the data in the bandwidth log. To ensure data integrity, the system will set checkpoints to verify and re-collect missing or abnormal bandwidth data to ensure that the bandwidth status information of each period is fully collected. Next, the system extracts the bandwidth usage records of each period from the bandwidth log one by one, summarizes and organizes the data, and constructs the bandwidth usage data of each period into a bandwidth load status set. The bandwidth status data contained in the set will be marked with the period information so that it can be called according to the period sequence in subsequent analysis.

[0105] According to the bandwidth load state set, the formula is used:

[0106]

[0107] Calculate the average change in bandwidth usage ΔC avg , get the bandwidth change trend analysis results in all time periods;

[0108] Where, ΔC avg represents the overall trend of bandwidth usage in all time periods, n′ is the total number of time periods, C i′ is the bandwidth usage in the i′th period, which is directly read from the bandwidth usage data record through the real-time monitoring tool. i′-1 It is the bandwidth usage of the previous period, which is calculated by difference with the bandwidth usage of the current period to calculate the change in each period.

[0109] For example, the system records the bandwidth usage of 4 time periods, namely 100MB / s, 130MB / s, 120MB / s and 140MB / s. The system first extracts data from the bandwidth monitoring records for each time period and calculates the bandwidth change in adjacent time periods. Substitute it into the formula for calculation:

[0110]

[0111] The results show that the average bandwidth usage increased by 10MB / s per period.

[0112] Get the average change in bandwidth usage ΔC avg After that, the system generates a bandwidth usage prediction sequence by calculating the bandwidth usage prediction value for each time period. current As a starting point, by superimposing the change trend of each period, the bandwidth usage of each period in the future is predicted in turn. For example, assuming that the bandwidth usage of the current period is 150MB / s, the average change ΔC avg The system can generate a bandwidth usage prediction sequence as follows: predicted bandwidth usage in period 1: 150+10=160MB / s, predicted bandwidth usage in period 2: 160+10=170MB / s, predicted bandwidth usage in period 3: 170+10=180MB / s. In this way, a prediction sequence of 150, 160, 170, 180 is generated to guide the prediction and allocation of the next bandwidth demand.

[0113] According to the bandwidth change trend analysis results in all time periods, the formula is used:

[0114] Q next =D current +(L×ΔC avg )

[0115] Calculate the predicted bandwidth requirement Q for the next backup task next , dynamically adjust bandwidth allocation according to the prediction results, and obtain the optimal configuration result of computer backup data backup;

[0116] Among them, D current is the bandwidth usage of the current backup task, which is obtained through real-time monitoring of the current bandwidth usage. L is the estimated backup task duration, which indicates the number of time periods for future backup tasks, which can be determined based on the average duration data of past backup tasks.

[0117] If the current bandwidth usage is set to D current The bandwidth is 150MB / s, the estimated duration of the next backup cycle is L, which is 3 periods, and the average bandwidth change is ΔC. avg The speed is 10MB / s. Substituting it into the formula, the calculation is as follows:

[0118] Q next =150+(3×10)=150+30=180MB / s

[0119] The result shows that the next backup task is predicted to require a bandwidth of 180 MB / s.

[0120] Based on the generated bandwidth usage prediction sequence, the prediction value Q in the sequence is called nextTo dynamically adjust bandwidth allocation and control bandwidth usage priority to ensure efficient execution of backup tasks under different bandwidth demand conditions. For example, if the predicted bandwidth demand Q next If the system bandwidth is 180MB / s and the current available bandwidth of the system is 200MB / s, the system will give priority to the backup task and reserve the remaining bandwidth of 20MB / s for other tasks; if the predicted bandwidth demand exceeds the available bandwidth, for example, 220MB / s is required, the system will give priority to the critical backup task and limit the bandwidth usage of non-important tasks to ensure the smooth completion of the backup task. This configuration result helps the system give priority to meeting the bandwidth demand of the backup task under limited bandwidth resources, and improves the efficiency and stability of data backup.

[0121] A cloud-based data backup method is implemented based on the cloud-based data backup system, and includes the following steps:

[0122] S1: Based on the cloud storage nodes, the growth rate, access frequency and storage location information of the backup data are extracted, the feature set of the computer backup data is constructed, and the feature similarity is evaluated and then the grouping is screened to obtain the partition data of the computer backup data;

[0123] S2: Based on the computer backup data partition data, extract the access frequency and importance information, sort the partitions by importance to generate a priority list, assign backup tasks and frequencies, and obtain the partition status adjustment record;

[0124] S3: Based on the partition status adjustment record, create the initial hash chain and generate the basic hash value, update the end of the hash chain, and compare the update of the data block of the chain tail hash change record in sequence to obtain the incremental backup result;

[0125] S4: Based on the incremental backup results, collect the bandwidth consumption in each period in real time, build a bandwidth load status set, analyze bandwidth trend changes, and obtain bandwidth usage prediction sequence;

[0126] S5: Based on the bandwidth usage prediction sequence, dynamically adjust bandwidth allocation, optimize task bandwidth priority, allocate bandwidth to cover task requirements, and generate computer backup data backup optimization configuration results.

[0127] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A cloud-based data backup system, characterized in that: The system comprises: The data load monitoring and partitioning module builds a feature set of computer backup data based on storage nodes in the cloud environment, identifies data sets with similar features, and determines whether the feature set of computer backup data needs to be re-partitioned according to the difference between feature parameters to obtain computer backup data partition data; The priority partition backup scheduling module prioritizes the partitions based on the computer backup data partition data, builds a partition priority list, determines the backup task and frequency of each partition according to the partition priority list, and obtains a partition status adjustment record; The incremental hash chain difference detection module creates an initial incremental hash chain based on the partition status adjustment record, and updates the incremental hash chain. By comparing the hash value of the chain tail with the previous version, the change status of the backup data block is determined to obtain the incremental backup result. The bandwidth prediction and dynamic optimization module obtains the bandwidth consumption data of each period in the backup process in real time according to the execution process of the incremental backup in the incremental backup result, and predicts the bandwidth demand before the next backup task is executed, and dynamically adjusts the bandwidth allocation according to the prediction result to obtain the computer backup data backup optimization configuration result.

2. The cloud-based data backup system according to claim 1, characterized in that: The steps of obtaining the data set with similar characteristics are specifically as follows: Based on the storage nodes in the cloud environment, the backup data increment and access frequency data of the nodes are obtained by extracting log records, the growth trend of the backup data is analyzed, and the storage path information is extracted to generate a feature set of computer backup data; According to the feature set of the computer backup data, the similarity of the growth rate and the access frequency is evaluated, the feature differences between the data records are compared, and similar data are divided into the same group to obtain a data set with similar features.

3. The cloud-based data backup system according to claim 2, characterized in that: The steps for obtaining the computer backup data partition data are specifically as follows: Based on the data set with similar characteristics, the formula is adopted: Calculate the comprehensive difference ΔF of the characteristic parameters between the current period and the previous period to obtain the difference information between the characteristic parameters; Among them, f i,t represents the value of the i-th characteristic parameter in the current period t, f i,t-1 represents the i-th characteristic parameter value in the previous period t-1, and n represents the total number of characteristic parameters; According to the difference information between the characteristic parameters, the difference between the current period and the previous period is evaluated, and by comparing the difference degree with a preset threshold, it is determined whether the characteristic set needs to be re-divided to obtain the computer backup data partition data.

4. The cloud-based data backup system according to claim 3, characterized in that: The steps for obtaining the partition status adjustment record are specifically as follows: According to the computer backup data partition data, the formula is used: Calculate the priority score P of each partition and integrate the data to get the partition priority list; Among them, S is the storage capacity of the partition, T s is the total storage capacity, B is the partition backup frequency, T b is the average backup frequency, W s and W b are weight factors for storage occupancy ratio and backup frequency ratio respectively; According to the partition priority list, the backup tasks of the partitions are sorted and allocated according to the priority scores, the backup status information of each partition is automatically updated, and the adjustment record of the partition status is generated.

5. The cloud-based data backup system according to claim 4, characterized in that: The acquisition steps for updating the incremental hash chain are specifically as follows: Based on the partition status adjustment record, filter the data blocks of the high priority partition, generate basic hash values ​​for the data blocks, and add the hash values ​​to the incremental hash chain in sequence to establish the initial incremental hash chain of the current data block state; Based on the initial incremental hash chain of the current data block state, newly added or updated data blocks in the partition are identified, new hash values ​​of the data blocks are generated and appended to the end of the hash chain, and incremental backup update records are generated.

6. The cloud-based data backup system according to claim 5, characterized in that: The steps for obtaining the incremental backup result are specifically as follows: Based on the incremental backup update record, read the latest hash value at the end of the chain and compare it with the hash value at the end of the previous version of the chain to determine whether the contents are consistent and generate a hash chain comparison result; According to the hash chain comparison result, a list of data blocks with marked changes is extracted in the next backup cycle, and a backup operation is performed on the data blocks in the list, and the incremental backup log is updated to obtain an incremental backup result.

7. The cloud-based data backup system according to claim 6, characterized in that: The steps for obtaining the computer backup data backup optimization configuration result are specifically as follows: Based on the execution process data of the incremental backup results, the bandwidth consumption of each period is obtained in real time, the bandwidth usage status is recorded and summarized regularly, and a bandwidth load status set is established; According to the bandwidth load state set, the formula is adopted: Calculate the average change in bandwidth usage ΔC avg , get the bandwidth change trend analysis results in all time periods; Where n′ is the total number of time periods, C i′ is the bandwidth usage in the i′th period, C i′-1 is the bandwidth usage in the previous period; According to the bandwidth change trend analysis results in all the time periods, the formula is used: Q next =D current +(L×ΔC avg ) Calculate the predicted bandwidth requirement Q for the next backup task next , dynamically adjust bandwidth allocation according to the prediction results, and obtain the optimal configuration result of computer backup data backup; Among them, D current is the bandwidth usage of the current backup task, and L is the estimated backup task duration.

8. A cloud-based data backup method, characterized in that: The cloud-based data backup system according to any one of claims 1 to 7 comprises the following steps: Based on cloud storage nodes, the growth rate, access frequency and storage location information of backup data are extracted, a feature set of computer backup data is constructed, and the feature similarity is evaluated and then filtered and grouped to obtain the partition data of computer backup data; Based on the computer backup data partition data, access frequency and importance information are extracted, partitions are sorted by importance to generate a priority list, backup tasks and frequencies are allocated, and partition status adjustment records are obtained; Based on the partition status adjustment record, an initial hash chain is created and a basic hash value is generated, the tail of the hash chain is updated, and the update status of the data blocks recorded by the hash change record at the tail of the chain is compared in sequence to obtain an incremental backup result; Based on the incremental backup results, the bandwidth consumption in each period is collected in real time, a bandwidth load status set is constructed, bandwidth trend changes are analyzed, and a bandwidth usage prediction sequence is obtained; Based on the bandwidth usage prediction sequence, the bandwidth allocation is dynamically adjusted, the task bandwidth priority is optimized, the bandwidth is allocated to cover the task requirements, and the computer backup data backup optimization configuration result is generated.

Images