Data backup performance optimization method based on machine learning
Through machine learning-based methods, we collect and organize information from the backup system, establish neural network models, and optimize data backup paths, and solve the performance problems of traditional data backup methods in large-scale and high-concurrency scenarios, achieving more efficient and stable data backup.
Patent Information
- Application Number
- CN202311721170.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-14
- Publication Date
- 2025-02-11
- Estimated Expiration
- 2043-12-14
AI Technical Summary
Traditional data backup methods are difficult to meet performance requirements in large-scale and high-concurrency scenarios, resulting in increased backup time, reduced efficiency and unstable backup paths, which may lead to disorder or failure of backup data.
Using machine learning-based data backup performance optimization method, we collect and organize backup information of the target backup system, establish a neural network model, analyze the data to be backed up, determine whether to back up the whole or separately, and select the best backup method. Ultimately optimize data backup performance.
It improves the efficiency of data backup, reduces backup time, avoids unnecessary data transmission, prevents the problem of missing data backup, and optimizes data backup performance by comparing backup performance statistics.
Smart Images

Figure CN117667519B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data backup, and more specifically, to a data backup performance optimization method based on machine learning. Background Art
[0002] With the popularization of computers and the development of information technology, information security and data storage technology have attracted more and more attention from major companies. Data backup can effectively prevent data loss, and the importance of data backup and recovery has become increasingly prominent. However, traditional data backup methods often cannot meet large-scale and high-concurrency requirements. When the data to be backed up is too large, it often takes more time to screen the backup path. Moreover, as the backup data increases, it is easy to cause confusion in the backup path, resulting in reduced efficiency and unstable channels, which ultimately leads to disordered backup data and backup failure. Therefore, how to improve the performance of data backup, reduce backup time, and improve backup efficiency has become a problem that needs to be solved urgently. Summary of the invention
[0003] To achieve the above object, the present invention provides the following technical solutions:
[0004] The data backup performance optimization method based on machine learning includes the following steps:
[0005] Step 1: Collect the backupable information of the target backup system;
[0006] Step 2: sorting the collected backup information and establishing a neural network model;
[0007] Step 3: Collect the data information to be backed up;
[0008] Step 4: Analyze the data to be backed up and the information that can be backed up, and determine whether to back up the data to be backed up in its entirety or separately according to the analysis results;
[0009] Step 5: input the neural network model according to the analysis result in step 4, select the best backup path according to the output result of the neural network model and back up the data to be backed up;
[0010] Step 6: When the backup is completed, obtain the current backup performance statistics. When the current backup performance statistics are greater than the historical backup performance statistics, optimize the data backup performance.
[0011] The backup information of the target backup system includes the backup path of the target backup system, the total storage capacity corresponding to the backup path, the current storage capacity, and the path distance. The backup path is marked as KBi, the total storage capacity corresponding to the backup path is marked as ZCi-m, the current storage capacity is marked as XCi-m, and the path distance is marked as LJi-m.
[0012] In a preferred embodiment, the operation of organizing the collected backupable information refers to:
[0013] Step 1: Obtain the backup path KBi, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, and the path distance LJi-m from the backup information of the target backup system;
[0014] Step 2: The total storage capacity ZCi-m, current storage capacity XCi-m, and path distance LJi-m corresponding to the backup path are unified into units, and the backup path KBi, the total storage capacity ZCi-m, current storage capacity XCi-m, and path distance LJi-m corresponding to the backup path are summarized into a backup path collection.
[0015] In a preferred embodiment, in step 2, establishing a neural network model refers to:
[0016] Step 1: Simulate the backup operation, select data with a data size of SJn as the backup data during the simulation, and then use the backup path KBi for backup. Each backup path KBi collects k sets of data sets, each set of data sets includes the backup path KBi, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, the path distance LJi-m, and the total time to complete the simulated backup. The total time to complete the simulated backup is marked as MNt.
[0017] Step 2: Send the k sets of data in step 1 to the neural network model, use the data size SJn, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, and the path distance LJi-m as the input data for model training, and use the total time MNt to complete the simulated backup as the output data for model training, and divide it into training set, validation set, and test set in the ratio of 6:2:2. When the performance of the model on the validation set no longer improves, the training is considered to be completed;
[0018] Step 3. Based on step 2, the total time MNt required to complete the simulated backup is assigned a value of 0 or 1 to train the model, where 0 indicates that the total time MNt required to complete the simulated backup is less than or equal to the set threshold Y1, and 1 indicates that the total time MNt required to complete the simulated backup is greater than the set threshold Y1.
[0019] In a preferred embodiment, in step three, the information on the data to be backed up includes the total storage capacity required for the data to be backed up, the storage capacity required for each separate backup data when it can be backed up separately, and the total storage capacity required for the data to be backed up is marked as ZC, and the storage capacity required for each separate backup data when it can be backed up separately is marked as FCp.
[0020] In a preferred embodiment, in step 4, analyzing the data information to be backed up and the information that can be backed up, and determining whether to back up the data information to be backed up in its entirety or separately according to the analysis result means:
[0021] First, the total storage capacity ZC required by the data to be backed up is compared with the preset threshold. If the total storage capacity ZC required by the data to be backed up is greater than the preset threshold Y2, it is backed up separately. If the total storage capacity ZC required by the data to be backed up is less than the preset threshold Y2, a secondary analysis process is performed.
[0022] The logic of the secondary analysis process is as follows:
[0023] S1. Substitute the total storage capacity ZC required for the data to be backed up, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, and the path distance LJi-m into the calculation formula of the judgment value PD, PD= LJi-m, is a specific proportional coefficient. When the judgment value PD is greater than or equal to the preset threshold value Y3, and When the result is greater than or equal to the preset threshold value Y4, the data information to be backed up is backed up in its entirety; otherwise, the data information to be backed up is backed up separately.
[0024] In a preferred embodiment, in step 5, inputting the neural network model according to the analysis result in step 4, selecting the best backup path according to the output result of the neural network model and backing up the data to be backed up refers to:
[0025] When the data to be backed up is backed up in its entirety, the total storage capacity ZC required by the data to be backed up, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, and the path distance LJi-m are input into the neural network model, and the backup paths KBi corresponding to the result of assigning the total time MNt to complete the simulated backup to 0 are sorted in ascending order according to the total time MNt to complete the simulated backup, and the backup path KBi ranked first is taken as the best backup path and the data to be backed up is backed up;
[0026] When the data information to be backed up is backed up separately, the size of the storage capacity FCp required for each separate backup data is sorted in descending order. The storage capacity FCp required for each separate backup data, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, and the path distance LJi-m are input into the neural network model in the sorted order. The backup paths KBi corresponding to the result of assigning the total time MNt to complete the simulated backup to 0 are sorted in ascending order according to the size of the total time MNt to complete the simulated backup. The backup path KBi ranked first is used as the best backup path corresponding to the storage capacity FCp required for the current separate backup data. Backup path, and then delete this backup path KBi from the neural network model until there is no result that the total time MNt for completing the simulated backup is assigned to 0, and then all the backup paths KBi are put into the neural network model again for the next batch matching, and the current storage capacity XCi-m in all backup paths KBi that have completed the matching with the storage capacity FCp required for separate backup data is updated, until all the storage capacities FCp required for separate backup data have completed the matching with the backup path KBi, all the matching results are listed in the order of the time when the matching is completed, and then the backup data is backed up in the order of the matching batches.
[0027] Technical effects and advantages of the present invention:
[0028] The present invention can improve the backup efficiency. By collecting the backable information of the target backup system and organizing the information, and by establishing a neural network model, the optimal backup path can be determined more accurately, thereby improving the backup efficiency. Moreover, based on the analysis results of the data information to be backed up and the backable information, it can be determined whether the data to be backed up is a whole backup or a separate backup, thereby avoiding unnecessary data transmission and reducing the backup time.
[0029] The present invention can simulate the backup of the backup data and train the neural network model to predict the problems that may occur in the data transmission process, such as the balance being easily broken, so as to exclude the paths that may cause the loss of data backup during path selection, thus avoiding the problem of the loss of data backup. Moreover, by obtaining the current backup performance statistics and comparing them with the historical backup performance statistics, the data backup performance can be optimized, thereby further improving the backup efficiency and reducing the backup time. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to facilitate understanding by those skilled in the art, the present invention is further described below in conjunction with the accompanying drawings;
[0031] Figure 1 This is a schematic diagram of the data backup performance optimization method based on machine learning in the present invention. DETAILED DESCRIPTION
[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0033] Example 1
[0034] The data backup performance optimization method based on machine learning includes the following steps:
[0035] Step 1. Collect the backup information of the target backup system; the backup information of the target backup system includes the backup path of the target backup system, the total storage capacity corresponding to the backup path, the current storage capacity, and the path distance, and the backup path is marked as KBi, the total storage capacity corresponding to the backup path is marked as ZCi-m, the current storage capacity is marked as XCi-m, and the path distance is marked as LJi-m. The i, m, n, p, and t in this article are only sequences or types of items, and do not involve specific meanings. They will not be elaborated in the following text. i, m, n, p, and t are all positive integers when taking values.
[0036] Step 2: sort out the collected backup information and establish a neural network model; specifically: Step 1: obtain the backup path KBi, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, and the path distance LJi-m from the backup information of the target backup system;
[0037] Step 2: The total storage capacity ZCi-m, current storage capacity XCi-m, and path distance LJi-m corresponding to the backup path are uniformly processed. Unit uniformity processing refers to uniform conversion processing according to international standard units, so that the units in subsequent calculations are uniform, and the backup path KBi, the total storage capacity ZCi-m, current storage capacity XCi-m, and path distance LJi-m corresponding to the backup path are summarized as a backup path collection, which is convenient for subsequent backup path KBi search, and establishing a neural network model refers to:
[0038] Step 1: Simulate the backup operation, select data with a data size of SJn as the backup data during the simulation, and then use the backup path KBi for backup. Each backup path KBi collects k sets of data sets, each set of data sets includes the backup path KBi, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, the path distance LJi-m, and the total time to complete the simulated backup. The total time to complete the simulated backup is marked as MNt.
[0039] Step 2: Send the k sets of data in step 1 to the neural network model, use the data size SJn, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, and the path distance LJi-m as the input data for model training, and use the total time MNt to complete the simulated backup as the output data for model training, and divide it into training set, validation set, and test set in the ratio of 6:2:2. When the performance of the model on the validation set no longer improves, the training is considered to be completed;
[0040] Step 3. Based on step 2, the total time MNt required to complete the simulated backup is assigned a value of 0 or 1 to train the model, where 0 indicates that the total time MNt required to complete the simulated backup is less than or equal to the set threshold Y1, and 1 indicates that the total time MNt required to complete the simulated backup is greater than the set threshold Y1.
[0041] Step 3, collect information about the data to be backed up; the information about the data to be backed up includes the total storage capacity required for the data to be backed up, the storage capacity required for each separate backup data when it can be backed up separately, and the total storage capacity required for the data to be backed up is marked as ZC, and the storage capacity required for each separate backup data when it can be backed up separately is marked as FCp.
[0042] Step 4: Analyze the data to be backed up and the backupable information, and determine whether to back up the data to be backed up in its entirety or separately based on the analysis results; specifically: first compare the total storage capacity ZC required for the data to be backed up with a preset threshold; if the total storage capacity ZC required for the data to be backed up is greater than the preset threshold Y2, back them up separately; if the total storage capacity ZC required for the data to be backed up is less than the preset threshold Y2, perform a secondary analysis.
[0043] The logic of the secondary analysis process is as follows:
[0044] S1. Substitute the total storage capacity ZC required for the data to be backed up, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, and the path distance LJi-m into the calculation formula of the judgment value PD, PD= LJi-m, is a specific proportional coefficient. When the judgment value PD is greater than or equal to the preset threshold value Y3, and When the result is greater than or equal to the preset threshold value Y4, the data information to be backed up is backed up in its entirety; otherwise, the data information to be backed up is backed up separately.
[0045] Step 5: inputting the neural network model according to the analysis results in step 4, selecting the best backup path according to the output results of the neural network model, and backing up the data to be backed up; In step 5, inputting the neural network model according to the analysis results in step 4, selecting the best backup path according to the output results of the neural network model, and backing up the data to be backed up refers to:
[0046] When the data to be backed up is completely backed up, the data to be backed up is backed up in accordance with the pre-set backup strategy;
[0047] When the data information to be backed up is backed up separately, the data to be backed up is backed up according to the pre-set backup strategy 2.
[0048] Step six: obtain the current backup performance statistics when the backup is completed. When the current backup performance statistics are greater than the historical backup performance statistics, the data backup performance is optimized. In this embodiment, the backup performance can refer to the total time taken from the start to the completion of the backup. If a shorter time is spent, it can be determined that the data backup performance has been optimized.
[0049] Example 2
[0050] The data backup performance optimization method based on machine learning includes the following steps:
[0051] Step 1: Collect the backupable information of the target backup system;
[0052] Step 2: sorting the collected backup information and establishing a neural network model;
[0053] Step 3: Collect the data information to be backed up;
[0054] Step 4: Analyze the data to be backed up and the information that can be backed up, and determine whether to back up the data to be backed up in its entirety or separately according to the analysis results;
[0055] Step 5: input the neural network model according to the analysis result in step 4, select the best backup path according to the output result of the neural network model and back up the data to be backed up;
[0056] Step 6: When the backup is completed, obtain the current backup performance statistics. When the current backup performance statistics are greater than the historical backup performance statistics, optimize the data backup performance.
[0057] The backup information of the target backup system includes the backup path of the target backup system, the total storage capacity corresponding to the backup path, the current storage capacity, and the path distance. The backup path is marked as KBi, the total storage capacity corresponding to the backup path is marked as ZCi-m, the current storage capacity is marked as XCi-m, and the path distance is marked as LJi-m.
[0058] The operation of organizing the collected backupable information refers to:
[0059] Step 1: Obtain the backup path KBi, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, and the path distance LJi-m from the backup information of the target backup system;
[0060] Step 2: The total storage capacity ZCi-m, current storage capacity XCi-m, and path distance LJi-m corresponding to the backup path are unified into units, and the backup path KBi, the total storage capacity ZCi-m, current storage capacity XCi-m, and path distance LJi-m corresponding to the backup path are summarized into a backup path collection.
[0061] In step 2, building a neural network model means:
[0062] Step 1: Simulate the backup operation, select data with a data size of SJn as the backup data during the simulation, and then use the backup path KBi for backup. Each backup path KBi collects k sets of data sets, each set of data sets includes the backup path KBi, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, the path distance LJi-m, and the total time to complete the simulated backup. The total time to complete the simulated backup is marked as MNt.
[0063] Step 2: Send the k sets of data sets in step 1 to the neural network model, use the data size SJn, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, and the path distance LJi-m as the input data for model training, and use the total time MNt to complete the simulated backup as the output data for model training, and divide them into training set, validation set, and test set in the ratio of 6:2:2. When the performance of the model on the validation set no longer improves, the training is considered to be completed;
[0064] Step 3: Based on step 2, the total time MNt for completing the simulated backup is assigned a value of 0 or 1 to train the model, wherein 0 indicates that the total time MNt for completing the simulated backup is less than or equal to the set threshold value Y1, and 1 indicates that the total time MNt for completing the simulated backup is greater than the set threshold value Y1. Regarding how to confirm the completion of the training, an early stopping mechanism can be set, that is, when the performance of the model on the validation set no longer improves, the training is considered to be completed, or a maximum training cycle can be set. When a certain number of training times is reached, the training is stopped regardless of whether the model performance is still improving. In this embodiment, an early stopping mechanism is selected, and the neural network in the neural network model is selected as a convolutional neural network. The neural network model can be set on a learning machine. The neural network model is established in the above manner. When the total time MNt for completing the simulated backup is greater than the set threshold value Y1, it is determined that the balance in the data transmission process is easily broken, which prolongs the overall data transmission time, and even easily causes the problem of missing data backup. Therefore, the total time MNt for completing the simulated backup greater than the set threshold value Y1 will be excluded in the subsequent path selection.
[0065] In step three, the information of the data to be backed up includes the total storage capacity required by the data to be backed up, the storage capacity required by each separate backup data when it can be backed up separately, and the total storage capacity required by the data to be backed up is marked as ZC, and the storage capacity required by each separate backup data when it can be backed up separately is marked as FCp. The splitting method of each separate backup data when it is backed up separately follows the preset data splitting strategy, such as setting the number of folders at the minimum level, and then subdividing each folder into the minimum level and splitting it.
[0066] In step 4, the data to be backed up and the backupable information are analyzed, and the whole backup or separate backup of the data to be backed up is determined according to the analysis result.
[0067] First, the total storage capacity ZC required by the data to be backed up is compared with the preset threshold. If the total storage capacity ZC required by the data to be backed up is greater than the preset threshold Y2, it is backed up separately. If the total storage capacity ZC required by the data to be backed up is less than the preset threshold Y2, a secondary analysis process is performed.
[0068] The logic of the secondary analysis process is as follows:
[0069] S1. Substitute the total storage capacity ZC required for the data to be backed up, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, and the path distance LJi-m into the calculation formula of the judgment value PD, PD= LJi-m, is a specific proportional coefficient. When the judgment value PD is greater than or equal to the preset threshold value Y3, and When the result is greater than or equal to the preset threshold value Y4, the data information to be backed up is backed up in its entirety; otherwise, the data information to be backed up is backed up separately. When the judgment value PD is less than the preset threshold value Y3, it is determined that the balance in the data transmission process is easily broken, which prolongs the overall data transmission time and may even lead to data backup missing problems.
[0070] In step 5, inputting the neural network model according to the analysis result in step 4, selecting the best backup path according to the output result of the neural network model and backing up the data to be backed up means:
[0071] When the entire copy of the data to be backed up is backed up, the data to be backed up is backed up according to the pre-set backup strategy; specifically: the total storage capacity ZC required for the data to be backed up, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, and the path distance LJi-m are input into the neural network model, and the backup paths KBi corresponding to the result of assigning the total time MNt to complete the simulated backup to 0 are sorted in ascending order according to the total time MNt to complete the simulated backup, and the backup path KBi ranked first is used as the best backup path and the data to be backed up is backed up; the smaller the total time MNt to complete the simulated backup, the more stable the transmission performance of the backup path KBi is.
[0072] When the data information to be backed up is backed up separately, the backup data is backed up according to the pre-set backup strategy 2; specifically: the storage capacity FCp required for each separate backup data when it can be backed up separately is sorted in descending order, and the storage capacity FCp required for each separate backup data, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, and the path distance LJi-m are input into the neural network model in the sorted order, and the backup paths KBi corresponding to the result of assigning the total time MNt for completing the simulated backup to 0 are sorted in ascending order according to the size of the total time MNt for completing the simulated backup, and the backup path KBi ranked first is used as the best backup path corresponding to the storage capacity FCp required for the current separate backup data, and then this backup path KBi is deleted from the neural network model until there is no result of assigning the total time MNt for completing the simulated backup to 0, and then all the backup paths KBi are re-entered into the neural network model for the next batch matching, and all the backup paths that have completed the matching with the storage capacity FCp required for the separate backup data are matched. The current storage capacity XCi-m in KBi is updated until the storage capacity FCp required for all separate backup data is matched with the backup path KBi, and all matching results are listed in the order of the time when the matching is completed, and then the backup data is backed up in the order of the matching batches. It should be noted that: in the same batch, only one separate backup data required storage capacity FCp is successfully matched with the only backup path KBi, and all matching results are listed in the order of the time when the matching is completed, and then the backup data is backed up in the order of the matching batches. This means that first, a backup channel is established between the storage capacity FCp required for the separate backup data that successfully matched the previous batch and the corresponding backup path KBi. After the backup of the backup path KBi of this batch is completed, it is the turn of the storage capacity FCp required for the corresponding separate backup data in the next batch, and then a backup channel is established between the two until all data to be backed up are transmitted, and then the current backup performance statistics are obtained. When the current backup performance statistics are greater than the historical backup performance statistics, the data backup performance is optimized.
[0073] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.
[0074] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0075] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0076] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0077] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A data backup performance optimization method based on machine learning, characterized in that: The following steps are involved: Step 1: Collect the backupable information of the target backup system; Step 2: sorting the collected backup information and establishing a neural network model; Step 3: Collect the data information to be backed up; Step 4: Analyze the data to be backed up and the information that can be backed up, and determine whether to back up the data to be backed up in its entirety or separately according to the analysis results; Step 5: input the neural network model according to the analysis result in step 4, select the best backup path according to the output result of the neural network model and back up the data to be backed up; Step 6: When the backup is completed, obtain the current backup performance statistics. When the current backup performance statistics are greater than the historical backup performance statistics, optimize the data backup performance. The backup performance refers to the total time taken from the start to the completion of the backup. The backupable information of the target backup system includes the backupable path of the target backup system, the total storage capacity corresponding to the backupable path, the current storage capacity, and the path distance, and the backupable path is marked as KBi, the total storage capacity corresponding to the backupable path is marked as ZCi-m, the current storage capacity is marked as XCi-m, and the path distance is marked as LJi-m; The operation of organizing the collected backupable information refers to: Step 1: Obtain the backup path KBi, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, and the path distance LJi-m from the backup information of the target backup system; Step 2: The total storage capacity ZCi-m, the current storage capacity XCi-m, and the path distance LJi-m corresponding to the backupable path are uniformly processed, and the backupable path KBi, the total storage capacity ZCi-m, the current storage capacity XCi-m, and the path distance LJi-m corresponding to the backupable path are summarized into a backupable path arrangement set; In step 2, building a neural network model means: Step 1: Simulate the backup operation, select data with a data size of SJn as the backup data during the simulation, and then use the backup path KBi for backup. Each backup path KBi collects k sets of data sets, each set of data sets includes the backup path KBi, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, the path distance LJi-m, and the total time to complete the simulated backup. The total time to complete the simulated backup is marked as MNt. Step 2: Send the k sets of data in step 1 to the neural network model, use the data size SJn, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, and the path distance LJi-m as the input data for model training, and use the total time MNt to complete the simulated backup as the output data for model training, and divide it into training set, validation set, and test set in the ratio of 6:2:
2. When the performance of the model on the validation set no longer improves, the training is considered to be completed; Step 3: Based on step 2, the total time MNt for completing the simulated backup is assigned to 0 or 1 to train the model, where 0 indicates that the total time MNt for completing the simulated backup is less than or equal to the set threshold Y1, and 1 indicates that the total time MNt for completing the simulated backup is greater than the set threshold Y1; In step 3, the information of the data to be backed up includes the total storage capacity required by the data to be backed up and the storage capacity required by each separate backup data when the backup can be separated, and the total storage capacity required by the data to be backed up is marked as ZC, and the storage capacity required by each separate backup data when the backup can be separated is marked as FCp; In step 4, the data to be backed up and the backupable information are analyzed, and the whole backup or separate backup of the data to be backed up is determined according to the analysis result. First, the total storage capacity ZC required by the data to be backed up is compared with the preset threshold value. If the total storage capacity ZC required by the data to be backed up is greater than the preset threshold value Y2, then the data is backed up separately. If the total storage capacity ZC required by the data to be backed up is less than the preset threshold value Y2, then a secondary analysis process is performed; The logic of the secondary analysis process is as follows: S1. Substitute the total storage capacity ZC required for the data to be backed up, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, and the path distance LJi-m into the calculation formula of the judgment value PD, PD= LJi-m, is a specific proportional coefficient. When the judgment value PD is greater than or equal to the preset threshold value Y3, and When the result is greater than or equal to the preset threshold value Y4, the data information to be backed up is backed up in its entirety, otherwise the data information to be backed up is backed up separately; In step 5, inputting the neural network model according to the analysis result in step 4, selecting the best backup path according to the output result of the neural network model and backing up the data to be backed up means: When the data to be backed up is completely backed up, the data to be backed up is backed up in accordance with the pre-set backup strategy; Specifically, the total storage capacity ZC required by the data to be backed up, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, and the path distance LJi-m are input into the neural network model, and the backup paths KBi corresponding to the result of assigning the total time MNt to complete the simulated backup to 0 are sorted in ascending order according to the total time MNt to complete the simulated backup, and the backup path KBi ranked first is taken as the best backup path and the data to be backed up is backed up; When the data information to be backed up is backed up separately, the data to be backed up is backed up according to the pre-set backup strategy 2; Specifically, the storage capacity FCp required for each separate backup data when it can be backed up separately is sorted in descending order, and the storage capacity FCp required for each separate backup data, the total storage capacity ZCi-m corresponding to the backup path, the current storage capacity XCi-m, and the path distance LJi-m are input into the neural network model in the sorted order, and the backup paths KBi corresponding to the result of assigning the total time MNt to complete the simulated backup to 0 are sorted in ascending order according to the total time MNt to complete the simulated backup, and the backup path KBi ranked first is used as the best backup path corresponding to the storage capacity FCp required for the current separate backup data, and then this backup path KBi is deleted from the neural network model. Until there is no result that the total time MNt for completing the simulated backup is assigned a value of 0, then all the backup paths KBi are re-put into the neural network model for the next batch of matching, and the current storage capacity XCi-m in all the backup paths KBi that have completed the matching with the storage capacity FCp required for the separate backup data is updated, until all the storage capacities FCp required for the separate backup data are matched with the backup paths KBi, all the matching results are listed in the chronological order of the completion of the matching, and then the backup data is backed up in the order of the matching batches, and in the same batch, only one storage capacity FCp required for separate backup data is successfully matched with the only backup path KBi.
Citation Information
Patent Citations
Database backup scheduling method and device
CN114625570A