Data backup method and device based on large model

Abstract

The invention provides a data backup method based on a large model, and the method comprises the steps: S1, building a data model, and carrying out the data understanding and analysis and model training and optimization; s2, making a backup strategy: performing intelligent analysis on a current to-be-backed-up data set by utilizing the trained large model; dynamically adjusting and optimizing a data backup strategy according to an analysis result of the large model, respectively adopting incremental backup or full backup or differential backup, and reasonably distributing backup resources and priorities according to importance and recovery requirements of data; s3, data backup execution: executing data backup operation according to the backup strategy formulated in S2; and S4, backup management and optimization: managing and optimizing the backup data regularly.

Description

Technical Field

[0001] The present invention relates to the field of computer data technology, and in particular to a method and device for data backup based on a large model. Background Art

[0002] With the rapid development of information technology, various business systems are increasingly dependent on data and the amount of data is growing explosively. Data backup has become a key link in ensuring data security and business continuity. However, existing data backup methods are often unable to cope with the backup needs of large-scale data, and there are problems such as long backup time, easy data loss, low recovery efficiency, and unreasonable resource allocation. Especially when dealing with complex data models, traditional backup methods often find it difficult to accurately capture the inherent correlation and dynamic changes of data, resulting in incomplete or inconsistent backup data, which cannot effectively support business recovery and affects subsequent data recovery and business continuity. Summary of the invention

[0003] The present invention proposes a method and device for data backup based on a large model. The present invention proposes a method and device for data backup based on a large model. By introducing the intelligent analysis and processing capabilities of the large model, the invention aims to solve the following key problems:

[0004] Improve data accuracy: Use the intelligent analysis capabilities of large models to accurately capture the inherent correlation and dynamic changes of data, and ensure the integrity and consistency of backup data;

[0005] Improve backup efficiency: Through the prediction and optimization capabilities of large models, dynamically adjust the backup strategy to improve backup efficiency.

[0006] Reasonable allocation of resources: Based on the analysis results of the large model, dynamically adjust resource allocation to ensure the smooth progress of the backup process.

[0007] Speed ​​up recovery: Use the big model's intelligent analysis and processing capabilities for backup data to speed up the data recovery process.

[0008] Specifically, the present invention provides a method for data backup based on a large model, comprising:

[0009] S1. Data model construction, data understanding and analysis, model training and optimization;

[0010] S2. Backup strategy formulation: Use the trained big model to intelligently analyze the current data set to be backed up; dynamically adjust and optimize the data backup strategy based on the analysis results of the big model, using incremental backup, full backup, or differential backup respectively, and reasonably allocate backup resources and priorities based on the importance and recovery requirements of the data;

[0011] S3, data backup execution, executes data backup operation according to the backup strategy formulated in S2;

[0012] S4. Backup management and optimization: regularly manage and optimize backup data.

[0013] Furthermore, in S1, the data understanding and analysis refers to collecting and analyzing the characteristics of the data to be backed up, including but not limited to data type, structure, scale and change frequency, and then using the Transformer large model architecture and algorithm to train the model based on the characteristics of the data to be backed up and in combination with business needs.

[0014] Furthermore, in S1, the model training and optimization refers to the use of historical data or sample data, using a large amount of labeled or unlabeled data to train a large model to improve the model's ability to recognize, classify and compress data. Specifically, based on a given sample, a known training data set, sample features, predicted values, independent variables, and model parameters, the probability of each feature appearing in each category is calculated, and by continuously adjusting model parameters, optimizing algorithms, and adding regularization terms, the model is prevented from overfitting, the generalization ability of the model is improved, and the backup data is ensured to accurately reflect the true state of the original data, and to identify the inherent correlation and dynamic changes of the data; the independent variables are original features or transformed features.

[0015] Furthermore, the probability P(t) of each feature appearing in each category is calculated, and the specific calculation formula is as follows: Based on a given sample x, a known training data set T = {(x1, y1), (x2, y2), ..., (x n ,y n )},in is the jth feature of the ith sample, and the training probability is calculated by statistical training data set as P(x), y = θ0 + θ1z + θ2z 2 +θ3z 3 +…+θ n z n , where y is the predicted value, z is the independent variable, θ0, θ1, θ2, θ3, θ n It is the parameter of the model. The prediction probability obtained by prediction calculation is defined as P(y|z). Finally, the probability P(t) of each feature appearing in each category is calculated. The formula is as follows: where P(c) is the evidence factor used for normalization.

[0016] Furthermore, in S2, the incremental backup is used for data that changes frequently; the incremental backup first reads the information of the last backup, obtains the time and device information of the last backup, then scans all data files, counts the size and modification time of the files, and thus compares the current file modification time with the last backup file modification time, finds out the changed files, copies the changed files to the backup device, and finally updates the backup time and device information.

[0017] Furthermore, the specific calculation formula for the incremental backup is:

[0018] I={f{i1}, f{i2}, f{i3},..., f{i j}}, where I represents the incremental set and represents the jth file that has changed.

[0019] Furthermore, in S2, the full backup is used for relatively stable data parts; the full backup first scans all data files, counts the size and modification time of the files, selects a suitable backup device, copies all data files to the backup device, and finally records the backup time and device information.

[0020] Furthermore, the specific calculation formula for the full backup is: T = {f1, f2, f3, ..., f i}, where T represents the backup set, f i Represents the i-th file.

[0021] Furthermore, in S3, the data backup execution operation is to use the prediction ability of the big model to predict possible data changes or risks in advance during the backup process to ensure the timeliness and accuracy of the backup data; after the backup is completed, the backup data generated by the big model is verified to ensure the integrity and consistency of the data; after the verification is passed, the backup data is stored in a safe and reliable medium.

[0022] Furthermore, in S4, the backup data is regularly managed and optimized, including deleting expired or duplicate backup files and updating backup strategies. At the same time, the feedback mechanism of the large model is used to continuously optimize the backup process and improve the backup efficiency and recovery speed.

[0023] On the other hand, the present invention provides a data backup device based on a large model, comprising:

[0024] Data model building module, used for data understanding and analysis and model training and optimization;

[0025] Backup strategy formulation module: Use the trained big model to intelligently analyze the current data set to be backed up; dynamically adjust and optimize the data backup strategy based on the analysis results of the big model, using incremental backup, full backup, or differential backup respectively, and reasonably allocate backup resources and priorities based on the importance and recovery requirements of the data;

[0026] The data backup execution module executes the data backup operation according to the backup strategy formulated by the backup strategy formulation module;

[0027] The backup management and optimization module is used to regularly manage and optimize backup data.

[0028] On the other hand, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer program instructions. When the computer program instructions are executed by a computer, the computer performs any step of the above-mentioned data backup method.

[0029] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising a memory and a processor. The memory is used to store computer program instructions executable by the processor, and the processor implements the data backup method provided by the embodiment of the present invention when executing the computer program.

[0030] The beneficial effects of the embodiments of the present invention are as follows:

[0031] The present invention effectively solves the problems of data accuracy, backup efficiency, resource allocation and recovery speed existing in traditional backup methods by introducing the intelligent analysis and processing capabilities of large models, and provides a more efficient, reliable and intelligent solution for the field of data storage and backup. The technical effects achieved include the following aspects:

[0032] 1. Improve the accuracy and integrity of backup data: Through the intelligent analysis and prediction capabilities of the big model, ensure that the backup data can accurately reflect the true status of the original data, identify the inherent correlation and dynamic changes of the data, and ensure the integrity and consistency of the backup data.

[0033] 2. Improve backup and recovery efficiency and allocate resources reasonably: Dynamically adjust resource allocation, backup strategy and optimize backup process, analyze data change patterns and trends, predict future backup needs, so as to reasonably allocate backup resources and priorities, and reduce unnecessary resource consumption and time waste.

[0034] 3. Support large-scale data processing: To cope with the backup needs of large-scale and complex data sets, quickly identify key information and dependencies in the backup data, and provide precise guidance for data recovery, thereby shortening the recovery time and improving business continuity. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A schematic diagram of a data backup method provided by an embodiment of the present invention;

[0036] Figure 2 A schematic diagram of the composition of a data monitoring system provided by an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of the composition of an electronic device provided by an embodiment of the present invention;

[0038] Figure 4 A schematic diagram of the composition of a computer storage medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] Certain embodiments of the present invention will be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all, embodiments are shown. Indeed, various embodiments of the present invention may be implemented in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure may satisfy applicable legal requirements.

[0040] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0041] See also Figure 1 The present invention provides a method for data backup based on a large model, comprising:

[0042] S1. Data model construction, data understanding and analysis, model training and optimization;

[0043] S2. Backup strategy formulation: Use the trained big model to intelligently analyze the current data set to be backed up; dynamically adjust and optimize the data backup strategy based on the analysis results of the big model, using incremental backup, full backup, or differential backup respectively, and reasonably allocate backup resources and priorities based on the importance and recovery requirements of the data;

[0044] S3, data backup execution, executes data backup operation according to the backup strategy formulated in S2;

[0045] S4. Backup management and optimization: regularly manage and optimize backup data.

[0046] Specifically, in S1, the data understanding and analysis refers to collecting and analyzing the characteristics of the data to be backed up, including but not limited to data type, structure, scale and change frequency, and then using the Transformer large model architecture and algorithm to train the model based on the characteristics of the data to be backed up and in combination with business needs.

[0047] Specifically, in S1, the model training and optimization refers to the use of historical data or sample data, using a large amount of labeled or unlabeled data to train a large model to improve the model's ability to recognize, classify and compress data. Specifically, based on a given sample, a known training data set, sample features, predicted values, independent variables, and model parameters, the probability of each feature appearing in each category is calculated, and by continuously adjusting model parameters, optimizing algorithms, and adding regularization terms, the model is prevented from overfitting, the generalization ability of the model is improved, and the backup data is ensured to accurately reflect the true state of the original data, and to identify the inherent correlation and dynamic changes of the data; the independent variables are original features or transformed features.

[0048] Specifically, the probability P(t) of each feature appearing in each category is calculated, and the specific calculation formula is as follows: Based on a given sample x, a known training data set T = {(x1, y1), (x2, y2), ..., (x n ,y n )},in is the jth feature of the ith sample, and the training probability is calculated by statistical training data set as P(x), y = θ0 + θ1z + θ2z 2 +θ3z 3 +…+θ n z n , where y is the predicted value, z is the independent variable, θ0, θ1, θ2, θ3, θ n It is the parameter of the model. The prediction probability obtained by prediction calculation is defined as P(y|z). Finally, the probability P(t) of each feature appearing in each category is calculated. The formula is as follows: where P(c) is the evidence factor used for normalization.

[0049] Specifically, in S2, the incremental backup is used for data that changes frequently; the incremental backup first reads the information of the last backup, obtains the time and device information of the last backup, then scans all data files, counts the file size and modification time, and thus compares the current file modification time with the last backup file modification time, finds out the changed files, copies the changed files to the backup device, and finally updates the backup time and device information.

[0050] Specifically, the specific calculation formula for the incremental backup is: I = {f{i1}, f{i2}, f{i3}, ..., f{i j}}, where I represents the increment set, f{i j} indicates the jth file that has changed.

[0051] Specifically, in S2, the full backup is used for relatively stable data parts; the full backup first scans all data files, counts the size and modification time of the files, selects a suitable backup device, copies all data files to the backup device, and finally records the backup time and device information.

[0052] Specifically, the full backup calculation formula is: T = {f1, f2, f3, ..., f i}, where T represents the backup set, f i Represents the i-th file.

[0053] Specifically, in S3, the data backup execution operation is to use the prediction ability of the big model to predict possible data changes or risks in advance during the backup process to ensure the timeliness and accuracy of the backup data; after the backup is completed, the backup data generated by the big model is verified to ensure the integrity and consistency of the data; after the verification is passed, the backup data is stored in a safe and reliable medium, such as a disk array or cloud storage.

[0054] Specifically, in S4, the backup data is regularly managed and optimized, including deleting expired or duplicate backup files and updating backup strategies. At the same time, the feedback mechanism of the large model is used to continuously optimize the backup process and improve the backup efficiency and recovery speed.

[0055] Another embodiment of the present invention provides a data backup device based on a large model, such as Figure 2 As shown, including:

[0056] Data model building module, used for data understanding and analysis and model training and optimization;

[0057] Backup strategy formulation module: Use the trained big model to intelligently analyze the current data set to be backed up; dynamically adjust and optimize the data backup strategy based on the analysis results of the big model, using incremental backup, full backup, or differential backup respectively, and reasonably allocate backup resources and priorities based on the importance and recovery requirements of the data;

[0058] The data backup execution module executes the data backup operation according to the backup strategy formulated by the backup strategy formulation module;

[0059] The backup management and optimization module is used to regularly manage and optimize backup data.

[0060] Ginseng Figure 3 and Figure 4A computer-readable storage medium provided in an embodiment of the present invention is used to store computer program instructions. The storage medium can be embedded in the electronic device. When the computer program instructions are executed by the computer, the computer executes the data backup method described in the aforementioned method embodiment.

[0061] An embodiment of the present invention further provides an electronic device. Figure 4 As shown, the electronic device 30 includes: a processor 301, a memory 302, and a communication bus 303. The communication bus 303 is used to realize the connection communication of the components, the communication interface is used to communicate signaling or data with other node devices, and the memory 302 stores machine-readable instructions executable by the processor 301. When the electronic device 30 is running, the processor 302 communicates with the memory 301 through the communication bus 303, and the above-mentioned data backup method is executed when the machine-readable instructions are called by the processor 301.

[0062] In addition, the above-mentioned figures are only schematic illustrations of the processing included in the method according to the embodiment of the present invention, and are not intended to limit the purpose. The order in the figures does not limit the time sequence of these operations, and these operations can be performed simultaneously or independently in multiple modules. Moreover, multiple steps can be combined into one step for execution, or one step can be disassembled into multiple steps for execution.

[0063] The above description is only a specific implementation mode of the present invention and is not intended to limit the protection scope of the present invention. For those skilled in the art, the above embodiments can be changed or modified within the scope of the present invention, and these changes and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A method for data backup based on a large model, characterized in that: include: S1. Data model construction, data understanding and analysis, model training and optimization; S2. Backup strategy formulation: Use the trained large model to perform intelligent analysis on the current data set to be backed up; According to the analysis results of the large model, the data backup strategy is dynamically adjusted and optimized, and incremental backup, full backup, or differential backup is adopted respectively. At the same time, the backup resources and priorities are reasonably allocated according to the importance and recovery requirements of the data. S3, data backup execution, executes data backup operation according to the backup strategy formulated in S2; S4. Backup management and optimization: regularly manage and optimize backup data.

2. The data backup method according to claim 1, characterized in that: In S1, the data understanding and analysis refers to collecting and analyzing the characteristics of the data to be backed up, including but not limited to data type, structure, scale and change frequency, and then using the Transformer large model architecture and algorithm to train the model based on the characteristics of the data to be backed up and in combination with business needs.

3. The data backup method according to claim 1, characterized in that: In S1, the model training and optimization refers to the use of historical data or sample data, using a large amount of labeled or unlabeled data to train a large model to improve the model's ability to recognize, classify and compress data. Specifically, based on a given sample, a known training data set, sample features, predicted values, independent variables, and model parameters, the probability of each feature appearing in each category is calculated, and by continuously adjusting model parameters, optimizing algorithms, and adding regularization terms, the model is prevented from overfitting, the generalization ability of the model is improved, and the backup data is ensured to accurately reflect the true state of the original data, and the inherent correlation and dynamic changes of the data are identified; the independent variables are original features or transformed features.

4. The data backup method according to claim 3, characterized in that: The probability P(t) of each feature appearing in each category is calculated as follows: Based on a given sample x, a known training data set T = {(x1, y1), (x2, y2), ..., (x n ,y n )},in is the jth feature of the ith sample, and the training probability is calculated by statistical training data set as P(x), y = θ0 + θ1z + θ2z 2 +θ3z 3 +…+θ n z n , where y is the predicted value, z is the independent variable, θ0, θ1, θ2, θ3, θ n It is the parameter of the model. The prediction probability obtained by prediction calculation is defined as P(y|z). Finally, the probability P(t) of each feature appearing in each category is calculated. The formula is as follows: where P(c) is the evidence factor used for normalization.

5. The data backup method according to claim 1, characterized in that: In S2, the incremental backup is used for data that changes frequently; the incremental backup first reads the information of the last backup, obtains the time and device information of the last backup, then scans all data files, counts the size and modification time of the files, and thus compares the current file modification time with the last backup file modification time, finds out the changed files, copies the changed files to the backup device, and finally updates the backup time and device information.

6. The data backup method according to claim 5, characterized in that: The specific calculation formula for the incremental backup is: I = {f{i1}, f{i2}, f{i3}, ..., f{i j }}, where I represents the increment set, f{i j } indicates the jth file that has changed.

7. The data backup method according to claim 1, characterized in that: In S2, the full backup is used for relatively stable data parts; the full backup first scans all data files, counts the size and modification time of the files, selects a suitable backup device, copies all data files to the backup device, and finally records the backup time and device information.

8. The data backup method according to claim 7, characterized in that: The specific calculation formula for the full backup is: T = {f1, f2, f3, ... f i }, where T represents the backup set, f i Represents the i-th file.

9. The data backup method according to claim 1, characterized in that: In S3, the data backup execution operation is to use the prediction ability of the big model to predict possible data changes or risks in advance during the backup process to ensure the timeliness and accuracy of the backup data; After the backup is completed, the backup data generated by the large model is verified to ensure the integrity and consistency of the data; After verification, the backup data will be stored in a safe and reliable medium.

10. The data backup method according to claim 1, characterized in that: In S4, the backup data is regularly managed and optimized, including deleting expired or duplicate backup files and updating backup strategies. At the same time, the feedback mechanism of the large model is used to continuously optimize the backup process and improve the backup efficiency and recovery speed.

11. A data backup device based on a large model, characterized in that: include: Data model building module, used for data understanding and analysis and model training and optimization; Backup strategy formulation module: Use the trained large model to perform intelligent analysis on the current data set to be backed up; According to the analysis results of the large model, the data backup strategy is dynamically adjusted and optimized, and incremental backup, full backup, or differential backup is adopted respectively. At the same time, the backup resources and priorities are reasonably allocated according to the importance and recovery requirements of the data. The data backup execution module executes the data backup operation according to the backup strategy formulated by the backup strategy formulation module; The backup management and optimization module is used to regularly manage and optimize backup data.

12. A computer-readable storage medium storing computer program instructions, wherein when the computer program instructions are executed by a computer, the computer executes the steps of the data backup method according to any one of claims 1 to 10.

Images