Automatic database backup and recovery method and system based on artificial intelligence

Abstract

The invention relates to the technical field of database management, in particular to an automatic database backup and recovery method and system based on artificial intelligence, and the method comprises the steps: collecting the operation data of a database in real time, and constructing a dynamic data feature model; dynamically generating a backup strategy through a backup strategy generation module based on the data feature model; executing backup operation according to the backup strategy, creating and updating a metadata index used for rapid recovery in the backup process, monitoring the backup state in real time, and automatically triggering a grading retry mechanism when backup abnormity is detected; automatically executing a recovery test, verifying the integrity and recoverability of the backup data, and if verification fails, triggering an alarm and performing backup again; and when a data recovery request is received, quickly positioning and recovering target data based on the metadata index and the transaction log, and monitoring a recovery state in real time in a recovery process. The rigidity of a traditional fixed strategy is overcome, and the backup efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of database management technology, specifically to an artificial intelligence-based method and system for automatic database backup and recovery. Background Technology

[0002] As the core of modern information infrastructure, database systems bear the responsibility of storing and managing critical business data. The security and availability of this data directly affect the stable operation of the system and the continuity of business. To ensure data reliability, traditional database systems typically employ backup and recovery mechanisms based on fixed strategies, such as performing full backups, incremental backups, or differential backups at fixed intervals. While these methods provide a degree of data protection, their static and lagging nature leads to significant shortcomings in dynamically changing real-world application environments. First, fixed backup strategies cannot adapt to dynamic changes in data volume and access patterns. For example, during peak business periods, data updates are frequent, and fixed-period backups may not capture all critical changes, increasing the risk of data loss; while during periods of stable data, unnecessary backup redundancy may occur, leading to wasted storage resources and increased costs. Second, traditional methods lack awareness of business context and data relationships. Backup strategy formulation often ignores differences in data importance, inter-table transaction dependencies, and the real-time load of the business system, making it difficult to achieve an effective balance between backup efficiency, storage costs, and recovery reliability.

[0003] Although artificial intelligence technology has demonstrated powerful optimization and prediction capabilities in many fields in recent years, and machine learning methods are particularly adept at learning patterns from historical data and making dynamic decisions, the mature solutions that have been deeply applied to database backup and recovery are mostly limited to the improvement of single functions, and have not yet formed a systematic solution covering the entire chain of backup strategy generation, execution monitoring, recovery verification and intelligent recovery. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides an artificial intelligence-based method and system for automatic database backup and recovery.

[0005] In a first aspect, the present invention provides an automatic database backup and recovery method based on artificial intelligence, comprising the following steps: S1. Real-time collection of database operation data, including performance metrics, system logs, and transaction logs; the performance metrics include at least data write rate, query response time, and number of concurrent connections, and a dynamic data feature model is constructed. S2. Based on the data feature model, a backup strategy is dynamically generated by the backup strategy generation module. The backup strategy includes backup type selection, backup time window, backup frequency, and storage path optimization. The backup type is automatically selected as full backup, incremental backup, or differential backup based on the data update frequency and importance. The backup strategy generation module uses a reinforcement learning-based algorithm to continuously learn and adjust the backup strategy using historical backup success rate, recovery time, and storage cost as reward signals. S3. Perform backup operations according to the backup strategy, and create and update a metadata index for fast recovery during the backup process. The metadata index records at least the location, timestamp, and data dependencies of the backup data blocks. At the same time, monitor the backup status in real time, and automatically trigger a tiered retry mechanism when a backup anomaly is detected, and record the fault information for strategy optimization. S4. Automatically perform a recovery test after each backup to verify the integrity and recoverability of the backup data. If the verification fails, trigger an alarm and re-backup. S5. When a data recovery request is received, the target data is quickly located and recovered based on the metadata index and transaction log of the backup data, and the recovery status is monitored in real time during the recovery process.

[0006] As a further limitation of the technical solution of the present invention, step S1 includes: S11. Collect performance metrics, system logs, and transaction logs in real time through an agent program deployed on the database host; S12. Preprocess the collected running data, including handling missing values, normalizing numerical features, and encoding categorical features. S13. Extract key features from the preprocessed data. The key features include table-level update frequency, data growth trend slope, access popularity score within a preset time window, and the strength of dependency relationships between transactions. S14. Input the key features into an unsupervised learning algorithm for training to construct the dynamic data feature model; the unsupervised learning algorithm uses a clustering algorithm to identify the pattern categories presented by data with different features, and continuously updates the model based on new running data.

[0007] By deploying agents for real-time data collection, the system ensures that the data sources used for analysis are authentic, comprehensive, and up-to-date, laying a solid foundation for generating accurate backup strategies. Through continuous updates to the data feature model, the system can automatically adapt to the long-term evolution of business data, avoiding strategy failures caused by model aging.

[0008] As a further limitation of the technical solution of the present invention, step S13, the step of extracting key features includes: S131. Count the number of transactions that perform insert, update, and delete operations on each data table within a unit of time, and calculate the update frequency of the table. S132. Based on the historical data volume of the time series, use the linear regression algorithm to fit its growth curve, and use the slope value of the obtained regression line as the slope of the data growth trend. S133. Based on the frequency of data table being queried and accessed within a preset time window and the weight of system input / output resources consumed, generate access popularity score through a weighted calculation model. S134. Parse the transaction log, construct a transaction-data table association graph, and use graph algorithms to analyze the frequency and order of common access to different data tables in a transaction, thereby quantifying the strength of the dependency relationship between the transactions.

[0009] As a further limitation of the technical solution of the present invention, in S133, the specific steps for generating the access popularity score through the weighted calculation model include: Get the number of times data table i is queried within a preset time window T. ; Obtain the total amount of system input / output resources consumed by data table i due to being accessed and queried within the time window T. The total resource amount is quantified as the cumulative I / O operation time or the number of data blocks read, expressed in milliseconds. To each and Normalization is performed to obtain the normalized access frequency score. and resource consumption score ; The final visit popularity score of data table i is generated using a weighted calculation formula. :

[0010] in, and These are weighting coefficients, and The weighting coefficients and The initial value is preset by the system and then dynamically adjusted through an independent feedback loop mechanism; this feedback loop mechanism optimizes the accuracy of the popularity score by analyzing the correlation between historical access patterns and system performance indicators.

[0011] As a further limitation of the technical solution of the present invention, the specific steps in step S134 for quantifying the strength of the dependency relationship between the transactions include: The database transaction log is parsed, each transaction is abstracted as a node, and the data table accessed by the transaction is abstracted as another type of node; if a transaction accesses a data table, an undirected edge is established between the corresponding transaction node and the data table node, thereby constructing a bipartite graph structure of transaction-data table association graph. For any two data table nodes, calculate the number of times they are accessed simultaneously in all transactions, and use this number of co-occurrences as the basic indicator to measure the strength of the dependency relationship between the two tables. The order in which two co-occurring data tables are accessed in a transaction is analyzed to introduce time-series weights to the basic indicators; among them, table pairs with a strict order of write operations are given higher time-series weights. By combining the co-occurrence frequency and temporal weight, the final inter-transaction dependency strength value is calculated using a predefined quantification formula; the strength value is used to identify the logical affinity between data tables.

[0012] In the formula, This represents the strength of the dependency between data tables m and n. This represents the number of times data m and n co-occur. This represents the maximum number of occurrences of the CCP across all tables. Represents the weights of time-series relationships. To balance the weights.

[0013] By extracting specific and quantifiable key features such as "update frequency," "growth trend slope," "access popularity score," and "transaction dependency strength," machine learning models can more accurately understand the state and behavior of data. In particular, the "transaction dependency strength" feature provides a crucial basis for ensuring transaction consistency during subsequent recovery.

[0014] As a further limitation of the technical solution of the present invention, in step S2, the backup strategy generation module uses a reinforcement learning-based algorithm for continuous learning and backup strategy adjustment, which specifically includes: Back up the state of the decision-making environment Defined as a feature vector, which includes at least the current data feature model output, system load indicators, storage subsystem health status, and historical backup success records; Define backup strategy decisions as actions. The action is a multi-dimensional vector whose dimensions include the type of backup operation to be performed, the start time of the time window allocated to the backup operation, the frequency level of backup execution, and the identifier of the target storage path. Constructing a reward function The reward function is:

[0015] In the formula, The normalized success rate For action The average recovery time under these conditions The target is the maximum acceptable recovery time. For action Storage costs below Storage cost budget is obtained by normalizing the actual cost by dividing the budget. These are the weighting coefficients.

[0016] Using a deep Q-network algorithm, actions are continuously tested in simulated environments and real historical data. Observe the new state They receive rewards to train and iteratively update their policy network. Ultimately, a backup strategy generation model is learned that can maximize long-term cumulative rewards.

[0017] By designing a reward function that integrates backup success rate, recovery time, and storage cost, the reinforcement learning backup strategy generation module learns to find the optimal balance among multiple competing objectives. This results in a globally optimal backup strategy in terms of efficiency, cost, and reliability—something that manual rules or single-objective optimization cannot achieve. Through continuous trial and error learning in simulations and historical data, the backup strategy generation module can discover complex strategies that human experts might overlook and adapt to constantly changing environments, ensuring that the system's backup strategy remains consistently optimal.

[0018] As a further limitation of the technical solution of the present invention, the specific steps of real-time monitoring of the backup status and triggering the graded retry mechanism when an anomaly is detected in step S3 include: During the backup task execution, real-time metrics are collected, including progress metrics, resource utilization, and error logs. The indicator items are analyzed in real time by an anomaly detection model. The anomaly detection model adopts the isolated forest algorithm. When the anomaly score output by the model exceeds a dynamically adjusted decision threshold ξ, it is determined to be a backup anomaly. When a backup anomaly is detected, a tiered retry mechanism is triggered based on the anomaly type. The tiered retry mechanism includes: first, performing a momentary error retry locally; if this fails, switching the backup path and retrying; if this fails again, downgrading the backup method and retrying. Fault information, retry process and results are recorded in the knowledge base as feedback input for optimizing the backup strategy generation module.

[0019] At each monitoring time point t, the collected multidimensional indicators (including progress indicator Pt, resource utilization vector Rt, and error log count Et) are combined into a unified multidimensional feature vector Xt. The feature vector Xt at the current time point is input into a pre-trained anomaly detection model, which outputs an anomaly score Zt; the anomaly detection model is trained using the isolated forest algorithm. The anomaly score Zt is compared with a dynamically adjusted decision threshold ξ; if Zt > ξ, the current system state is determined to be a backup anomaly. Based on the time series of recent historical outlier scores, the decision threshold ξ is dynamically updated using statistical process control methods to adapt to long-term gradual changes in system load and performance patterns.

[0020] Metadata indexing records the location and dependencies of data blocks, supporting rapid recovery and enhancing backup data management capabilities. An anomaly detection model based on the Isolation Forest algorithm and dynamic thresholds enables earlier and more accurate detection of potential faults, rather than relying on simple timeout checks. A tiered retry mechanism employs differentiated recovery measures for different fault types, significantly improving the final success rate of backup tasks.

[0021] As a further limitation of the technical solution of the present invention, step S4 specifically includes: S41. In a sandbox environment isolated from the production environment, automatically restore the latest completed backup data and corresponding metadata indexes to a temporary test database instance; S42. Calculate the data checksum of the restored data using a hash algorithm, and compare it with the calculation result of the backup source database at the time of backup. If they are inconsistent, the integrity verification is deemed to have failed. S43. Send predefined test queries and transaction commands to the recovered test database instance to verify whether the database service can start normally, whether the data can be accessed, and whether the transaction logic is executed correctly. S44. After verification is completed, the temporary test database instance is automatically destroyed to release resources, and a verification report is generated. S45. If the integrity verification fails or the recoverability verification fails, the highest priority alarm will be automatically triggered and the backup retry process will be started. At the same time, the verification result of this failure will be sent as a negative feedback signal to the backup strategy generation module to optimize the backup strategy decision.

[0022] Performing real-world recovery tests in an isolated sandbox verifies not only the existence of data but also its usability, thus providing the highest level of confidence. Feedback of failed verification results to the policy generation module prevents the regeneration of backup policies that lead to failure; this is a crucial step in enabling the system to self-optimize and continuously improve.

[0023] As a further limitation of the technical solution of the present invention, step S5 includes: S51. Parse the user's data recovery request, the request containing at least a target time point T and a target data identifier D; the target data identifier D is a database name, table name, record primary key, or transaction ID; S52. Query the metadata index to quickly locate all backup data block sets and their physical storage paths that are related to the target data identifier D before time point T; wherein, if D is a transaction ID, the data tables and records affected by the transaction are determined by tracing back the transaction log, and then converted into data identifiers. S53. Based on the data block set, transaction log, and data dependencies, generate an optimal recovery sequence; the recovery sequence ensures transactional consistency of the data and prioritizes the recovery of parent node data in the dependency graph; S54. According to the recovery sequence, read the data block set from the backup storage in parallel and apply it to the target database instance; during the application process, perform forward or rollback operations on the data according to the transaction log to make it reach a consistent state at time point T. S55. During the recovery process, monitor the recovery progress, throughput, and error status of each data block in real time; after the overall recovery is completed, automatically perform a data verification to confirm the integrity of the recovered data.

[0024] It supports recovery from the entire database down to a single record, and even to a specific transaction, meeting diverse user recovery needs and avoiding the drawbacks of traditional methods that require full recovery. By analyzing transaction logs and data dependencies and generating the optimal recovery sequence, it ensures that even with partial recovery, the inherent logical constraints and consistency of the data are met, avoiding logical errors caused by improper recovery order. Through metadata indexing to quickly locate data blocks and combined with parallel recovery technology, it significantly shortens the time required for data recovery, meeting the stringent requirements of critical business systems for short recovery time objectives (RTO).

[0025] Secondly, the present invention also provides an artificial intelligence-based automatic database backup and recovery system for implementing the method described in the first aspect, the system comprising: The data feature analysis module is used to collect database operation data in real time and build a dynamic data feature model; the operation data includes performance indicators, system logs and transaction logs. The backup strategy generation module is used to dynamically generate backup strategies based on the dynamic data feature model and using a reinforcement learning-based algorithm; the backup strategy includes backup type selection, backup time window, backup frequency, and storage path optimization. The backup execution and monitoring module is used to execute backup operations according to the backup strategy, create and update metadata indexes during the backup process, monitor the backup status in real time, and automatically trigger a tiered retry mechanism when a backup anomaly is detected; the metadata index records at least the location, timestamp, and data dependencies of the backup data blocks. The recovery verification module is used to automatically perform recovery tests after each backup is completed, verify the integrity and recoverability of the backup data, and trigger an alarm when the verification fails. The intelligent recovery module is used to quickly locate and recover the target data based on the metadata index and transaction log when a data recovery request is received, and to monitor the recovery status in real time during the recovery process.

[0026] The intelligent recovery module utilizes pre-built metadata indexes and transaction logs to quickly locate and recover data at a specific point in time, in a specific table, or even in a specific record. It supports recovery needs at various granularities, overcoming the inefficiencies of traditional recovery methods that require restoring the entire backup set, and significantly shortening the recovery time objective (RTO). The entire process forms a complete closed loop from data acquisition, policy generation, execution monitoring to verification feedback.

[0027] As can be seen from the above technical solutions, this application has the following advantages: Through the data feature analysis module and the reinforcement learning-based strategy generation module, the system can automatically adjust the backup frequency, type and storage strategy according to real-time factors such as data changes and business load, which completely overcomes the rigidity of traditional fixed strategies and improves backup efficiency.

[0028] By introducing an automatic recovery verification step after backup, the recoverability and integrity of the data can be verified immediately after the backup is completed. Backup failures or data corruption issues can be detected in a timely manner and retries can be triggered, which greatly reduces the risk of discovering that the backup is invalid only when recovery is actually needed, and provides double protection for data security. Attached Figure Description

[0029] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a flowchart illustrating the method provided in an embodiment of the present invention.

[0031] Figure 2 A block diagram of a system provided in an embodiment of the present invention. Detailed Implementation

[0032] To make the purpose, features, and advantages of this application more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solution protected by this application. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0033] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this application and in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0034] like Figure 1 As shown, this embodiment of the invention provides an artificial intelligence-based automatic database backup and recovery method, including the following steps: S1. Real-time collection of database operation data, including performance metrics, system logs, and transaction logs; the performance metrics include at least data write rate, query response time, and number of concurrent connections, and a dynamic data feature model is constructed. The purpose of this step is to comprehensively understand the operational status of the database, providing a data foundation for intelligent decision-making. Specifically, it includes: Deploy a lightweight data collection agent on the database server. This agent collects the following data at a fixed frequency (e.g., every minute) through performance views provided by the database system (such as MySQL's Performance Schema, Oracle's AWR report), direct parsing of database transaction logs (such as Redo Log, Write-Ahead Log), and monitoring system resources (such as iostat, vmstat): Performance metrics include, but are not limited to, data write rate (MB / s), average query response time (ms), current concurrent connections, cache hit rate, and disk I / O utilization.

[0035] System logs: Collect database error logs and slow query logs to identify potential system anomalies and performance bottlenecks.

[0036] Transaction Log: Parse the transaction log to obtain the transaction ID, operation type (INSERT / UPDATE / DELETE), affected data tables and record primary keys, transaction commit time, etc.

[0037] The collected raw data is sent to a central management server for preprocessing. This includes: Data cleaning: handling missing values ​​and removing outliers.

[0038] Feature extraction: Extracting key features from the cleaned data, including: Table-level update frequency: Counts the number of DML operations performed on each data table per unit of time.

[0039] Data growth trend slope: Apply linear regression to the time series of historical data for each table to fit its daily growth rate.

[0040] Access popularity score: Calculate a weighted score based on the frequency of table queries and the I / O resources consumed by the query (e.g., popularity = 0.7 * (normalized access count) + 0.3 * (normalized resource consumption)).

[0041] Transaction dependency strength: By analyzing transaction logs, a "transaction-data table" bipartite graph is constructed. Graph algorithms are used to calculate the frequency and order of co-occurrence of different data tables in a transaction, thus quantifying their dependency strength.

[0042] The extracted features are then fed into an unsupervised learning algorithm (such as K-Means clustering) for training. This model categorizes the data table into different classes (e.g., "high-frequency updated hot data," "low-frequency access cold data," "stable growth data," etc.) and forms a dynamic data feature model. The model is periodically (e.g., daily) retrained with new data to adapt to changes in data patterns.

[0043] S2. Based on the data feature model, a backup strategy is dynamically generated by the backup strategy generation module. The backup strategy includes backup type selection, backup time window, backup frequency, and storage path optimization. The backup type is automatically selected as full backup, incremental backup, or differential backup based on the data update frequency and importance. The backup strategy generation module uses a reinforcement learning-based algorithm to continuously learn and adjust the backup strategy using historical backup success rate, recovery time, and storage cost as reward signals. This step utilizes the output of S1 to intelligently generate the optimal backup strategy.

[0044] Enhance the learning environment setup: State: The state vector is the output of the dynamic data feature model built in S1, and it incorporates real-time indicators such as the current system load and the remaining capacity of the storage pool.

[0045] Action: An action is a multi-dimensional decision that includes: [backup type (full / incremental / differential), execution time window (start time), target storage path (high-performance SSD / low-cost archive storage)].

[0046] Reward: The design of the reward function is crucial. An example reward function formula is as follows: R = ω1*(+1 for success, -1 for failure) - ω2*(time taken for this recovery) - ω3*(storage cost of this backup) Among them, ω1, ω2, and ω3 are weighting coefficients used to balance the different importance of reliability, speed, and cost.

[0047] Policy learning and decision-making: A Deep Q-Network (DQN) algorithm is employed. The agent (i.e., the policy generation module) continuously tries different actions in a simulated environment, observes new states, and obtains rewards R. Through numerous iterations, the Q-Network gradually learns to select the action that yields the maximum long-term cumulative reward in a specific state. In practical applications, the system directly calls the trained Q-Network to output the optimal backup policy based on the current state and distributes it to the execution module.

[0048] S3. Perform backup operations according to the backup strategy, and create and update a metadata index for fast recovery during the backup process. The metadata index records at least the location, timestamp, and data dependencies of the backup data blocks. At the same time, monitor the backup status in real time, and automatically trigger a tiered retry mechanism when a backup anomaly is detected, and record the fault information for strategy optimization. This step is responsible for reliably executing the policy and logging metadata.

[0049] After receiving the policy, the backup executor (such as a modified xtrabackup, pg_basebackup, or a custom script) starts the backup task at the specified time. Simultaneously, as data blocks are written to the backup storage, the system generates a metadata index file (such as JSON or a dedicated database format). This index records the following for each data block: block_id: A unique identifier for a data block.

[0050] storage_path: The physical path in the backup storage.

[0051] backup_timestamp: The timestamp when the backup was completed.

[0052] table_id: The data table to which it belongs.

[0053] dependent_blocks: A list of other data block IDs that have transactional dependencies on this data block.

[0054] Real-time monitoring and tiered retry: Monitor the backup task's progress bar, CPU / memory / network usage, and error log stream in real time.

[0055] The monitoring data stream is input into a lightweight Isolation Forest anomaly detection model, which calculates anomaly scores in real time. If the score exceeds a dynamic threshold, it is determined to be an anomaly.

[0056] Trigger retry mechanism: Level 1: Wait 1 minute and then retry in place (to resolve issues such as network interruptions).

[0057] Level 2: If the retry fails, the task will be automatically switched to another backup path or storage node for retry (to resolve temporary storage node failures).

[0058] Level 3: If the backup fails again, the backup method will be automatically downgraded (e.g., from a full backup to an incremental backup) and then retried. All failure information and retry results are recorded for S2 strategy optimization.

[0059] S4. Automatically perform a recovery test after each backup to verify the integrity and recoverability of the backup data. If the verification fails, trigger an alarm and re-backup. This step verifies the validity of the backup immediately after backup, as a preventative measure.

[0060] The system automatically and quickly starts a temporary test database instance in a sandbox isolated from the production environment.

[0061] Restore the backup data and metadata index that were just completed to the test instance.

[0062] Calculate the cryptographic hash value (such as SHA-256) of the restored database and compare it with the hash value of the source database at the start of the backup to ensure that the data is correct at the bit level.

[0063] The automated script executes a series of predefined smoke test queries on the test instance, such as: SELECT COUNT(*) FROM important_table; UPDATE table SET ... WHERE ...; (followed by rollback), to check whether the database service is working properly, whether the basic query can be executed, and whether the transaction logic is correct.

[0064] Upon successful verification, the sandbox environment is automatically destroyed. If any step of the verification fails, an alarm is immediately triggered (such as via SMS, email, DingTalk / Lark message) and the system automatically redirects to S3 to initiate the re-backup process.

[0065] S5. When a data recovery request is received, the target data is quickly located and recovered based on the metadata index and transaction log of the backup data, and the recovery status is monitored in real time during the recovery process.

[0066] This step provides a fast and accurate recovery when users need to initiate a recovery request through the management interface.

[0067] The user submits a recovery request, specifying "restore a table to its state at 3:15 PM today" or "restore all data before a specific transaction ID".

[0068] The intelligent recovery module queries the metadata index to quickly find all data blocks related to the target data before a specified time point and their storage locations.

[0069] Based on the data dependencies recorded in the metadata index, an optimal recovery sequence is generated to avoid problems such as foreign key conflicts (e.g., recover the parent table first, then the child table).

[0070] According to the recovery plan, the required data blocks are read in parallel from the backup storage and applied to the production database. Simultaneously, the transaction logs are parsed, and the data is rolled forward or backward to ensure accurate data recovery to the user-specified point in time.

[0071] During the recovery process, the UI displays the recovery progress and estimated completion time to the user in real time. Once the recovery is complete, a final verification is automatically performed, and a notification of successful recovery is sent to the user.

[0072] In some embodiments, step S1 includes: S11. Collect performance metrics, system logs, and transaction logs in real time through an agent program deployed on the database host; S12. Preprocess the collected running data, including handling missing values, normalizing numerical features, and encoding categorical features. S13. Extract key features from the preprocessed data. The key features include table-level update frequency, data growth trend slope, access popularity score within a preset time window, and the strength of dependency relationships between transactions. S14. Input the key features into an unsupervised learning algorithm for training to construct the dynamic data feature model; the unsupervised learning algorithm uses a clustering algorithm to identify the pattern categories presented by data with different features, and continuously updates the model based on new running data.

[0073] In some embodiments, step S13, the step of extracting key features, includes: S131. Count the number of transactions that perform insert, update, and delete operations on each data table within a unit of time, and calculate the update frequency of the table. S132. Based on the historical data volume of the time series, use the linear regression algorithm to fit its growth curve, and use the slope value of the obtained regression line as the slope of the data growth trend. S133. Based on the frequency of data table being queried and accessed within a preset time window and the weight of system input / output resources consumed, generate access popularity score through a weighted calculation model. S134. Parse the transaction log, construct a transaction-data table association graph, and use graph algorithms to analyze the frequency and order of common access to different data tables in a transaction, thereby quantifying the strength of the dependency relationship between the transactions.

[0074] In some embodiments, the specific steps in S133 for generating a visit popularity score using a weighted calculation model include: Get the number of times data table i is queried within a preset time window T. ; Obtain the total amount of system input / output resources consumed by data table i due to being accessed and queried within the time window T. The total resource amount is quantified as the cumulative I / O operation time or the number of data blocks read, expressed in milliseconds. To each and Normalization is performed to obtain the normalized access frequency score. and resource consumption score ; The final visit popularity score of data table i is generated using a weighted calculation formula. :

[0075] in, and These are weighting coefficients, and The weighting coefficients and The initial value is preset by the system and then dynamically adjusted through an independent feedback loop mechanism; this feedback loop mechanism optimizes the accuracy of the popularity score by analyzing the correlation between historical access patterns and system performance indicators.

[0076] In some embodiments, step S134, specifically the steps of quantifying the strength of the dependencies between the transactions, include: The database transaction log is parsed, each transaction is abstracted as a node, and the data table accessed by the transaction is abstracted as another type of node; if a transaction accesses a data table, an undirected edge is established between the corresponding transaction node and the data table node, thereby constructing a bipartite graph structure of transaction-data table association graph. For any two data table nodes, calculate the number of times they are accessed simultaneously in all transactions, and use this number of co-occurrences as the basic indicator to measure the strength of the dependency relationship between the two tables. The order in which two co-occurring data tables are accessed in a transaction is analyzed to introduce time-series weights to the basic indicators; among them, table pairs with a strict order of write operations are given higher time-series weights. By combining the co-occurrence frequency and temporal weight, the final inter-transaction dependency strength value is calculated using a predefined quantification formula; the strength value is used to identify the logical affinity between data tables.

[0077] In the formula, This represents the strength of the dependency between data tables m and n. This represents the number of times data m and n co-occur. This represents the maximum number of occurrences of the CCP across all tables. Represents the weights of time-series relationships. To balance the weights.

[0078] In some embodiments, in step S2, the backup strategy generation module uses a reinforcement learning-based algorithm for continuous learning and backup strategy adjustment, specifically including: Back up the state of the decision-making environment Defined as a feature vector, which includes at least the current data feature model output, system load indicators, storage subsystem health status, and historical backup success records; Define backup strategy decisions as actions. The action is a multi-dimensional vector whose dimensions include the type of backup operation to be performed, the start time of the time window allocated to the backup operation, the frequency level of backup execution, and the identifier of the target storage path. Constructing a reward function The reward function is:

[0079] In the formula, The normalized success rate For action The average recovery time under these conditions The target is the maximum acceptable recovery time. For action Storage costs below Storage cost budget is obtained by normalizing the actual cost by dividing the budget. These are the weighting coefficients.

[0080] Using a deep Q-network algorithm, actions are continuously tested in simulated environments and real historical data. Observe the new state They receive rewards to train and iteratively update their policy network. Ultimately, a backup strategy generation model is learned that can maximize long-term cumulative rewards.

[0081] In some embodiments, step S3, which involves real-time monitoring of the backup status and triggering a tiered retry mechanism when an anomaly is detected, includes the following specific steps: During the backup task execution, real-time metrics are collected, including progress metrics, resource utilization, and error logs. The indicator items are analyzed in real time by an anomaly detection model. The anomaly detection model adopts the isolated forest algorithm. When the anomaly score output by the model exceeds a dynamically adjusted decision threshold ξ, it is determined to be a backup anomaly. When a backup anomaly is detected, a tiered retry mechanism is triggered based on the anomaly type. The tiered retry mechanism includes: first, performing a momentary error retry locally; if this fails, switching the backup path and retrying; if this fails again, downgrading the backup method and retrying. Fault information, retry process and results are recorded in the knowledge base as feedback input for optimizing the backup strategy generation module.

[0082] At each monitoring time point t, the collected multidimensional indicators (including progress indicator Pt, resource utilization vector Rt, and error log count Et) are combined into a unified multidimensional feature vector Xt. The feature vector Xt at the current time point is input into a pre-trained anomaly detection model, which outputs an anomaly score Zt; the anomaly detection model is trained using the isolated forest algorithm. The anomaly score Zt is compared with a dynamically adjusted decision threshold ξ; if Zt > ξ, the current system state is determined to be a backup anomaly. Based on the time series of recent historical outlier scores, the decision threshold ξ is dynamically updated using statistical process control methods to adapt to long-term gradual changes in system load and performance patterns.

[0083] In some embodiments, step S4 specifically includes: S41. In a sandbox environment isolated from the production environment, automatically restore the latest completed backup data and corresponding metadata indexes to a temporary test database instance; S42. Calculate the data checksum of the restored data using a hash algorithm, and compare it with the calculation result of the backup source database at the time of backup. If they are inconsistent, the integrity verification is deemed to have failed. S43. Send predefined test queries and transaction commands to the recovered test database instance to verify whether the database service can start normally, whether the data can be accessed, and whether the transaction logic is executed correctly. S44. After verification is completed, the temporary test database instance is automatically destroyed to release resources, and a verification report is generated. S45. If the integrity verification fails or the recoverability verification fails, the highest priority alarm will be automatically triggered and the backup retry process will be started. At the same time, the verification result of this failure will be sent as a negative feedback signal to the backup strategy generation module to optimize the backup strategy decision.

[0084] In some embodiments, step S5 includes: S51. Parse the user's data recovery request, the request containing at least a target time point T and a target data identifier D; the target data identifier D is a database name, table name, record primary key, or transaction ID; S52. Query the metadata index to quickly locate all backup data block sets and their physical storage paths that are related to the target data identifier D before time point T; wherein, if D is a transaction ID, the data tables and records affected by the transaction are determined by tracing back the transaction log, and then converted into data identifiers. S53. Based on the data block set, transaction log, and data dependencies, generate an optimal recovery sequence; the recovery sequence ensures transactional consistency of the data and prioritizes the recovery of parent node data in the dependency graph; S54. According to the recovery sequence, read the data block set from the backup storage in parallel and apply it to the target database instance; during the application process, perform forward or rollback operations on the data according to the transaction log to make it reach a consistent state at time point T. S55. During the recovery process, monitor the recovery progress, throughput, and error status of each data block in real time; after the overall recovery is completed, automatically perform a data verification to confirm the integrity of the recovered data.

[0085] like Figure 2 As shown, this embodiment of the invention also provides an artificial intelligence-based automatic database backup and recovery system for implementing the methods described in the above embodiments. The system includes: The data feature analysis module is used to collect database operation data in real time and build a dynamic data feature model; the operation data includes performance indicators, system logs and transaction logs. The backup strategy generation module is used to dynamically generate backup strategies based on the dynamic data feature model and using a reinforcement learning-based algorithm; the backup strategy includes backup type selection, backup time window, backup frequency, and storage path optimization. The backup execution and monitoring module is used to execute backup operations according to the backup strategy, create and update metadata indexes during the backup process, monitor the backup status in real time, and automatically trigger a tiered retry mechanism when a backup anomaly is detected; the metadata index records at least the location, timestamp, and data dependencies of the backup data blocks. The recovery verification module is used to automatically perform recovery tests after each backup is completed, verify the integrity and recoverability of the backup data, and trigger an alarm when the verification fails. The intelligent recovery module is used to quickly locate and recover the target data based on the metadata index and transaction log when a data recovery request is received, and to monitor the recovery status in real time during the recovery process.

[0086] In some embodiments, the data feature analysis module includes: The data acquisition submodule is used to collect performance metrics, system logs, and transaction logs in real time through an agent program deployed on the database host. The data preprocessing submodule is used to clean and standardize the collected operational data. The feature engineering submodule is used to extract key features from the preprocessed data. These key features include table-level update frequency, data growth trend slope, access popularity score, and the strength of dependencies between transactions. The model training submodule is used to input the key features into an unsupervised learning algorithm for training, and to build and continuously update the dynamic data feature model.

[0087] In some embodiments, the backup strategy generation module includes: The state space management submodule is used to maintain the state feature vector of the backup decision-making environment; The Action Space Management submodule is used to define the multi-dimensional vector space of backup operation actions; The reward calculation submodule is used to calculate the reward value based on historical backup success rate, recovery time, and storage cost. The policy learning submodule is used to run the deep Q-network algorithm, which iteratively updates the policy network by interacting with the environment to learn the optimal backup policy.

[0088] In some embodiments, the backup execution and monitoring module includes: The task scheduling submodule is used to automatically create and distribute backup tasks according to the time window and frequency settings in the backup strategy; The data extraction and index building submodule is used to extract data from the source database and generate metadata indexes; The anomaly detection submodule is used to analyze backup metrics in real time using the isolated forest algorithm and trigger alarms when an anomaly is detected. The tiered retry submodule is used to perform local retries, path switching retries, or downgraded retries based on the exception type.

[0089] In some embodiments, the recovery verification module includes: The sandbox environment management submodule is used to provide a testing environment isolated from the production environment; The data recovery testing submodule is used to restore backup data to a temporary test instance and perform integrity verification. The functional verification submodule is used to send test queries and transaction commands to the test instance to verify its recoverability. The feedback processing submodule is used to trigger an alarm and feed the result back to the backup strategy generation module when verification fails.

[0090] In some embodiments, the intelligent recovery module includes: The request parsing submodule is used to parse the user's data recovery request and determine the target time point and data identifier; The data location submodule is used to query the metadata index and quickly locate the relevant backup data blocks; The recovery planning submodule is used to generate the optimal recovery sequence based on data dependencies; The parallel recovery submodule is used to read and apply data blocks from the backup storage in parallel according to the recovery sequence; The status monitoring submodule is used to monitor the recovery progress and perform data verification upon completion.

[0091] In some embodiments, the system further includes: A visual management interface is provided to display system status, backup policies, monitoring alarms and recovery reports to users, and also provides an interface for manually adjusting policies.

[0092] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An automatic database backup and recovery method based on artificial intelligence, characterized in that, Includes the following steps: S1. Real-time collection of database operation data, including performance metrics, system logs, and transaction logs; the performance metrics include at least data write rate, query response time, and number of concurrent connections, and a dynamic data feature model is constructed. S2. Based on the data feature model, a backup strategy is dynamically generated by the backup strategy generation module. The backup strategy includes backup type selection, backup time window, backup frequency, and storage path optimization. The backup type is automatically selected as full backup, incremental backup, or differential backup based on the data update frequency and importance. The backup strategy generation module uses a reinforcement learning-based algorithm to continuously learn and adjust the backup strategy using historical backup success rate, recovery time, and storage cost as reward signals. S3. Perform backup operations according to the backup strategy, and create and update a metadata index for fast recovery during the backup process. The metadata index records at least the location, timestamp, and data dependencies of the backup data blocks. At the same time, monitor the backup status in real time, and automatically trigger a tiered retry mechanism when a backup anomaly is detected, and record the fault information for strategy optimization. S4. Automatically perform a recovery test after each backup to verify the integrity and recoverability of the backup data. If the verification fails, trigger an alarm and re-backup. S5. When a data recovery request is received, the target data is quickly located and recovered based on the metadata index and transaction log of the backup data, and the recovery status is monitored in real time during the recovery process.

2. The artificial intelligence-based automatic database backup and recovery method according to claim 1, characterized in that, Step S1 includes: S11. Collect performance metrics, system logs, and transaction logs in real time through an agent program deployed on the database host; S12. Preprocess the collected running data, including handling missing values, normalizing numerical features, and encoding categorical features. S13. Extract key features from the preprocessed data. The key features include table-level update frequency, data growth trend slope, access popularity score within a preset time window, and the strength of dependency relationships between transactions. S14. Input the key features into an unsupervised learning algorithm for training to construct the dynamic data feature model; the unsupervised learning algorithm uses a clustering algorithm to identify the pattern categories presented by data with different features, and continuously updates the model based on new running data.

3. The artificial intelligence-based automatic database backup and recovery method according to claim 2, characterized in that, Step S13, the steps for extracting key features, include: S131. Count the number of transactions that perform insert, update, and delete operations on each data table within a unit of time, and calculate the update frequency of the table. S132. Based on the historical data volume of the time series, use the linear regression algorithm to fit its growth curve, and use the slope value of the obtained regression line as the slope of the data growth trend. S133. Based on the frequency of data table being queried and accessed within a preset time window and the weight of system input / output resources consumed, generate access popularity score through a weighted calculation model. S134. Parse the transaction log, construct a transaction-data table association graph, and use graph algorithms to analyze the frequency and order of common access to different data tables in a transaction, thereby quantifying the strength of the dependency relationship between the transactions.

4. The artificial intelligence-based automatic database backup and recovery method according to claim 3, characterized in that, In S133, the specific steps for generating a visit popularity score using a weighted calculation model include: Get the number of times data table i is queried within a preset time window T. ; Obtain the total amount of system input / output resources consumed by data table i due to being accessed and queried within the time window T. The total resource amount is quantified as the cumulative I / O operation time or the number of data blocks read, expressed in milliseconds. To each and Normalization is performed to obtain the normalized access frequency score. and resource consumption score ; The final visit popularity score of data table i is generated using a weighted calculation formula. : in, and These are weighting coefficients, and .

5. The artificial intelligence-based automatic database backup and recovery method according to claim 4, characterized in that, In step S134, the specific steps for quantifying the strength of the dependencies between the transactions include: The database transaction log is parsed, each transaction is abstracted as a node, and the data table accessed by the transaction is abstracted as another type of node; if a transaction accesses a data table, an undirected edge is established between the corresponding transaction node and the data table node, thereby constructing a bipartite graph structure of transaction-data table association graph. For any two data table nodes, calculate the number of times they are accessed simultaneously in all transactions, and use this number of co-occurrences as the basic indicator to measure the strength of the dependency relationship between the two tables. The order in which two co-occurring data tables are accessed in a transaction is analyzed to introduce time-series weights to the basic indicators; among them, table pairs with a strict order of write operations are given higher time-series weights. By combining the co-occurrence frequency and temporal weight, the final inter-transaction dependency strength value is calculated using a predefined quantification formula; the strength value is used to identify the logical affinity between data tables. In the formula, This represents the strength of the dependency between data tables m and n. This represents the number of times data m and n co-occur. This represents the maximum number of occurrences of the CCP across all tables. Represents the weights of time-series relationships. To balance the weights.

6. The artificial intelligence-based automatic database backup and recovery method according to claim 5, characterized in that, In step S2, the backup strategy generation module uses a reinforcement learning-based algorithm for continuous learning and adjustment of the backup strategy, specifically including: Back up the state of the decision-making environment Defined as a feature vector, which includes at least the current data feature model output, system load indicators, storage subsystem health status, and historical backup success records; Define backup strategy decisions as actions. The action is a multi-dimensional vector whose dimensions include the type of backup operation to be performed, the start time of the time window allocated to the backup operation, the frequency level of backup execution, and the identifier of the target storage path. Constructing a reward function The reward function is: In the formula, The normalized success rate For action The average recovery time under these conditions The target is the maximum acceptable recovery time. For action Storage costs below Storage cost budget is obtained by normalizing the actual cost by dividing the budget. These are the weighting coefficients; Using a deep Q-network algorithm, actions are continuously tested in simulated environments and real historical data. Observe the new state They receive rewards to train and iteratively update their policy network. Ultimately, a backup strategy generation model is learned that can maximize long-term cumulative rewards.

7. The artificial intelligence-based automatic database backup and recovery method according to claim 6, characterized in that, In step S3, the specific steps for real-time monitoring of the backup status and triggering the tiered retry mechanism when an anomaly is detected include: During the backup task execution, real-time metrics are collected, including progress metrics, resource utilization, and error logs. The indicator items are analyzed in real time by an anomaly detection model. The anomaly detection model adopts the isolated forest algorithm. When the anomaly score output by the model exceeds a dynamically adjusted decision threshold ξ, it is determined to be a backup anomaly. When a backup anomaly is detected, a tiered retry mechanism is triggered based on the anomaly type. The tiered retry mechanism includes: first, performing a momentary error retry locally; if this fails, switching the backup path and retrying; if this fails again, downgrading the backup method and retrying. Fault information, retry process and results are recorded in the knowledge base as feedback input for optimizing the backup strategy generation module.

8. The artificial intelligence-based automatic database backup and recovery method according to claim 7, characterized in that, The steps in S4 specifically include: S41. In a sandbox environment isolated from the production environment, automatically restore the latest completed backup data and corresponding metadata indexes to a temporary test database instance; S42. Calculate the data checksum of the restored data using a hash algorithm, and compare it with the calculation result of the backup source database at the time of backup. If they are inconsistent, the integrity verification is deemed to have failed. S43. Send predefined test queries and transaction commands to the recovered test database instance to verify whether the database service can start normally, whether the data can be accessed, and whether the transaction logic is executed correctly. S44. After verification is completed, the temporary test database instance is automatically destroyed to release resources, and a verification report is generated. S45. If the integrity verification fails or the recoverability verification fails, the highest priority alarm will be automatically triggered and the backup retry process will be started. At the same time, the verification result of this failure will be sent as a negative feedback signal to the backup strategy generation module to optimize the backup strategy decision.

9. The artificial intelligence-based automatic database backup and recovery method according to claim 8, characterized in that, The steps in S5 include: S51. Parse the user's data recovery request, the request containing at least a target time point T and a target data identifier D; the target data identifier D is a database name, table name, record primary key, or transaction ID; S52. Query the metadata index to quickly locate all backup data block sets and their physical storage paths that are related to the target data identifier D before time point T; wherein, if D is a transaction ID, the data tables and records affected by the transaction are determined by tracing back the transaction log, and then converted into data identifiers. S53. Based on the data block set, transaction log, and data dependencies, generate an optimal recovery sequence; the recovery sequence ensures transactional consistency of the data and prioritizes the recovery of parent node data in the dependency graph; S54. According to the recovery sequence, read the data block set from the backup storage in parallel and apply it to the target database instance; during the application process, perform forward or rollback operations on the data according to the transaction log to make it reach a consistent state at time point T. S55. During the recovery process, monitor the recovery progress, throughput and error status of each data block in real time; after the overall recovery is completed, automatically perform a data verification to confirm the integrity of the recovered data.

10. An artificial intelligence-based automatic database backup and recovery system, used to implement the method of any one of claims 1 to 9, characterized in that, The system includes: The data feature analysis module is used to collect database operation data in real time and build a dynamic data feature model; the operation data includes performance indicators, system logs and transaction logs. The backup strategy generation module is used to dynamically generate backup strategies based on the dynamic data feature model and using a reinforcement learning-based algorithm; the backup strategy includes backup type selection, backup time window, backup frequency, and storage path optimization. The backup execution and monitoring module is used to execute backup operations according to the backup strategy, create and update metadata indexes during the backup process, monitor the backup status in real time, and automatically trigger a tiered retry mechanism when a backup anomaly is detected; the metadata index records at least the location, timestamp, and data dependencies of the backup data blocks. The recovery verification module is used to automatically perform recovery tests after each backup is completed, verify the integrity and recoverability of the backup data, and trigger an alarm when the verification fails. The intelligent recovery module is used to quickly locate and recover the target data based on the metadata index and transaction log when a data recovery request is received, and to monitor the recovery status in real time during the recovery process.

Images