A data backup method, system, device and storage medium of a database

Abstract

The application relates to the technical field of database management, and provides a data backup method, system and equipment of a database and a storage medium, the method comprising the following steps: collecting data change information and historical data query frequency of the database at a fixed time; generating a backup execution instruction in response to the data change information and the historical data query frequency meeting a preset composite logical rule; the preset composite logical rule comprises a first rule and a third rule, a second rule for executing judgment after any one of the first rule and the third rule is met, and a fourth rule for executing judgment after the second rule is met; acquiring a real-time performance index of a database host in response to the backup execution instruction; and storing a backup file to a storage target according to a backup type and by executing a corresponding backup operation in response to the real-time performance index of the database host meeting the fourth rule. The application can realize automatic hot backup and intelligent data management with high reliability, high resource utilization and without manual intervention.

Description

Technical Field

[0001] This application relates to the field of database management technology, and in particular to a database data backup method, a database data backup system, a corresponding electronic device, and a corresponding computer-readable storage medium. Background Technology

[0002] Among the technologies related to MySQL (My Structured Query Language, a relational database management system) database backup, there are manual backups, automatic backups based on fixed time periods, backups based on storage space status, and backups based on replication technology.

[0003] However, manual backup and fixed-period backup solutions fail to capture the actual workload of the business. Fixed-period backup lacks flexibility, while disk-space-based backup cannot differentiate backup strategies based on data importance, resulting in a lack of business insight. Furthermore, all of these backup solutions require manual intervention in formulating backup strategies, monitoring backup tasks, and managing the lifecycle of backup files (such as cleaning up old backups), limiting automation and increasing operational burden and costs. Summary of the Invention

[0004] This application provides a database data backup method, system, device, and storage medium, which can achieve highly reliable, high resource utilization, and truly automated hot backup and intelligent data management without human intervention.

[0005] In one aspect, this application provides a database backup method, the method comprising:

[0006] The database is periodically collected for data change information and historical data query frequency; the data change information is used to indicate the status of the data itself, and the historical data query frequency is used to indicate the data business value.

[0007] In response to the data change information and the historical data query frequency satisfying a preset composite logic rule, a backup execution instruction is generated; the preset composite logic rule includes a first rule and a third rule, a second rule is executed after either the first rule or the third rule is satisfied, and a fourth rule is executed after the second rule is satisfied; the backup execution instruction includes a backup type and a storage target;

[0008] In response to the backup execution command, obtain the real-time performance metrics of the database host;

[0009] In response to the database host's real-time performance indicators meeting the fourth rule, the corresponding backup operation is performed according to the backup type, and the backup file is stored in the storage target.

[0010] On the other hand, this application provides a database backup system, the system comprising:

[0011] The data detection module is used to periodically collect data change information and historical data query frequency from the database; the data change information is used to indicate the status of the data itself, and the historical data query frequency is used to indicate the data business value.

[0012] The intelligent judgment module is used to generate a backup execution instruction in response to the data change information and the historical data query frequency meeting a preset composite logic rule; the preset composite logic rule includes a first rule and a third rule, a second rule is executed after either the first rule or the third rule is met, and a fourth rule is executed after the second rule is met; the backup execution instruction includes a backup type and a storage target;

[0013] The performance metrics acquisition module is used to acquire real-time performance metrics of the database host in response to the backup execution command.

[0014] The backup execution module is used to respond to the database host's real-time performance indicators meeting the fourth rule, perform the corresponding backup operation according to the backup type, and store the backup file to the storage target.

[0015] In another aspect, this application also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the data backup method of any of the databases described in the present application.

[0016] In another aspect, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the database backup method described in any one of the claims.

[0017] In another aspect, this application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the database backup method described in the above aspects.

[0018] The database backup method, system, device, and storage medium provided in this application periodically collect database data change information and historical data query frequency. When the data change information and historical data query frequency meet preset composite logic rules, a backup execution instruction is generated. Then, the real-time performance indicators of the database host are obtained by responding to the backup execution instruction. When the real-time performance indicators of the database host meet a fourth rule, the corresponding backup operation is performed according to the backup type carried by the instruction, and the backup file is stored in the storage target carried by the instruction. The data change information can be used to indicate the state of the data itself, the historical data query frequency can be used to indicate the data business value, and the preset composite logic rules can include a first rule and a third rule. A second rule is executed after either the first or third rule is met, and a fourth rule is executed after the second rule is met. By monitoring database operating status indicators in real time and making intelligent judgments based on composite logic rules, the database backup process breaks away from rigid time strategies and passive response modes. Instead, it is driven by the data's own status and business value, making it intelligent and adaptive. Furthermore, when a backup is determined to be executed, the system can also monitor the real-time performance indicators of the database host. This allows for further judgment of the database host's real-time performance indicators after the backup trigger conditions are met, avoiding conflicts between backup tasks and business resources. As a result, it achieves highly reliable, high-resource-utilization, and automated hot backup and intelligent data tiered management without human intervention. Attached Figure Description

[0019] Figure 1 This is a structural block diagram of a database backup system provided in an embodiment of this application;

[0020] Figure 2 This is a flowchart illustrating the steps of a database backup method provided in an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of the data backup workflow provided in the embodiments of this application;

[0022] Figure 4 This is a structural block diagram of an electronic device provided in an embodiment of this application;

[0023] Figure 5 This is a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] In related technologies, backup solutions for MySQL databases can include manual backup, automatic backup based on fixed time periods, backup based on storage space status, and backup based on replication technology.

[0026] Specifically, manual backup involves the database administrator (DBA) periodically performing backup operations using command-line tools such as mysqldump (a logical backup tool provided by MySQL) and mysqlhotcopy, or graphical tools. This is the most basic and primitive backup method. Automatic backup based on fixed time periods involves using scheduled task tools such as cron in Linux systems to automatically execute backup scripts at preset times (e.g., 2 AM every day). This is currently a widely used automated backup solution. Backup based on storage space status involves monitoring the remaining space capacity of the server disk. When the available space falls below a certain preset threshold, an alarm is triggered or backup and log cleanup operations are automatically performed to prevent service interruption due to disk fullness. Backup based on replication technology involves using MySQL master-slave replication technology to build one or more slave databases to synchronize data with the master database in real time. The slave database itself can serve as a backup of the master database, but its main purpose is load balancing and high availability, and strictly speaking, it is not a dedicated backup solution.

[0027] However, manual backups and fixed-period backups fail to capture the actual workload of the business. During off-peak periods, they may repeatedly back up data that hasn't changed significantly, wasting resources. Conversely, during peak periods or after major changes, backups may fail to occur on time, increasing the risk of data loss. These solutions are inefficient and unable to back up critical data promptly. Furthermore, fixed-period backups are a preventative strategy, consuming fixed computing, network, and storage resources to back up full or incremental data regardless of its value or changes. This lacks flexibility, increases storage costs, and suffers from rigidity and low resource utilization. Disk-based backups, on the other hand, are a passive defense strategy. When triggered, the system may already be at risk. Passive backups ignore the business value of the data itself, such as which data is frequently accessed, and fail to differentiate backup strategies based on data importance, lacking business insight. In addition, all of the above backup solutions require manual intervention in formulating backup strategies, monitoring backup tasks, and managing the lifecycle of backup files (such as cleaning up old backups). Their limited automation increases the operational burden and costs.

[0028] This application's embodiments enable the linkage between backup strategies and actual business needs, significantly reducing storage costs, optimizing resource utilization, and achieving truly unmanned intelligent operation and maintenance. Specifically, by real-time monitoring of database operating status indicators and intelligent judgment based on composite logic rules, the database backup process breaks away from rigid time strategies and passive response modes, instead being driven by the data's own state and business value. This provides intelligence and adaptability. Furthermore, when determining to execute a backup, it can also add monitoring of the database host's real-time performance indicators. This allows for further judgment of the database host's real-time performance indicators after the backup trigger conditions are met, avoiding conflicts between backup tasks and business resources. This achieves highly reliable, high-resource-utilization, and automated hot backup and intelligent data tiered management without manual intervention. In addition, a global resource coordination manager can be introduced. When the local resources of a database node are busy and the execution of a backup task needs to be postponed, the global resource coordination manager can schedule the backup task to another database node with idle resources. This allows for the use of idle resources from a global perspective to complete the backup task, achieving cross-node / cross-cluster resource coordination.

[0029] Reference Figure 1 The diagram illustrates a structural block diagram of a database backup system provided in an embodiment of this application. This system can be an intelligent detection-decision-execution system. Specifically, it can collect database operating status indicators in real time from multiple dimensions, make judgments based on preset, configurable composite logic rules, and ultimately automatically trigger appropriate backup operations.

[0030] Optional, such as Figure 1 As shown, the system may include a data detection module 101, an intelligent judgment module 102, a performance indicator acquisition module 103, and a backup execution module 104.

[0031] In this embodiment of the application, backup can be triggered based on multi-dimensional indicators when the judgment conditions are met. This is achieved by simultaneously detecting two different dimensions of indicators: data change information and historical data query frequency, and combining them through a set of logical rules to determine whether to trigger a backup operation.

[0032] In some embodiments of this application, the data detection module 101 is responsible for periodically collecting operational status indicators from the database. These indicators may include multiple performance metrics, specifically periodically collecting data change information and historical data query frequency from the database. For example, the data detection module 101 may perform data collection tasks at fixed time intervals, such as 5 minutes.

[0033] Among them, data change information can be used to indicate the status of the data itself, and the frequency of historical data queries reflects the data access popularity and can be used to indicate the data business value. The collection of data change information and historical data query frequency enables data backup to be driven based on the status of the data itself and business value.

[0034] In some embodiments of this application, the intelligent judgment module 102 is responsible for receiving the data reported by the data detection module 101 and comparing the reported data with the preset threshold rule in the preset composite logic rule to make an intelligent judgment, so as to trigger the corresponding backup operation and realize the data backup scheme driven by the data's own state and business value.

[0035] Specifically, the intelligent judgment module 102 can be used to generate backup execution instructions in response to data change information and historical data query frequency meeting preset composite logic rules. The preset composite logic rules are not simple single-condition judgments; they can include a first rule and a third rule, a second rule executed after either the first or third rule is met, and a fourth rule executed after the second rule is met. The judgment result of the second rule can be used to determine the data access frequency. Optionally, the backup type and storage target can be determined based on the judgment result of the aforementioned preset composite logic rules. The backup type indicates the data storage level, and the storage target indicates the storage location; that is, intelligent data hierarchical management can be achieved based on the selection of backup type and storage target.

[0036] In this embodiment, intelligent hierarchical backup can be achieved based on data popularity. Specifically, the storage level of the backup data is dynamically determined according to the real-time query frequency of historical data, such as high-performance hot storage or low-cost cold storage.

[0037] In some embodiments of this application, the intelligent judgment module 102, upon determining that the backup triggering conditions are met, will not execute immediately but will instead enter the resource-aware scheduling process. Specifically, the performance indicator acquisition module 103 is responsible for detecting the real-time performance indicators of the database host according to the instructions of the intelligent judgment module 102. The real-time performance indicators of the database host include, but are not limited to, CPU (Central Processing Unit) utilization, memory utilization, disk IOPS (Input / Output Operations Per Second), IO (Input / Output) latency (i.e., response time), and network throughput. This application embodiment does not limit these metrics.

[0038] The backup execution module 104 is responsible for responding to the database host's real-time performance indicators meeting the fourth rule, and calling the corresponding backup tool to perform the operation according to the instructions of the intelligent judgment module 102.

[0039] Specifically, the backup execution instruction generated by the intelligent judgment module 102 is based on the backup type and storage target. The fourth rule is mainly a resource avoidance rule. When the fourth rule is met, it means that the execution of the current backup task will not conflict with the business resource occupation of the database host. The backup execution module 104 can be used to perform the corresponding backup operation according to the backup type carried by the instruction when the real-time performance indicators of the database host meet the fourth rule, and store the backup file in the storage target carried by the instruction. In order to automatically trigger the appropriate backup operation, such as high-performance hot storage or low-cost cold storage, while avoiding the conflict between the backup task and the business resource occupation.

[0040] In some embodiments of this application, the backup priority of the backup task can also be determined to meet the fifth rule. The fifth rule is mainly a resource preemption strategy. When the fifth rule is met, it indicates that the backup priority of the backup task is extremely high. For backup tasks with extremely high backup priority, a resource restriction strategy is adopted, that is, the resource usage of the backup process is restricted. This application does not impose such restrictions.

[0041] In some embodiments of this application, when the backup task is postponed, a global resource coordination manager can be introduced to realize cross-node backup execution. This is manifested by satisfying the sixth rule based on the idle resource status. The sixth rule is mainly a cross-node scheduling rule. When the sixth rule is satisfied, the backup task is scheduled to another database node with idle resources for execution through the global resource coordination manager. This can utilize idle resources from a global perspective to complete the backup task and realize cross-node / cross-cluster resource coordination.

[0042] In some embodiments of this application, reference is made to Figure 2 The diagram illustrates a flowchart of a database backup method according to an embodiment of this application, which may specifically include the following steps:

[0043] Step S201: Periodically collect data change information and historical data query frequency from the database.

[0044] By periodically collecting database data change information and historical data query frequency through a data detection module, database backups can be driven by the data's own state and business value. Dynamically triggering backups based on the actual internal state of the database and external business needs makes backup behavior more meaningful and timely, shifting the data backup method from time-driven to data and business-driven, greatly reducing the risk of data loss.

[0045] Optionally, data change information may include data change frequency and data change amount.

[0046] Specifically, the amount of data changes can be detected using the data detection module. This can be quantified based on the total volume of the binary logs, assessing the degree of data change since the last backup. For example, the `SHOW BINARY LOGS` command (a core management command in MySQL used to view information about all binary log files on the server) can be executed to obtain the current list of binary log files and their total volume, thus quantifying the degree of data change since the last backup. The frequency of data changes can be measured based on the data operation rate and the time interval since the last successful backup. The data operation rate indicates the data operation throughput per unit time, which can be obtained through data operation traffic detection using the data detection module. For example, this can be achieved by querying `SHOW GLOBAL`. The STATUS command (a core monitoring command in MySQL used to query global server status metrics) retrieves the values ​​of status variables such as COM_insert, COM_update, and COM_delete, calculates the data operation throughput per unit time, and thus obtains the INSERT rate (the frequency of INSERT statements executed on a database table per unit time), UPDATE rate (the frequency of UPDATE statements executed on a database table per unit time), and DELETE rate (the frequency of DELETE statements executed on a database table per unit time). The time interval since the last successful backup can be obtained through time interval detection by the data detection module, specifically recording the time interval since the last successful backup.

[0047] Optionally, the historical data query frequency can be obtained by detecting the historical data query frequency through the data detection module. For example, it can be obtained by querying tables in the MySQL performance_schema database (such as events_statements_summary_by_table) or parsing the slow query log to count the frequency of SELECT queries (times / hour / day) for historical data tables (such as data created more than 30 days ago).

[0048] It should be noted that, in addition to the database operation status indicators collected above, other operation status indicators can also be collected based on actual needs. For example, data attribute indicators (such as data sensitivity, data volume and growth rate), system status indicators (such as database load status, remaining storage resources), and business scenario indicators (such as business peak period identifiers, data lifecycle stages, etc.) can be collected. Among them, the data lifecycle stages can include the new data stage, the active data stage, the archived data stage, and the eviction data stage. Different stages can be bound to different backup frequencies and storage strategies, and this application embodiment does not impose any restrictions on this.

[0049] Step S202: If the response data change information and the historical data query frequency meet the preset composite logic rules, a backup execution instruction is generated.

[0050] The intelligent judgment module receives data reported by the data detection module and compares the reported data with the preset threshold rules in the preset composite logic rules to make an intelligent judgment.

[0051] The preset composite logic rule is not a simple single-condition judgment; it can include a first rule and a third rule, as well as a second rule that executes a judgment after either the first or third rule is satisfied. Optionally, the judgment made by the first rule can ensure data security, the judgment made by the third rule can cope with sudden access, and the judgment made by the second rule can realize intelligent hierarchical data management.

[0052] In some embodiments of this application, the data backup workflow can be as follows: Figure 3 As shown. The data backup system loads preset composite logic rules upon startup, and after the data detection module periodically collects indicators, the intelligent judgment module can perform judgments based on these preset composite logic rules. Specifically, this means that the response data change information meets the first rule, or the historical data query frequency meets the third rule, i.e., as shown... Figure 3 As shown, if the conditions of either the first rule or the third rule are met, the process can proceed to the next step, triggering a backup request. Specifically, the frequency of historical data queries can be judged based on the second rule to obtain the judgment result. Then, based on the judgment result, the backup type and storage target are determined, and based on the backup type and storage target, a backup execution instruction is generated.

[0053] The first rule primarily determines whether the amount of data changes or the volume of operations exceed a threshold, ensuring data security. Optionally, if the total volume of the binary log exceeds a preset file size threshold, or the data operation rate exceeds a preset rate threshold and the time interval exceeds a preset first time threshold, the first rule is satisfied, and the process proceeds to the next step, triggering a backup request. For example, assuming a preset file size threshold of 1.5GB, a preset rate threshold of 5000 operations / hour, and a preset first time threshold of 6 hours, the first rule can be expressed as: IF (total binlog volume > 1.5GB) OR ((data operation rate > 5000 operations / hour) AND (time since last backup > 6 hours)) THEN trigger backup request.

[0054] The third rule is mainly used to determine whether the query frequency has surged. The judgment made can cope with sudden access and respond to changes in business operations.

[0055] Optionally, if the frequency of historical data queries exceeds N times the average frequency within a preset time period, the third rule is satisfied. In this case, the process continues, triggering a backup request. This ensures that when business suddenly requires frequent historical data queries, the system can automatically optimize data location and guarantee query efficiency. Here, N is a positive number. For example, assuming N=20 and the preset time period is 1 hour, the second rule can be expressed as: IF (The query frequency of a historical data table surges within 1 hour, exceeding 20 times the daily average) THEN triggers a backup request. In practical applications, the hot backup request triggered in this situation is usually a hot backup. After immediately triggering a hot backup, the most recent backup file of the historical data table can be migrated from object storage back to high-performance storage to handle potential analytical query needs.

[0056] It should be noted that if none of the conditions in the first and third rules are met, the data backup system can return to the detection state, and the data detection module can continue with the next data collection.

[0057] In some embodiments of this application, when a backup requirement exists, the data backup system will further determine the data's access frequency to trigger an appropriate backup operation, thus realizing a data backup solution driven by the data's own state and business value. Specifically, the query frequency of historical data involved in the data to be backed up can be further analyzed to determine access frequency.

[0058] The determination of access popularity can be based on the judgment of the second rule, which enables intelligent data hierarchical management.

[0059] Optionally, the historical data query frequency collected by the data detection module can be used to determine the average query frequency. If the average query frequency is greater than a preset first frequency threshold, the judgment result of the second rule can be hot data; if the average query frequency is less than a preset second frequency threshold, the judgment result of the second rule can be cold data. For example, assuming the preset first frequency threshold is 100 times / day and the preset second frequency threshold is 10 times / day, the second rule can be expressed as: IF (average query frequency of relevant historical data > 100 times / day) THEN is determined to be hot data; ELSE IF (average query frequency of relevant historical data < 10 times / day) THEN is determined to be cold data.

[0060] In some embodiments of this application, the determination of backup type and storage target can be expressed as follows: if the judgment result of the second rule is hot data, then the backup type can be determined to be hot backup type and the storage target is high-performance storage, such as high-performance SSD (Solid State Drive) storage; if the judgment result is cold data, then the backup type can be determined to be cold backup type and the storage target is object storage, such as low-cost WSS3 (Web Services Simple Storage Service) / OSS (Object Storage Service) objects, etc.

[0061] This application introduces query frequency as a decision factor to achieve intelligent data lifecycle management, automatically separating hot and cold data and adopting different storage strategies for each. While ensuring high-performance access to hot data, it significantly reduces the long-term storage cost of cold data, achieving a balance between cost and performance.

[0062] It should be noted that the embodiments of this application can also dynamically configure the rules, such as configuring differentiated rules according to database instance, data type, business line, etc. For example, the change threshold of the core database can be set to 50 times, and the change threshold of the archive database can be set to 200 times. The rule version rollback function can also be added. If a new rule is accidentally triggered after configuration, it can be quickly rolled back to a historical stable version. The rule threshold can also be automatically adjusted based on historical data, and anomaly detection can be introduced. The embodiments of this application do not limit these aspects.

[0063] Step S203: Respond to the backup execution command and obtain real-time performance metrics of the database host.

[0064] If the backup trigger condition is met, i.e., a backup execution command is generated, and the database host is experiencing a business peak (such as high CPU utilization, large disk I / O latency, etc.), the execution of the backup task (especially a full hot backup) will consume a large amount of CPU, memory and I / O resources, thus creating fierce competition with the business, resulting in slower business queries, request timeouts, or even service unavailability.

[0065] To avoid conflicts between backup tasks and database host business resource usage, embodiments of this application may introduce a dynamic backup scheduler based on real-time system load.

[0066] In some embodiments of this application, based on the detected data change information and historical data query frequency, the performance indicator acquisition module can be used to add the detection of real-time performance indicators of the database host, so as to realize the resource-aware scheduling process based on the detected real-time performance indicators of the database host.

[0067] Step S204: If the real-time performance indicators of the response database host meet the fourth rule, perform the corresponding backup operation according to the backup type and store the backup file to the storage target.

[0068] The resource-aware scheduling process involves determining whether the fourth rule is met based on the real-time performance metrics of the database host. The fourth rule is primarily a resource avoidance rule; if the fourth rule is met, it means that the execution of the current backup task will not conflict with the database host's business resource usage.

[0069] Optionally, if the utilization rate does not exceed the preset percentage threshold and the disk response time does not exceed the preset second time threshold, then the fourth rule is satisfied. In this case, the corresponding backup operation can be performed according to the backup type, and the backup file can be stored to the storage target. If the utilization rate exceeds the preset percentage threshold or the disk response time exceeds the preset second time threshold, then the fourth rule is not satisfied, and the backup task will be postponed. For example, the utilization rate can refer to the current system CPU utilization rate, and the disk response time can be reflected based on the disk I / O latency. Assuming the preset percentage threshold is 80% and the preset second time threshold is 100ms, the fourth rule can be expressed as: IF (backup trigger condition satisfied) AND (current system CPU utilization rate > 80% OR disk I / O latency > 100ms) THEN postpone the execution of the backup task and mark the task as waiting for resources to become available.

[0070] Optionally, the backup priority of the backup task can be judged to meet the fifth rule. The fifth rule is mainly a resource preemption strategy. When the fifth rule is met, it means that the backup priority of the backup task is extremely high. For backup tasks with extremely high backup priority, such as in an emergency where the binlog is about to be cleared after a business peak, a forced execution mode can be set to force the corresponding backup operation to be performed according to the backup type. However, at this time, a resource limitation strategy is adopted, that is, the resource usage of the backup process is limited. This is manifested by using tools such as cpulimit (a CPU resource limitation tool under Linux system) and ionice (an I / O priority adjustment tool under Linux system) to limit the backup process when starting the backup tool (such as mysqldump or the open-source and free MySQL physical backup tool XtraBackup). For example, the CPU usage of the backup process is limited to no more than 30%, and the disk I / O scheduling priority is set to the lowest level (such as idle level), so that when the backup must be performed, the impact of the backup task on the business resources can be minimized.

[0071] Optionally, in the case of postponing backup tasks, as an example, resource metrics can be continuously monitored until the resource load drops below a safe threshold (e.g., current system CPU utilization <60% AND disk I / O latency <20ms) before triggering the backup execution operation; as another example, a global resource collaboration manager can be introduced to enable cross-node backup execution.

[0072] In an environment with multiple database nodes or clusters, there may be a situation where one node is unable to perform a backup due to resource constraints, while another node may be in a resource-idle state. In this embodiment of the application, the introduced global resource coordination manager can schedule the backup task to another database node with idle resources for execution based on the condition that the idle resource state meets the sixth rule. From a global perspective, the idle resources are used to complete the backup task, thereby realizing cross-node backup.

[0073] Specifically, a lightweight collaborative agent can be deployed on each database node. These agents are uniformly scheduled by a global coordinator, which can be a standalone server or embedded within a master node; this embodiment does not impose any limitations on this. In response to a database host's real-time performance metrics failing to meet the fourth rule, a backup task request can be sent to the global coordinator via the collaborative agent of the current database node. Upon receiving the backup task request, the global coordinator can broadcast a query for the current idle resource status to the collaborative agents of all database nodes in the resource pool, and based on the idle resource status, schedule the backup task of the current database node to the target database node, triggering the backup execution operation on the target database node.

[0074] For example, when the intelligent decision module of database node A decides to perform a backup, but needs to postpone the backup task due to busy local resources, the collaborative agent of database node A can send a backup task request to the global coordinator. The backup task request carries the data identifier and metadata to be backed up. After receiving the backup task request, the global coordinator broadcasts a query to the collaborative agents of all database nodes in the resource pool to check the current idle resource status (such as CPU, memory, disk, network, etc.). If the collaborative agent of database node B reports that its own resources are idle (such as CPU utilization < 20%), and the network between database node B and database node A is unobstructed, the global coordinator can then make a decision to schedule the backup task of database node A to be executed on database node B.

[0075] After receiving the backup task, database node B's backup execution module will remotely connect to the MySQL database on database node A and use tools such as `mysqldump -h <node A_IP>...` or `xtrabackup --host=<node A_IP>...` to directly pull data from database node B for backup. The final backup file still follows the instructions of the intelligent decision-making module and can be stored locally on database node B or written directly to shared object storage, according to the corresponding requirements.

[0076] For example, the sixth rule can be expressed as: IF (This node backup is triggered AND This node resources are busy AND There are remote node resources available AND Network latency < 50ms) THEN Trigger cross-node backup task scheduling.

[0077] In summary, in one scenario, the backup execution module can respond to the database host's real-time performance indicators to meet the fourth rule, and then, according to the instructions of the intelligent judgment module, call the corresponding backup tool to perform the operation. In another scenario, the backup execution module can respond to the fifth rule, and, with resource usage restrictions on the backup process, call the corresponding backup tool to perform the operation according to the instructions of the intelligent judgment module. In yet another scenario, the backup execution module can respond to the sixth rule but not the fourth rule, and on the target database node to which the backup task is scheduled, call the corresponding backup tool to perform the operation according to the instructions of the intelligent judgment module.

[0078] The backup execution command generated by the intelligent judgment module is based on the backup type and storage target. That is, the backup execution command carries the backup type and storage target. The backup type is used to indicate the storage level of the data, and the storage target is used to indicate the storage location. In other words, intelligent data hierarchical management can be achieved based on the selection of backup type and storage target.

[0079] Specifically, such as Figure 3 As shown, the backup execution module branch performs the backup operation.

[0080] As an example, in response to a backup execution command, for frequently accessed data, a hot backup can be performed according to the hot backup type, and the backup file can be stored on high-performance storage. For instance, hot backup tools such as mysqldump --single-transaction (--single-transaction is a core parameter of mysqldump, a logical backup tool provided by MySQL) or XtraBackup can be used for online lossless backups, and the backup files can be stored on high-performance SSD storage to ensure fast recovery and access.

[0081] As another example, for cold data, a backup can be performed as a cold backup and the backup file stored in object storage. For instance, after performing the same backup, the backup file can be transferred to low-cost object storage (such as WSS3) for archiving to save on storage costs.

[0082] It should be noted that, in cases such as Figure 3 After all steps in the illustrated process are completed, the data backup system will not stop working. Instead, it will return to the data detection step, whereby the data detection module will continue to periodically monitor the database's operational status indicators, thus forming a 24 / 7 uninterrupted, adaptive intelligent backup closed loop. This embodiment avoids unnecessary backup operations when data remains unchanged or changes are minimal, saving CPU, I / O, and network bandwidth, and improving resource utilization. Furthermore, the entire process requires no manual intervention, forming a complete closed-loop automation from detection and judgment to execution and archiving, improving the level of intelligence and operational efficiency, resulting in a higher degree of automation.

[0083] Optionally, during the backup process, backup filenames containing timestamps and trigger reasons can be automatically generated. The naming structure can adopt a structured naming rule of function identifier + trigger reason + timestamp + core related information, making it both intuitive and easy to retrieve and trace backup scenarios later. Specifically, it can be represented as: "backup_[backup type]_[timestamp]_[trigger reason / related object].[file format]". Here, "backup" is a fixed identifier for the backup file, the middle fields can be used to distinguish the backup scenario, time, and trigger conditions, and the suffix "tar.gz" indicates a standard compressed file format, i.e., tar packaging + gzip compression. "tar" is used to package multiple backup files, and "gzip" compression reduces storage usage, balancing storage efficiency and cross-platform compatibility.

[0084] For example, a hot backup file could be named backup_hot_20231027_1320_binlog_size.tar.gz, where backup is a fixed identifier for the backup file; hot indicates the hot backup type; 20231027_1320 refers to the timestamp of 13:20 on October 27, 2023; binlog_size indicates that the hot backup is triggered when the binlog file reaches a specified size, i.e., the backup is automatically triggered at this time to avoid log overflow and retain change records; tar.gz is the compressed format.

[0085] For example, a cold backup file could be named backup_cold_20231027_0800_archive.tar.gz, where backup is a fixed identifier for the backup file; cold indicates the type of cold backup; 20231027_0800 refers to 08:00 on October 27, 2023; archive indicates that the purpose of this cold backup is for archival storage; and tar.gz is the compressed format.

[0086] It should be noted that the field can also be extended, for example, by adding db_mysql to identify the database type, such as backup_hot_20231027_1320_binlog_size_db_mysql.tar.gz. The core logic is to structure and split key information. This application does not limit the extension of the field.

[0087] In some embodiments of this application, a linkage bridge for rapid recovery can be built by embedding key search fields in the file name, so that the target can be located simply by searching the file name without having to traverse all backup files during recovery.

[0088] Optionally, based on the existing rule "backup_[backup type]_[timestamp]_[trigger reason / associated object].[file format]", core fields for data range and data type can be added to form a naming rule adapted to the recovery scenario: "backup_[data type]_[backup type]_[timestamp]_[trigger reason / associated object]_[data range].[file format]". After performing data backup to obtain backup files and generating backup file names that conform to the aforementioned naming rules, the target backup file can be located by data type, timestamp, and data range.

[0089] For example, suppose the recovery requirement is: data type = db_mysql (i.e., MySQL database), timestamp = 20231027_13xx, data range = tbl_order, backup type = hot. As one example, you can directly search for the backup file name. Specifically, in the backup storage directory, you can filter by prefix (backup_db_mysql_hot), timestamp (20231027_13), and data range (tbl_order) to locate the target backup file backup_db_mysql_hot_20231027_1320_binlog_size_tbl_order.tar.gz. As another example, you can search through the index. Specifically, in the backup index, you can select "data type = MySQL, time range = 2023-10-27 13:00-14:00, data range = order table". The index will directly return the corresponding backup file name and storage path without manually browsing the directory. Then, the corresponding database recovery tool can be called to directly load the located backup file and restore only the order table data. There is no need to decompress the full backup package, and the recovery time is reduced from hours to minutes, which can realize semi-automation / full automation of the recovery process.

[0090] Optionally, after the backup is complete, cleanup tasks can be performed automatically, such as resetting the state counter, cleaning up expired binlogs, and sending success / failure notifications.

[0091] In this embodiment, by periodically collecting data change information and historical data query frequency from the database, and responding when the data change information and historical data query frequency meet preset composite logic rules, a backup execution instruction is generated. Then, by responding to the backup execution instruction, the real-time performance indicators of the database host are obtained. If the real-time performance indicators of the database host meet the fourth rule, the corresponding backup operation is performed according to the backup type carried by the instruction, and the backup file is stored in the storage target carried by the instruction. Here, data change information can be used to indicate the status of the data itself, historical data query frequency can be used to indicate the data business value, preset composite logic rules can include a first rule and a third rule, a second rule is executed after either the first rule or the third rule is met, and a fourth rule is executed after the second rule is met. By monitoring database operating status indicators in real time and making intelligent judgments based on composite logic rules, the database backup process breaks away from rigid time strategies and passive response modes. Instead, it is driven by the data's own status and business value, making it intelligent and adaptive. Furthermore, when a backup is determined to be executed, the system can also monitor the real-time performance indicators of the database host. This allows for further judgment of the database host's real-time performance indicators after the backup trigger conditions are met, avoiding conflicts between backup tasks and business resources. As a result, it achieves highly reliable, high-resource-utilization, and automated hot backup and intelligent data tiered management without human intervention.

[0092] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.

[0093] This application also provides an electronic device, which is described in reference to... Figure 4 The provided electronic device 400 includes a memory 410, a processor 420, and a computer program 411 stored in the memory 410 and capable of running on the processor 420. When the computer program 411 is executed by the processor, it implements the various processes of the above-described database data backup method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0094] This application also provides a computer-readable storage medium, see embodiments thereof. Figure 5The computer-readable storage medium 500 provides a computer program 411 stored on it. When the computer program 411 is executed by the processor, it implements the various processes of the above-described database data backup method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0095] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0096] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of the embodiments of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or devices. The division of modules in the embodiments of this application is merely a logical division; in actual applications, there may be other division methods. For example, multiple modules may be combined into or integrated into another system, or some features may be ignored or not performed. Additionally, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interface, and the indirect coupling or communication connection between modules may be electrical or other similar forms, none of which are limited in the embodiments of this application. Furthermore, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed among multiple circuit modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of this application.

[0097] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0098] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0099] In the embodiments provided in this application, it should be understood that the disclosed systems, methods, and approaches can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between systems or modules may be electrical, mechanical, or other forms.

[0100] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0101] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0102] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0103] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable system. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0104] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0105] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction set implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes; these computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0106] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0107] Finally, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0108] The technical solutions provided in the embodiments of this application have been described in detail above. Specific examples have been used in the embodiments of this application to illustrate the principles and implementation methods of the embodiments of this application. The description of the above embodiments is only for the purpose of helping to understand the methods and core ideas of the embodiments of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the embodiments of this application. Therefore, the content of this specification should not be construed as a limitation on the embodiments of this application.

Claims

1. A database data backup method, characterized in that, The method includes: The database is periodically collected for data change information and historical data query frequency; the data change information is used to indicate the status of the data itself, and the historical data query frequency is used to indicate the data business value. In response to the data change information and the historical data query frequency satisfying a preset composite logic rule, a backup execution instruction is generated. The preset composite logic rule includes a first rule and a third rule; a second rule is executed after either the first or third rule is satisfied; and a fourth rule is executed after the second rule is satisfied. The backup execution instruction includes a backup type and a storage target. The backup type includes hot backup and cold backup, and the storage target includes high-performance SSD storage and object storage. The data change information includes data change frequency and data change amount. The data change amount is quantified based on the total volume of the binary log to measure the degree of data change since the previous backup. The data change frequency is based on the data change frequency... The operation rate and the time interval since the last successful backup are measured. If the total volume of the binary log is greater than a preset file size threshold, or the data operation rate is greater than a preset rate threshold and the time interval is greater than a preset first time threshold, then the first rule is satisfied. If the historical data query frequency exceeds N times the average preset frequency within a preset time period, then the third rule is satisfied. N is a positive number. The first rule is used to indicate whether the amount of data changes or the operation traffic exceeds a threshold to ensure data security. The third rule is used to indicate whether the historical data query frequency has surged to cope with sudden business access demands. The second rule is used to indicate that data access popularity is determined based on the historical data query frequency to achieve intelligent hierarchical management. In response to the backup execution command, obtain real-time performance metrics of the database host; In response to the database host's real-time performance indicators meeting the fourth rule, the corresponding backup operation is performed according to the backup type, and the backup file is stored in the storage target; the fourth rule is used to indicate whether the database host's real-time performance indicators meet the conditions to avoid conflicts between the execution of the current backup task and the database host's business resources; In this method, a collaborative agent is deployed on each database node, and the collaborative agent is scheduled by a global coordinator. The method further includes: In response to the database host's real-time performance metrics failing to meet the fourth rule, a backup task request is sent to the global coordinator through the collaborative agent of the current database node. The backup task request instructs the global coordinator to broadcast a query for the current idle resource status to the collaborative agents of all database nodes in the resource pool, and to schedule the backup task of the current database node to the target database node based on the idle resource status. On the target database node, the corresponding backup operation is performed according to the backup type, and the backup file is stored in the storage target. During the backup process, the method further includes: Automatically generate backup file names that include timestamps and trigger reasons. The naming structure of the backup file name is: backup_[backup type]_[timestamp]_[trigger reason / associated object].[file format]. Wherein, backup is a fixed identifier used to represent backup files, and the middle fields are used to distinguish backup scenarios, time, and trigger conditions.

2. The method according to claim 1, characterized in that, The response that the data change information and the historical data query frequency meet the preset composite logic rules generates a backup execution instruction, including: If the data change information satisfies the first rule or the historical data query frequency satisfies the third rule, the historical data query frequency is judged based on the second rule to obtain a judgment result; The backup type and storage target are determined based on the judgment results; Based on the backup type and the storage target, a backup execution instruction is generated.

3. The method according to claim 2, characterized in that, The historical data query frequency is used to determine the average query frequency. When the average query frequency is greater than a preset first frequency threshold, the judgment result is hot data. When the average query frequency is less than a preset second frequency threshold, the judgment result is considered cold data. The process of determining the backup type and storage target based on the judgment result includes: If the determination result is hot data, then the backup type is determined to be a hot backup type, and the storage target is high-performance storage; If the determination result is cold data, then the backup type is determined to be a cold backup type, and the storage target is object storage.

4. The method according to claim 3, characterized in that, The real-time performance metrics of the database host include utilization and disk response time. The response that the real-time performance indicators of the database host meet the fourth rule, performing the corresponding backup operation according to the backup type, and storing the backup file to the storage target includes: If the usage rate does not exceed a preset percentage threshold and the disk response time does not exceed a preset second time threshold, then the fourth rule is satisfied, the corresponding backup operation is performed according to the backup type, and the backup file is stored in the storage target.

5. The method according to claim 3 or 4, characterized in that, The step of performing the corresponding backup operation according to the backup type and storing the backup file to the storage target includes: In response to the backup execution command, perform a hot backup according to the hot backup type, and store the backup file in the high-performance storage; Alternatively, perform a backup according to the described cold backup type and store the backup file in the object storage.

6. The method according to claim 5, characterized in that, During the process of performing the corresponding backup operation according to the backup type, the method further includes: If the backup priority of the backup task meets the fifth rule, then the corresponding backup operation shall be performed according to the backup type, while limiting the resources used by the backup process.

7. A database backup system, characterized in that, The system includes: The data detection module is used to periodically collect data change information and historical data query frequency from the database; the data change information is used to indicate the status of the data itself, and the historical data query frequency is used to indicate the data business value. An intelligent judgment module is used to generate a backup execution instruction in response to the data change information and the historical data query frequency meeting preset composite logic rules. The preset composite logic rules include a first rule and a third rule; a second rule is executed after either the first or the third rule is met; and a fourth rule is executed after the second rule is met. The backup execution instruction includes a backup type and a storage target. The backup type includes hot backup and cold backup, and the storage target includes high-performance SSD storage and object storage. The data change information includes data change frequency and data change amount. The data change amount is quantified based on the total volume of the binary log to determine the degree of data change since the previous backup. The rate is measured based on the data operation rate and the time interval since the last successful backup. If the total volume of the binary log is greater than a preset file size threshold, or the data operation rate is greater than a preset rate threshold and the time interval is greater than a preset first time threshold, then the first rule is satisfied. If the historical data query frequency exceeds N times the preset frequency average within a preset time period, then the third rule is satisfied. N is a positive number. The first rule is used to indicate whether the data change volume or the operation traffic exceeds a threshold to ensure data security. The third rule is used to indicate whether the historical data query frequency has surged to cope with sudden business access demands. The second rule is used to indicate that data access popularity is determined based on the historical data query frequency to achieve intelligent hierarchical management. The performance metrics acquisition module is used to acquire real-time performance metrics of the database host in response to the backup execution command. The backup execution module is used to respond to the database host's real-time performance indicators meeting the fourth rule, perform the corresponding backup operation according to the backup type, and store the backup file to the storage target; the fourth rule is used to indicate whether the database host's real-time performance indicators meet the conditions to avoid conflicts between the execution of the current backup task and the database host's business resources; a collaborative agent is deployed on each database node, and the collaborative agent is scheduled by a global coordinator. The backup execution module is also used to respond to the database host's real-time performance indicators not meeting the fourth rule by sending a backup task request to the global coordinator through the collaborative agent of the current database node; wherein, the backup task request is used to indicate the global... The coordinator broadcasts a query for the current idle resource status to the collaborative agents located on all database nodes in the resource pool. Based on the idle resource status, it schedules the backup task of the current database node to the target database node. On the target database node, it performs the corresponding backup operation according to the backup type and stores the backup file in the storage target. During the backup process, a backup file name containing a timestamp and trigger reason is automatically generated. The naming structure of the backup file name is: backup_[backup type]_[timestamp]_[trigger reason / associated object].[file format], where backup is a fixed identifier used to represent the backup file, and the middle field is used to distinguish the backup scenario, time, and trigger condition.

8. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the data backup method for the database as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the data backup method for the database as described in any one of claims 1 to 6.

Images