34 results about "Database backup" patented technology

The invention relates to the technical field of database backup, and particularly provides a MySQL database incremental encryption backup method and device based on MySQLdump, and the device comprises a total backup module, an incremental backup module, an encryption module and a backup management module; the full backup module adopts a MySQL official tool mysqldump to realize complete backup of the database; the incremental backup module is realized on the basis of a MySQL binary log binlog, and data changes are captured by analyzing event records in the binlog; the encryption module is constructed based on an OpenSSL library and provides enterprise-level data protection capability; and the backup management module realizes backup full-life-cycle management. Compared with the prior art, the method has the advantages that the problem of high consumption of traditional total backup resources and the safety problem can be solved, and meanwhile, the complexity and extra cost of a third-party backup tool are avoided.

In one embodiment, a method comprises generating a backup of a table of a relational database by partitioning a plurality of records of the table among a plurality of Parquet files; and extracting metadata of the table into one or more metadata files; and requesting access of at least one of the plurality of Parquet files of the backup of the relational database responsive to an SQL query entered by a user into a user interface.

Embodiments of the present application provide a database backup method and device, electronic equipment and storage medium, the method comprising: in the case of reaching a preset time node, triggering the database to refresh the data page in the shared cache area to the disk, and locking the file operation of the database switching pre-write log; after creating the basic snapshot data of the database and storing the basic snapshot data, unlocking the file operation, and listening to a plurality of pre-write logs output by the database after the file operation recovers, generating incremental log data based on the plurality of pre-write logs and storing the incremental log data; determining the starting file identifier corresponding to the earliest pre-write log in the incremental log data, and sequentially marking other pre-write logs in the incremental log data based on the starting file identifier; creating metadata information corresponding to the basic snapshot data. The present scheme realizes the non-interrupted backup of the database, guarantees the data consistency and integrity, reduces the backup data volume, and improves the backup efficiency.

One or more embodiments of the invention provide a method and device for adjusting backup capacity, and the method comprises the steps: firstly obtaining historical data, related to the backup capacity, of a database in a target historical time period, and then generating time sequence features arranged according to a time sequence according to the historical data; model input data is generated through the time sequence features and serves as input of a hybrid model. After model input data are produced, the model input data are input into a hybrid model for capacity prediction, so that the predicted capacity of the database in the future time period is obtained, and the hybrid model comprises a first model for processing nonlinear short-term features and a second model for processing linear long-term features; each model outputs a prediction result according to the model input data, and the output results of all the models are comprehensively calculated to obtain the prediction capacity of the database output by the final hybrid model in the future time period, so that the backup capacity of the database in the future time period is dynamically adjusted according to the prediction capacity.

A system and method for the device may include reading a plurality of rows of a database. In addition, the device may include generating a secondary hash value for each row of the plurality of rows. The device may include generating a plurality of row groups, each including a group of unique rows of the plurality of rows, based at least on the secondary hash value. Moreover, the device may include generating a primary hash value for each row group of the plurality of row groups. Also, the device may include updating a backup of the database based on a first-row group of the plurality of row groups, in response to determining that a primary hash value of the first-row group does not match any primary hash value associated with the backup of the database.

The application discloses an archiving log processing method and system, when a database backup task is started, the number of backup channels is increased or the number of archiving processes is reduced according to the importance of the database system, so that the speed of generating the archiving log is slowed down, the archiving log is timely backed up to the tape library during the time of the archiving log generation peak, the archiving space of the database is released, and the risk of long-time non-response of the database caused by the full archiving space of the database is avoided. In addition, during the time of the archiving log generation peak, the archiving log is timely backed up to the tape library, the archiving log backup operation does not need to be frequently performed, the performance of executing the database business can be avoided from being affected by the frequent backup operation, and thus the efficiency of executing the database business is improved.

Embodiments of the present disclosure provide a database recovery method and device, electronic equipment and storage medium, relating to the field of databases, which can be used in the field of distributed databases, for example. The method comprises: determining a target time to be recovered and an upstream database; obtaining at least one target log record related to the target time from a plurality of log files backed up by the upstream database; determining a downstream storage node corresponding to the target log record; sending each target log record to the corresponding downstream storage node through multiple threads; and storing, by the downstream storage node, a target log record with the largest transaction identifier among a plurality of target log records received for the same row index, based on the transaction identifier in each target log record among the plurality of target log records, so as to restore the downstream database to the upstream database at the target time. The embodiments of the present disclosure improve the efficiency of data recovery.

A method and system for updating backup data for a database is presented. The method includes initiating a merging operation for a defined time interval; obtaining change data capture (CDC) data generated during the defined time interval; aggregating and ordering the CDC data by a key value; identifying backup data associated with the ordered CDC data; generating a candidate data object that represents data modified during the defined time interval; deduplicating the candidate data object by comparing a hash identifier of the candidate data object with a plurality of hash identifiers stored in a repository; storing the candidate data object when the comparison determines that no matching hash identifier exists in the repository; and updating backup metadata to associate the key value with the candidate data object.

The application discloses a database physical backup verification method and device, a storage medium and electronic equipment, and relates to the technical field of database backup and recovery. The method comprises the following steps: performing slice correctness verification on a physical backup file; if the slice correctness verification is passed, performing logical integrity verification on the physical backup file; if the logical integrity verification is passed, performing recovery preview verification on the physical backup file, which comprises the following steps: simulating a database recovery process, temporarily storing data blocks inconsistent with existing data on a disk in memory during the recovery process, and attempting to start a database instance after simulating the recovery; and judging whether to perform an actual database recovery operation according to the result of the recovery preview verification. Through active and layer-by-layer verification of the integrity and recoverability of the backup file before recovery, the problem of'secondary damage' caused by the use of damaged backup is eliminated, and the problems of additional storage overhead and prolonged recovery time caused by the traditional full copy scheme are avoided.

The application relates to the technical field of database backup and synchronization, in particular to a database master-slave file adaptive fast synchronization method, which comprises the following steps: performing difference evaluation based on the meta information of the database master-slave files, directly adopting a record-by-record synchronization mode if the overall difference is too large, or entering an incremental synchronization mode; the incremental synchronization identifies inconsistent blocks by comparing the check values of the data blocks and counts the number of the inconsistent blocks; the system respectively estimates the total time consumption of the two synchronization modes according to performance parameters, the total block number, the inconsistent block number and the total record number; a secondary decision is made by comparing the estimated time consumption, and the scheme with shorter time consumption is selected for execution; and the final consistency verification is performed on the master database file and the backup database file. Through the multi-layer threshold value judgment and the dynamic time consumption estimation model, the application intelligently selects the record-by-record synchronization or the block incremental synchronization strategy, guarantees the data consistency, and significantly improves the efficiency of the database master-slave file synchronization.

This application provides a method, device, storage medium, and program product for parallel merging of database backup sets, relating to the field of distributed systems. The method includes: obtaining the dependency chain of the backup sets to be merged; obtaining data block penetration markers based on the backup set dependency chain; processing data blocks in the full backup set in parallel to a temporary directory through multiple first sub-processes; processing at least one file dependent on an incremental backup set in parallel to a temporary directory through multiple second sub-processes; merging incremental data blocks from at least one incremental backup set in parallel to a temporary directory through multiple third sub-processes; and organizing and generating a merged full backup set based on the contents of the temporary directory. This method adds parallel scheduling capabilities based on data block penetration markers, improving the execution efficiency of merging backup sets.

A method and system for reducing deduplication in database backup are provided. The method includes selecting a first primary key of a database; extracting a plurality of row groups from the database based on the selected first primary key; storing the plurality of row groups arranged according to the first primary key in a file having a column-oriented data storage format as a database backup at a first time; reading at least a portion of the database backup at a second time to detect a plurality of changes; determining a position of each row corresponding to a change of the plurality of changes; selecting a second primary key of the database as a primary key; and generating a database backup of the database at the second time based on the selected second primary key.

The invention relates to the technical field of database synchronization, and provides a data synchronization method and device based on database restoration, and the method comprises the steps that a source end synchronization tool obtains a backup file generated by a source end database; sending the backup file to a destination synchronization tool to enable the destination synchronization tool to obtain a reference database; obtaining a minimum log serial number corresponding to an active transaction in a source end database at the backup starting moment and a maximum log serial number corresponding to the backup finishing moment; reading an operation log of which the log serial number is greater than the minimum log serial number in a log file of the source end database, and modifying a submission operation of which the log serial number is less than or equal to the maximum log serial number into a rollback operation to obtain a target operation log, the target end synchronization tool carries out data synchronization on the target end database according to the reference database and the target operation log. According to the method, the consistency of the data is effectively maintained on the premise that the active transaction is automatically rolled back after the backup of the destination end database is restored.

The application provides a database backup method and device, the method comprising: obtaining database configuration information, wherein the database configuration information comprises an asynchronous backup database IP address and a backup script IP address; judging whether the backup script IP address and the asynchronous backup database IP address are the same according to the database configuration information; replacing the backup script IP address with the asynchronous backup database IP address in the case that the backup script IP address and the asynchronous backup database IP address are different; and running the backup script to perform database backup. Through the database type judgment on the database corresponding to the execution script before the database backup on the database after the completion of the master-slave database switching, the database backup is performed in the case that the database is determined to be an asynchronous backup database. The database backup on the master database is effectively avoided, the business jam caused by the database backup on the master database is avoided, the safe operation of the master database business is ensured, and the security of the database is improved.

The embodiment of the invention provides a database backup method and device, electronic equipment and a storage medium, and the method comprises the steps: triggering a database to refresh a data page in a shared cache region to a disk under the condition that a preset time node is reached, and locking the file operation of switching a pre-write log by the database; after basic snapshot data of the database is created and stored, the file operation is unlocked, multiple pre-write logs output by the database after the file operation is recovered are monitored, incremental log data are generated based on the multiple pre-write logs, and the incremental log data are stored; determining an initial file identifier corresponding to the earliest pre-written log in the incremental log data, and marking other pre-written logs in the incremental log data in sequence based on the initial file identifier; and creating metadata information corresponding to the basic snapshot data. According to the scheme, the non-interruption backup of the database is realized, the data consistency and integrity are guaranteed, the backup data volume is reduced, and the backup efficiency is improved.

A method and system for reducing deduplication in database backup are provided. The method includes selecting a first primary key of a database; extracting a plurality of row groups from the database based on the selected first primary key; storing the plurality of row groups arranged according to the first primary key in a file having a column-oriented data storage format as a database backup at a first time; reading at least a portion of the database backup at a second time to detect a plurality of changes; determining a position of each row corresponding to a change of the plurality of changes; selecting a second primary key of the database as a primary key; and generating a database backup of the database at the second time based on the selected second primary key.

The present invention provides a database backup method and apparatus, and a computing device cluster. In embodiments, a first corrupted-page inspection operation is performed on a first page in a data table of a database, to determine a first inspection result, where the first is used to record metadata of the data table; the first page is backed up when the first inspection result indicates that the first page is a normal page; a second corrupted-page inspection operation is performed on a second page in the data table based on the first page, to determine a second inspection result, where the second is used to record data in the data table; and the second page is backed up when the second inspection result indicates that the second page is a corrupted page. In this way, a backup process of the data table has a check and failure tolerance capability.

This invention relates to the field of database backup and recovery technology, and more particularly to a rapid backup and recovery system for user databases in SaaS website building systems. The system includes a message queue module for maintaining data integrity; an encryption algorithm for generating hash values ​​to verify data integrity; a real-time communication module for updating the data backup status in real time; and an anomaly detection module for detecting and troubleshooting problems in the system. By combining the message queue module with the real-time communication module, this invention effectively improves the efficiency of the user backup and recovery process, enabling real-time monitoring during backup and recovery. Furthermore, the encryption algorithm, by generating hash values, ensures database consistency, which helps prevent errors that may occur during data recovery. Additionally, the encryption algorithm enhances system security, ensuring database security and mitigating potential security risks.

This invention discloses a method and system for database backup in a value log system. The method includes the following steps: generating a checkpoint and obtaining the LSN corresponding to the current checkpoint in the value log system; obtaining an LSM tree snapshot, collecting each subtree of the LSM index tree, and then generating a linked list of SSTables to be backed up; in the LSM index tree, one SSTable corresponds to one persistent index file; one SSTable is retrieved from the linked list of SSTables to be backed up, the data of the index file corresponding to the SSTable is backed up, and stored in the backup file, and so on, retrieving the next SSTable for backup until the linked list of SSTables to be backed up is empty; obtaining the value log file of the value log system, copying it from the beginning to the LSN offset corresponding to the checkpoint, and storing it in the backup file, thus completing the backup of the value log system; the system includes: a backup device, a checkpoint module, an SSTable subtree collection module, an LSM subtree backup module, and a value log backup module. This invention can ensure the integrity of user data and restore user data using backup.

Methods, systems, and computer programs are presented for providing backup services to a database. One method includes operations for installing a backup agent in a first database, receiving information about the first database, and executing, by the backup agent, queries to the first database to determine a topology of the first database. Further, the method includes configuring, based on the topology, a receiver service of a second database for backing up the first database in the second database. The backup agent configures an interface module of the first database to back up the first database to the second database. The configuration includes an interface to the receiver service of the second database and connection information for storing data in one or more nodes of the second database. The interface module streams updates from the first database to the second database based on the configuration of the interface module.

The invention provides a database backup method, device and equipment, relates to the technical field of data storage, and aims to solve the problem that a traditional data backup method cannot ensure the integrity of data storage during multi-database storage. The database backup method comprises the following steps: acquiring a data volume stored in a to-be-backed-up database, a data type stored in the to-be-backed-up database and historical backup information of the to-be-backed-up database; the to-be-backed-up database is used for storing data acquisition of the oil and gas pipeline and monitoring and controlling operation parameters of the SCADA system; determining an initial backup time interval of the database to be backed up according to the data volume and the data type; and determining a target backup time interval of the database to be backed up according to the historical backup information, and backing up the database to be backed up according to the target backup time interval.

This invention belongs to the field of database recovery technology. It discloses a method for assessing recovery progress and status based on data acquisition and assertion during database backup. The method includes an assertion data acquisition module, an assertion judgment generation module, an assertion judgment module, and an assertion output module. The data acquisition module collects backup data as baseline data for data recovery. The assertion judgment generation module performs pre-assertion judgments based on the baseline data, using these pre-assertion judgments as assertion data for the assertion judgment module during recovery. During database recovery, different assertion judgments are executed at different recovery stages to generate assertion results. These results are then used to determine the status and progress at each stage.

The invention provides a database online backup recovery fault tolerance method and device, and belongs to the technical field of database backup, and the method comprises the following steps: setting a database to directly act on metadata and a data file when data changes; before backup is started, the database is informed that backup is about to be started, and all database services before the informing moment are executed completely; performing copy-on-write on the new database business after the notification time; the database online backup adopts a mode of tolerating backup failure of a partial table, and meanwhile, backup copy-on-write content; after backup succeeds, copy-on-write of the new database service is stopped, and backup information is recorded and comprises table information of successful backup and table information of failed backup; when the backup data is used for data recovery, recovery is carried out according to the successfully backed-up table information. According to the method and the device, the fault-tolerant capability of backup recovery is provided under the scene of large data volume, the cost is reduced, more detailed backup information is provided, and the table which is successfully recovered and backed up can be recovered and appointed.

This invention proposes a fault-tolerant method and apparatus for online database backup and recovery, belonging to the field of database backup technology. The method includes: configuring the database to directly affect metadata and data files when data changes occur; notifying the database before backup begins, waiting for all database transactions prior to the notification time to complete; performing write-time replication for new database transactions after the notification time; the online database backup adopts a mode that tolerates backup failures of some tables while simultaneously backing up the write-time replicated content; after a successful backup, stopping write-time replication for new database transactions, recording backup information, including information on successfully backed-up tables and information on tables that failed to be backed up; and restoring data using the backup data according to the information on the successfully backed-up tables. This invention provides fault-tolerant backup and recovery capabilities in scenarios with large data volumes, reduces costs, and provides more detailed backup information, allowing for the recovery and specification of successfully backed-up tables.

This application discloses a VIP binding method and apparatus based on a database backup database, applied to read / write separation scenarios in an openGauss database cluster. The method includes: upon cluster startup, a collection and preemption phase is initiated, where the cluster management server distributes backup database VIP pool information, the proxy detects and reports any existing virtual IPs, and the server arbitrates allocation; after entering the balancing phase, only normal backup databases are allowed to hold virtual IPs, and the server evenly distributes the virtual IPs in the VIP pool to each normal backup database, prioritizing those with fewer virtual IPs; virtual IPs are released when the status is abnormal or the role is switched to the primary database; and all virtual IPs are released when the cluster shuts down. This invention, through dynamic binding of virtual IPs to backup database roles, ensures that the business side always connects to the backup database via a fixed IP, achieving IP-level load balancing and role adaptation, reducing the coupling between the business layer and the database high availability mechanism, and improving system read performance and fault tolerance.

The application discloses a data backup method and system of an in-memory database, and belongs to the technical field of in-memory database backup.The application determines consistent data and log data of each data partition in the in-memory database through a write-time copy mechanism; then, the consistent data is split to obtain a plurality of data blocks, and hash fingerprint data of each data block is calculated, and then a current hash signature digest tree of each data partition is constructed based on the hash fingerprint data; subsequently, by comparing nodes of a historical hash signature digest tree and the current hash signature digest tree, incremental data of the data partition is obtained; then, the incremental data and the hash fingerprint data are used to generate incremental backup data, and the incremental backup data is associated with the log data to form backup metadata, and the in-memory database is backed up with the backup metadata, so that the efficiency of data backup of the in-memory database can be improved, and the storage space occupation and consumption can be reduced.

The application provides a data warehousing method and system based on file arrangement. The method comprises the following steps: step 1, extracting valid data from a specified data source; step 2, executing a database backup command to obtain a data arrangement format of a data backup file corresponding to a current database type; step 3, arranging the extracted valid data according to the data arrangement format of the data backup file to form a structured data set; step 4, storing the structured data set in a pre-constructed writing folder to generate a file consistent with the format of the data backup file, and cutting the file in the writing folder to a pre-constructed to-be-imported folder when the writing is completed; and step 5, executing a data incremental loading command to load the file in the to-be-imported folder into a database, and cutting the file in the to-be-imported folder to a pre-constructed warehoused folder when the loading is completed.

The embodiment of the invention discloses a database data parallel backup method and device, a database data parallel restoration method and device, equipment and a medium, and the method comprises the steps that in response to a parallel backup instruction, to-be-backed-up data is obtained; splitting the to-be-backed-up data based on a preset split block size to determine a split data block set; each split data block corresponds to one backup task; determining a target backup parallelism degree according to the total number of the split data block sets and the initial backup parallelism degree; according to the use state of each backup thread in the backup task thread group set, distributing a corresponding target backup task thread group for each backup task; the backup task thread group set is created based on the target backup parallelism degree; and performing data backup on the corresponding split data blocks in parallel by utilizing the target backup task thread groups corresponding to the backup tasks respectively. According to the technical scheme, the read-write bandwidth and the CPU computing power of equipment are fully utilized, and the overall performance of the database backup and restoration process is improved.

The present specification relates to the technical field of computers, in particular to a data access method and device, computer equipment and readable medium. The data access method comprises: upon receiving a first data storage request, determining a service identifier according to a to-be-stored data identifier included in the first data storage request; processing a user identifier included in the first data storage request by using a processing algorithm corresponding to the service identifier to obtain seed information; storing the seed information in an available database instance; and storing the first data corresponding to the seed information in a first target database based on the available database instance and the seed information. According to the embodiments of the present specification, the seed information of the first data can be stored in the available database instance, thereby achieving the need for master-slave database backup, reducing the backup time and saving the computing resources.

The application discloses a database backup method, device, equipment, storage medium and product, and comprises the following steps: identifying external storage media corresponding to each database node in a distributed database, and recording the mapping relationship between each external storage media and the corresponding database node; according to the mapping relationship, mounting the path of each external storage media to the storage directory of the corresponding database node to obtain the backup directory corresponding to each database node; and in response to a data backup request, backing up the target data corresponding to each database node to the corresponding backup directory. The technical scheme of the embodiment of the application can improve the data backup and recovery efficiency in the distributed database, and ensure the reliability of the data backup result and the data recovery result.