Database migration method and device, equipment, storage medium and program product

Abstract

The invention relates to a database migration method and device, equipment, a storage medium and a program product, and the method comprises the steps: when it is detected that migration from a source database to a target database is needed, data in the source database is backed up, and the backed-up data is transmitted to the target database; and then under the condition that it is determined that the backed-up data is successfully transmitted to the target database, performing read-write service verification on the target database, and migrating the read-write service to the target database under the condition that the verification is passed. Wherein the data comprises initial full-amount data and incremental data generated in the process of backing up the initial full-amount data; and the source database is in a business read-write state. The method can realize smooth migration.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a database migration method, apparatus, device, storage medium, and program product. Background Technology

[0002] With the rapid development of information technology, the amount of data across industries has exploded, making the storage, management, and efficient access of massive amounts of data a significant challenge for enterprises. Against this backdrop, migrating on-premises source databases to the cloud has become a key measure for enterprises to achieve digital transformation, reduce operational costs, and improve business agility. Cloud databases, with their advantages of elastic scaling, high availability, automatic backup, and disaster recovery, are gradually replacing traditional source databases and becoming the preferred data storage solution for modern applications.

[0003] However, currently, it is difficult to achieve a smooth transition when migrating source databases to cloud databases, which affects business execution efficiency. Summary of the Invention

[0004] Therefore, it is necessary to provide a database migration method, apparatus, device, storage medium, and program product that can improve business execution efficiency in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a database migration method, which includes:

[0006] When a migration from the source database to the target database is detected, the data in the source database is backed up, and the backed-up data is transferred to the target database. The data includes the initial full data and the incremental data generated during the backup of the initial full data. The source database is in a business read / write state.

[0007] Once it is confirmed that the backed-up data has been successfully transferred to the target database, the read and write services of the target database are verified, and if the verification passes, the read and write services are migrated to the target database.

[0008] In some embodiments, the read-write service includes a read service and a write service. The read-write service is validated against the target database, and if the validation passes, the read-write service is migrated to the target database, including:

[0009] Validate write operations to the target database, and if the write operation validation passes, switch the write service from the source database to the target database;

[0010] After switching the write service to the target database, the read operations of the target database are validated, and if the read operation validation passes, the read service is switched from the source database to the target database.

[0011] In some embodiments, write operations to the target database are validated, including:

[0012] Obtain the first time incremental data is written to the source database, and the second time incremental data is written to the target database;

[0013] The time difference between the first and second times is compared with a preset time threshold. If the time difference is less than or equal to the preset time threshold, the write operation is deemed to have passed the verification. If the time difference is greater than the preset time threshold, the write operation is deemed to have failed the verification.

[0014] In some embodiments, the method further includes:

[0015] After switching the write service to the target database, the data generated by the write service in the target database is downloaded to the source database, until the read service is switched from the source database to the target database.

[0016] In some embodiments, data in the source database is backed up, and the backed-up data is transferred to the target database, including:

[0017] Perform a full backup of the initial full data in the source database, and then transfer the backup data to the target database;

[0018] If the initial full data backup is confirmed to be successful, acquire the incremental data generated during the backup process.

[0019] Perform incremental backups on the incremental data and then transfer the backup data to the target database.

[0020] In some embodiments, the method further includes:

[0021] Upload data from the source database to the cloud object storage service; the cloud object storage service is configured independently of the database.

[0022] The backed-up data is transferred to the target database, including:

[0023] Download the backed-up data from the object storage service to the target database.

[0024] Secondly, this application also provides a database migration apparatus, which includes:

[0025] The data migration module is used to back up the data in the source database and transfer the backed-up data to the target database when it is detected that a migration from the source database to the target database is required. The data includes the initial full data and the incremental data generated during the backup of the initial full data. The source database is in a business read / write state.

[0026] The service migration module is used to verify the read and write services of the target database after confirming that the backed-up data has been successfully transferred to the target database, and to migrate the read and write services to the target database if the verification passes.

[0027] Thirdly, this application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0028] When a migration from the source database to the target database is detected, the data in the source database is backed up, and the backed-up data is transferred to the target database. The data includes the initial full data and the incremental data generated during the backup of the initial full data. The source database is in a business read / write state.

[0029] Once it is confirmed that the backed-up data has been successfully transferred to the target database, the read and write services of the target database are verified, and if the verification passes, the read and write services are migrated to the target database.

[0030] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0031] When a migration from the source database to the target database is detected, the data in the source database is backed up, and the backed-up data is transferred to the target database. The data includes the initial full data and the incremental data generated during the backup of the initial full data. The source database is in a business read / write state.

[0032] Once it is confirmed that the backed-up data has been successfully transferred to the target database, the read and write services of the target database are verified, and if the verification passes, the read and write services are migrated to the target database.

[0033] Fifthly, this application also provides a computer program product, which includes a computer program that, when executed by a processor, performs the following steps:

[0034] When a migration from the source database to the target database is detected, the data in the source database is backed up, and the backed-up data is transferred to the target database. The data includes the initial full data and the incremental data generated during the backup of the initial full data. The source database is in a business read / write state.

[0035] Once it is confirmed that the backed-up data has been successfully transferred to the target database, the read and write services of the target database are verified, and if the verification passes, the read and write services are migrated to the target database.

[0036] The aforementioned database migration method, apparatus, equipment, storage medium, and program products, when detecting a need to migrate from a source database to a target database, back up the data in the source database and transfer the backed-up data to the target database. Then, after confirming successful data transfer to the target database, perform read / write service verification on the target database. If the verification passes, migrate the read / write service to the target database. The data includes the initial full data and incremental data generated during the backup of the initial full data. The source database is in a read / write state. In this method, during the migration process between the source and target databases, in addition to synchronizing the initial full backup data, the incremental data during the migration process is also synchronized. This incremental data synchronization strategy ensures data consistency between the source and target databases at all times, providing a significant advantage for migration rollback. If the target database fails, a seamless rollback to the source database can be implemented immediately, reducing service downtime and achieving a smooth migration. Furthermore, the source database maintains read / write services throughout the migration process. Before switching read / write services, the target database's read / write services are verified; if the verification succeeds, the switch is initiated; if the verification fails, a rapid rollback is possible. This read / write service verification strategy can ensure the stability of migration and the availability of read / write services. Attached Figure Description

[0037] Figure 1 This is one of the application environment diagrams for the database migration method in some embodiments;

[0038] Figure 2 This is one of the flowcharts illustrating the database migration method in some embodiments;

[0039] Figure 3 This is a second flowchart illustrating the database migration method in some embodiments;

[0040] Figure 4 This is the third flowchart illustrating the database migration method in some embodiments;

[0041] Figure 5 This is the fourth flowchart illustrating the database migration method in some embodiments;

[0042] Figure 6 This is the second application environment diagram for the database migration method in some embodiments;

[0043] Figure 7 This is the fifth flowchart illustrating the database migration method in some embodiments;

[0044] Figure 8 A schematic diagram illustrating the smooth migration process for incremental data synchronization and read / write service verification;

[0045] Figure 9 This is a structural block diagram of the database migration device in some embodiments;

[0046] Figure 10 This is a diagram showing the internal structure of a computer device in some embodiments. Detailed Implementation

[0047] In the embodiments of this application, the term "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0048] In the embodiments of this application, the term "multiple" refers to two or more, and other quantifiers are similar.

[0049] In the embodiments of this application, the term "at least one" means one or more. For example, at least one of A, B and C can represent six situations: A exists alone, B exists alone, C exists alone, A and B exist simultaneously, A and C exist simultaneously, B and C exist simultaneously, and A, B and C exist simultaneously.

[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0051] A cloud database is a database hosted by a cloud service provider and accessed and managed via the internet. Compared to traditional source databases, cloud databases eliminate the need for users to manage and maintain the underlying infrastructure, greatly simplifying operations and maintenance. Leveraging the elastic scalability of cloud computing, cloud databases can dynamically adjust resource allocation based on business needs, flexibly responding to business changes. Furthermore, the high availability, automatic backup, and disaster recovery capabilities of cloud platforms further enhance data security and system reliability. With the continuous growth of massive amounts of data, cloud databases have become a key pillar for data storage and access in modern application systems, and more and more enterprises are migrating their databases to the cloud to improve business efficiency, reduce costs, and enhance data security.

[0052] With the increasing demand for database migration to the cloud, migrating on-premises databases to cloud-based database instances has become a key technical challenge for enterprises in achieving digital transformation and cloud architecture. Current mainstream database migration solutions primarily rely on backup and recovery technologies. Typically, this involves first pausing read and write operations on the local database and performing a full backup. Then, using the backup file and corresponding recovery mechanisms, the local data is migrated to the cloud database instance. After migration, the stopped read and write services are switched from the local database to the cloud database instance. This technical solution has the following problems: 1. The complete backup file of the local database is large, and directly migrating it to the cloud database instance will occupy additional storage space on the cloud instance; 2. The migration process usually requires stopping read and write services on the local database, resulting in database unavailability during the migration period and making a smooth transition difficult; 3. If the read and write services of the migrated cloud instance fail, the read and write services need to be switched back to the local database, further extending the database downtime and increasing the risk of data migration. Therefore, how to efficiently, securely, and smoothly migrate on-premises databases to cloud database instances while ensuring business operations continue during the migration process is a critical technical challenge that urgently needs to be addressed in the current field of database migration to the cloud.

[0053] In view of this, embodiments of this application propose a database migration method, apparatus, device, storage medium, and program product, which can reduce service downtime and achieve smooth migration by synchronizing incremental data during the migration process between the source database and the target database, in addition to synchronizing the initial full backup data.

[0054] It should be noted that the beneficial effects or technical problems solved by the embodiments of this application are not limited to this one, but may also be other implicit or related problems. For details, please refer to the description of the embodiments below.

[0055] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0056] In some embodiments, the database migration method provided in this application can be applied to, for example... Figure 1In the application environment shown, computer device 101 communicates with source database 102 and target database 103 via a network. When it is necessary to migrate source database 102 to target database 103, computer device 101 is used to migrate all data and running business on source database 102 to target database 103, realizing database migration to the cloud. Database migration to the cloud refers to migrating a database system currently allowed on a local server to a cloud database service provided by a cloud service provider. By hosting the database in the cloud, enterprises can fully utilize the elastic scalability of cloud computing, adjust resources on demand, and meet business growth needs. At the same time, enterprises can rely on the high availability, automatic backup, and disaster recovery capabilities of the cloud platform to improve data security and system reliability. Computer device 101 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. Source database 102 can be integrated on a server or placed on the cloud or other network servers. Target database 103 can be integrated on a server or placed on the cloud or other network servers. Source database 102 and target database 103 are set up independently.

[0057] Those skilled in the art will understand that Figure 1 The structure shown is merely a block diagram of a portion of the structure related to the solution of this application and does not constitute a limitation on the application environment in which the solution of this application is applied. The specific application environment may include more or fewer components than shown in the figure, or a combination of certain components, or different component arrangements.

[0058] In some embodiments, such as Figure 2 As shown, a database migration method is provided, which can be applied to... Figure 1 Taking a computer device as an example, the explanation includes the following steps:

[0059] S201: When it is detected that a migration from the source database to the target database is required, the data in the source database is backed up and the backed-up data is transferred to the target database; the source database is in a business read / write state.

[0060] The data includes the initial full dataset and incremental data generated during the backup process of the initial full dataset. The initial full dataset refers to the complete data set in the source database at the start of the migration, while the incremental data refers to new data changes that occur in the source database during the backup process. The source database is the local database system to be migrated. The target database is a cloud-based database instance.

[0061] In this embodiment of the application, when the computer device detects that it needs to migrate from the source database to the target database, the computer device can select an appropriate backup tool according to the type of the source database (such as MySQL, Oracle, etc.) to perform a full physical backup of the source database and obtain the initial full data.

[0062] During the initial full data backup process, the computer device can continuously monitor data changes in the source database, capture incremental data, and then transmit the initial full data and incremental data to the target database via a high-speed network. Optionally, after the initial full data backup is complete, the computer device can acquire the incremental data generated in the source database during the initial full data backup process, and then transmit the incremental data to the target database via a high-speed network.

[0063] During the initial full data backup process, incremental data is generated during the backup process. At this time, the data in the source database and the cloud database are inconsistent, meaning that the data capabilities of the cloud database are not yet complete. To ensure that the read and write services are not interrupted, the read and write services during this process are provided by the source database. In other words, the source database always maintains a business read and write state during this process.

[0064] S202: If the backup data is successfully transferred to the target database, the read and write services of the target database are verified, and if the verification passes, the read and write services are migrated to the target database.

[0065] Among them, read and write service verification refers to verifying the data consistency and service availability of the target database through comparative testing, which includes two stages: write operation verification and read operation verification.

[0066] In this embodiment, after confirming that all backed-up data has been successfully transmitted to the target database, the computer device can perform read / write service verification on the target database. This simulates actual business operations and tests the read / write performance and data accuracy of the target database. Then, it checks whether the target database's transaction processing capabilities and concurrent access performance meet business requirements. Optionally, the computer device can simultaneously send the same test read / write requests to both the source and target databases, compare their response times and data persistence results, and verify the target database's read / write service based on these responses and persistence results.

[0067] After the read / write service verification of the target database passes, the read / write service of the business system can be migrated from the source database to the target database. Specifically, the database connection configuration file of the business system can be modified to point to the target database. Optionally, during the switchover process, it is necessary to ensure business continuity and avoid data loss or business interruption. A phased switchover approach can be adopted, first switching a portion of business traffic to the target database, observing and testing for a period of time, and then switching all business traffic over after confirming that everything is working correctly.

[0068] The database migration method provided in this application, when detecting a need to migrate from a source database to a target database, backs up the data in the source database and transmits the backed-up data to the target database. Then, after confirming the successful transmission of the backed-up data to the target database, it verifies the read / write service of the target database. If the verification passes, the read / write service is migrated to the target database. The data includes the initial full data and incremental data generated during the backup of the initial full data. The source database is in a read / write state. In this method, during the migration between the source and target databases, in addition to synchronizing the initial full backup data, the incremental data during the migration process is also synchronized. This incremental data synchronization strategy ensures data consistency between the source and target databases at all times, providing a significant advantage for migration rollback. If the target database fails, it can immediately and without interruption rollback to the source database, reducing service downtime and achieving a smooth migration. Furthermore, the source database maintains read / write services throughout the migration process. Before switching read / write services, the target database's read / write service is verified; if the verification succeeds, the switch is initiated; if the verification fails, a rapid rollback is possible. This read / write service verification strategy ensures the stability of the migration and the availability of the read / write service.

[0069] In some embodiments, a specific implementation method for verifying read and write services of the target database is also provided, such as... Figure 3 As shown, the phrase "perform read / write service verification on the target database, and migrate the read / write service to the target database if the verification passes" in S202 above includes:

[0070] S301 verifies the write operation to the target database, and if the write operation verification passes, switches the write service from the source database to the target database.

[0071] The reading and writing services include reading services and writing services.

[0072] In this embodiment, after confirming that the backed-up data has been successfully transmitted to the target database, the computer device can perform write operation verification on the target database. This is achieved by simultaneously sending the same test write request to both the source and target databases, comparing their response times and data persistence results, and verifying the write operation on the target database based on whether their response times and data persistence results are consistent. If the response times and data persistence results are consistent, the write operation verification passes; otherwise, it fails. If the verification passes, the write service is migrated to the target database. If the verification fails, the write verification operation can be retried. If the verification still fails after a preset number of retries, a migration rollback operation is performed, meaning the write service is not switched and continues to be provided by the source database.

[0073] Optionally, the method for verifying write operations to the target database may further include: obtaining the first time when incremental data is written to the source database and the second time when incremental data is written to the target database. The time difference between the first and second times is compared with a preset time threshold. If the time difference is less than or equal to the preset time threshold, the write operation verification is deemed successful; if the time difference is greater than the preset time threshold, the write operation verification is deemed unsuccessful.

[0074] S302: After switching the write service to the target database, the read operations of the target database are verified, and if the read operation verification passes, the read service is switched from the source database to the target database.

[0075] In this embodiment, after determining to switch the write service to the target database, the computer device can perform read operation verification on the target database. This is done by simultaneously sending the same test read request to both the source and target databases, comparing their response times and data persistence results, and verifying the read operation on the target database based on whether their response times and data persistence results are consistent. If their response times and data persistence results are consistent, the read operation verification passes; otherwise, it fails. If the verification passes, the read service is migrated to the target database. If the verification fails, the read verification operation can be retried. If the verification still fails after a preset number of retries, a migration rollback operation is performed, restoring the write service to the switched state while the read service remains unchanged, with the read service still provided by the source database.

[0076] Optionally, the above method for verifying read operations on the target database may further include: obtaining a preset value, such as 5000, and then determining whether a data record of the preset value can be read from the cloud database. If a data record of the preset value can be read from the cloud database, the read operation verification is determined to be successful. If the value of the read data record is not the preset value or a data record of the preset value cannot be read from the cloud database, the read operation verification is determined to be unsuccessful.

[0077] The method described in this application embodiment enables a smooth transition of business traffic from the source database to the target database by gradually switching between write and read services. After the write service switch is completed, the business system can first perform write operations on the target database while continuing to read data from the source database, which can reduce the impact of the switchover process on the business to a certain extent. After the read service switch is completed, the business system will be fully dependent on the target database, which ensures that the target database has been fully tested and verified and can stably support business operations.

[0078] In some embodiments, a specific implementation method is also provided for synchronizing data generated by write services in the target database to the source database. Figure 3 The database migration method also includes:

[0079] After switching the write service to the target database, the data generated by the write service in the target database is downloaded to the source database, until the read service is switched from the source database to the target database.

[0080] In this embodiment, after switching the write service to the target database, the computer device can download the data generated by the write service in the target database to the source database until the read service switches from the source database to the target database, ensuring data consistency between the target and source databases. Furthermore, in the event of an emergency in the target database, a copy of the data in the source database can serve as a basis for recovery, reducing the risk of data loss and ensuring data security and integrity.

[0081] In some embodiments, a specific implementation method for backing up data in the source database is also provided, such as... Figure 4 As shown, the phrase "backing up the data in the source database and transferring the backed-up data to the target database" in S201 above includes:

[0082] S401 performs a full backup of the initial full data in the source database and then transfers the backed-up data to the target database.

[0083] In this embodiment of the application, when performing a full backup of the initial full data in the source database, a consistent snapshot of the source database is first created to ensure the integrity and consistency of the backup data; then the backup data is transmitted to the target database through a high-speed network. During this process, the source database continues to provide business services normally, maintaining the continuity of read and write operations.

[0084] S402, if the initial full data backup is confirmed to be successful, acquire the incremental data generated during the backup process of the initial full data.

[0085] Incremental data includes at least one of the records that have undergone insertion, update, and deletion operations.

[0086] In this embodiment of the application, after confirming that the initial full data backup is successful, the start and end times of the full backup are determined by database log parsing technology. Then, the incremental data generated by the source database during this time period is determined. Thus, the incremental data generated by the source database during the initial full data backup process can be obtained, and the data change records of the source database can be captured in real time to ensure that no data operation is missed.

[0087] S403 performs incremental backups of the incremental data and then transfers the backup data to the target database.

[0088] In this embodiment, after obtaining incremental data, the computer device can back up the incremental data and save it as a separate backup file. The incremental backup method can be selected according to the characteristics and needs of the database; for example, the incremental data can be exported as an SQL script or a binary file. The incremental backup file can be transferred from the source database server to the target database server using the same or similar method as the full data transfer. Optionally, the incremental data can also be compressed and encrypted, and then the backup data can be transferred to the target database through a dedicated data transfer channel. The target database uses a log replay mechanism to merge the incremental data with the full data, ensuring data integrity and consistency.

[0089] The method described in this application embodiment can ensure the complete migration of basic data through full backup, achieve real-time synchronization of changed data through incremental data capture and backup, and guarantee the accuracy and consistency of data through secure and reliable data transmission, thus significantly improving migration efficiency and reliability.

[0090] In some embodiments, a method for database migration based on cloud object storage services is also provided, such as... Figure 5 As shown, Figure 2 The database migration method also includes:

[0091] S501 uploads data from the source database to the cloud object storage service; the cloud object storage service is configured independently of the database.

[0092] Among them, cloud object storage service refers to providing massive, secure, and low-cost cloud storage services that are independent of the database system and access data through standard API interfaces.

[0093] In the embodiments of this application, such as Figure 6 As shown, this step involves three media: the source database 102, the target database 103, and the cloud object storage service 104. The source and target databases do not directly transfer files; instead, the cloud object storage service is used as an intermediary to improve data transmission efficiency and save storage space on the cloud database instance.

[0094] When a migration from the source database to the target database is detected, a full physical backup of the source database is first performed. Then, using a high-speed network, the backup file generated from the full physical backup is uploaded to the cloud object storage service. During the transfer of the full backup file, the source database remains in a state of read and write operations. The computer equipment monitors the data changes in the source database in real time through the database's log system and captures incremental data.

[0095] Correspondingly, when the computer device executes "transfer the backed-up data to the target database" in S201, it specifically performs the following:

[0096] S502 downloads the backed-up data from the object storage service to the target database.

[0097] The download process refers to the operation of obtaining backup data from the cloud object storage service and restoring it to the target database.

[0098] In this embodiment, the target database downloads the previously uploaded full physical backup file from the cloud object storage service. Based on the type and characteristics of the target database, a corresponding database recovery mechanism is used to restore the full data to the target database. After the full data recovery is complete, the target database synchronizes the incremental data previously uploaded to the object storage service to the target database via log replay. Specifically, the log files corresponding to the incremental data are downloaded from the object storage service, and these data change operations are re-executed in the target database according to the log record order, thereby ensuring that the data in the target database remains consistent with the source database. Throughout the entire data backup, transmission, recovery, and synchronization process, the source database maintains normal business read and write services, ensuring that business operations are not affected by data migration operations.

[0099] Correspondingly, when the computer device executes the step "after switching the write service to the target database, download the data generated by the write service in the target database to the source database, until the read service is switched from the source database to the target database", it specifically performs the following: after switching the write service to the target database, transfer the data generated by the write service in the target database to the cloud object storage service, and download it from the cloud object storage service to the source database, until the read service is switched from the source database to the target database.

[0100] The method described in this application uses object storage mechanism to solve the problems of transmission efficiency and storage space during the migration process. Object storage is added as a transmission medium between the source database and the target database. With the elastic scalability of object storage and the transmission advantages of the cloud platform, resource configuration is dynamically adjusted according to the size of the source database. This does not require additional storage space occupied by the cloud database instance, thus achieving efficient and secure data transmission and improving migration efficiency.

[0101] In summary, based on all the above embodiments, a database migration method is also provided, the method comprising:

[0102] S601: When it is detected that a migration from the source database to the target database is required, a full backup of the initial full data in the source database is performed, and the backup data is uploaded to the cloud object storage service. The backup data is also downloaded from the object storage service to the target database. The source database is in a business read / write state.

[0103] S602: If the initial full data has been successfully transmitted to the target database, the incremental data generated during the backup process of the initial full data is obtained, and the source database is in a business read / write state.

[0104] S603 performs incremental backups of incremental data, uploads the backup data to the cloud object storage service, and downloads the backup incremental data from the object storage service to the target database. The source database is in a business read / write state.

[0105] S604: If it is confirmed that the backed-up data has been successfully transmitted to the target database, obtain the first time when incremental data is written to the source database and the second time when incremental data is written to the target database.

[0106] S605: Compare the time difference between the first time and the second time with a preset time threshold. If the time difference is less than or equal to the preset time threshold, the write operation is determined to pass the verification. If the time difference is greater than the preset time threshold, the write operation is determined to fail the verification. If the write operation is verified to pass, the write service is switched from the source database to the target database.

[0107] S606: After switching the write service to the target database, the read operations of the target database are verified, and if the read operation verification passes, the read service is switched from the source database to the target database.

[0108] S607, after switching the write service to the target database, transmits the data generated by the write service in the target database to the cloud object storage service, and downloads it from the cloud object storage service to the source database, until the read service switches from the source database to the target database.

[0109] This application aims to provide a method for smooth database migration to the cloud based on object storage, optimizing the issues of transmission efficiency, storage space, smooth migration, and read / write service interruption encountered during traditional database cloudification. This method first leverages the advantage of object storage's dynamic adjustment of storage capacity to solve the transmission efficiency and storage space problems during migration. Then, it employs a two-stage incremental data synchronization strategy and read / write operation verification to ensure uninterrupted and continuous normal operation of database read / write services during migration, achieving smooth migration and reducing data migration risks. This embodiment involves two main modules: a backup data file transfer process using object storage as the core medium, and a smooth migration process using incremental data synchronization and read / write service verification.

[0110] The overall migration process with object storage as the core medium is as follows: Figure 7 As shown, firstly, a full physical backup of the source database is performed, and then the full backup file is uploaded to the cloud platform's object storage. Next, the cloud database instance downloads the backup file from the object storage and uses the recovery mechanism to restore the full data to the cloud database on the same cloud platform. Throughout this process, read and write services are always provided by the source database until the read and write services are switched. Subsequently, incremental data from the source database is uploaded and restored to the cloud database on the same cloud platform via object storage. After successful incremental data restoration, read and write operations on the cloud database are verified. If the verification is successful, the read and write services are switched to the cloud database. If the verification fails, the read and write services are rolled back to the source database. Throughout the migration process, because the data in the original database and the cloud database remains completely consistent, the switching and rollback of read and write recovery are uninterrupted, achieving the goal of a smooth migration.

[0111] For example, such as Figure 8As shown, the smooth migration process of read and write services using an incremental data synchronization strategy is divided into five stages. Stage 1: Stage 1 first performs a full physical backup of the source database and restores the full backup to the cloud database using object storage services and the cloud database's recovery mechanism. At this stage, the cloud database's data capabilities are not yet complete, and read and write services are provided by the source database. Stage 2: Stage 2 mainly handles the incremental backups generated by the source database during Stage 1 and verifies the write operations of the cloud database. The incremental backups of the source database are also restored to the cloud database using object storage as the medium. During the incremental recovery of the source database, read and write services remain provided by the source database. After the incremental recovery of the source database is complete, the write operations of the cloud database are verified. If the verification is successful, the database write service is switched to the cloud database, while the read service remains provided by the source database, proceeding to Stage 3; if the verification fails, a migration rollback operation is performed, meaning the write service is not switched, the source database remains provided, and the process reverts to Stage 1. In Phase Two, the write operation verification method is as follows: a small pre-set threshold is used to obtain the time when incremental data is written from the source database to the source database and the time when incremental data is written from the source database to the cloud database. If the time difference between the two is within the threshold range, the write verification is considered successful; otherwise, the verification fails. The advantage of incremental synchronization in Phase Two is that, since the data in the source and target databases remains consistent, a rollback can be performed immediately without interruption, ensuring that the database write service is always available. Phase Three: In Phase Three, the write service has been switched to the cloud database. At this point, object storage is still needed as an intermediary to download the incremental data provided by the cloud database and restore it to the source database to ensure data consistency between the target and source databases. After synchronization is completed, Phase Four begins. Phase Four: Phase Four will verify the read operations of the cloud database. If the verification is successful, the read service will be switched to the cloud database, and Phase Five will begin. If the verification fails, it will revert to Phase Three, meaning the read service will not be switched, and the source database will continue to provide services. Phase Five: In this phase, all data and read / write services of the source database have been successfully migrated to the cloud database, and the cloud database will stably provide read / write services to the outside world. The data migration to the cloud is now complete.

[0112] The method described in this application, by introducing object storage services and pre-defined smooth migration strategies into database migration technology, effectively improves migration efficiency and service availability. It achieves efficient and secure data transmission, reduces service downtime, improves migration stability, and lowers data migration risks. This embodiment is applicable to optimizing migration efficiency and improving migration stability for various on-premises databases migrating to cloud databases. It can optimize the transmission efficiency, storage space, smooth migration, and read / write service interruption issues faced during traditional database cloudification processes, improving product performance and user experience.

[0113] The methods described in each of the above steps have been described in the foregoing embodiments. For details, please refer to the foregoing descriptions. They will not be repeated here.

[0114] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0115] Based on the same inventive concept, this application also provides a database migration apparatus for implementing the database migration method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more database migration apparatus embodiments provided below can be found in the limitations of the database migration method described above, and will not be repeated here.

[0116] In some embodiments, such as Figure 9 As shown, a database migration apparatus is provided, comprising:

[0117] The data migration module 11 is used to back up the data in the source database and transfer the backed-up data to the target database when it is detected that migration from the source database to the target database is required. The data includes the initial full data and the incremental data generated during the backup of the initial full data. The source database is in a business read / write state.

[0118] Service migration module 12 is used to verify the read and write services of the target database after confirming that the backed-up data has been successfully transferred to the target database, and to migrate the read and write services to the target database if the verification passes.

[0119] In some embodiments, the service migration module described above includes:

[0120] The first verification unit is used to verify the write operations of the target database, and if the write operation verification passes, the write service is switched from the source database to the target database.

[0121] The second verification unit is used to verify the read operations of the target database after the write service is switched to the target database, and to switch the read service from the source database to the target database if the read operation verification passes.

[0122] In some embodiments, the first verification unit includes:

[0123] The sub-unit is used to obtain the first time when incremental data is written to the source database and the second time when incremental data is written to the target database.

[0124] The verification subunit is used to compare the time difference between the first time and the second time with a preset time threshold. If the time difference is less than or equal to the preset time threshold, the write operation is determined to pass the verification. If the time difference is greater than the preset time threshold, the write operation is determined to fail the verification.

[0125] In some embodiments, the service migration module further includes:

[0126] The synchronization unit is used to download the data generated by the write service in the target database to the source database after the write service is switched to the target database, until the read service is switched from the source database to the target database.

[0127] In some embodiments, the data migration module described above includes:

[0128] The first migration unit is used to perform a full backup of the initial full data in the source database and then transfer the backup data to the target database.

[0129] The acquisition unit is used to acquire incremental data generated during the backup process of the initial full data, provided that the initial full data backup has been successfully completed.

[0130] The second migration unit performs incremental backups of the incremental data and then transfers the backup data to the target database.

[0131] In some embodiments, the data migration module further includes:

[0132] The upload unit is used to upload data from the source database to the cloud object storage service; the cloud object storage service is set up independently from the database.

[0133] The download unit is used to download backed-up data from the object storage service to the target database.

[0134] Each module in the aforementioned database migration device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0135] In some embodiments, a computer device is provided, which may be a terminal or a server, and its internal structure diagram may be as follows. Figure 10As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a database migration method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0136] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0137] In some embodiments, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the database migration method described in any of the above embodiments.

[0138] The computer device provided in the above embodiments has a similar implementation principle and technical effect to the above method embodiments, and will not be described again here.

[0139] In some embodiments, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the database migration method described in any of the above embodiments.

[0140] The computer-readable storage medium provided in the above embodiments has a similar implementation principle and technical effect to the above method embodiments, and will not be described again here.

[0141] In some embodiments, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the database migration method described in any of the above embodiments.

[0142] The computer program product provided in the above embodiments has a similar implementation principle and technical effect to the above method embodiments, and will not be described again here.

[0143] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0144] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0145] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A database migration method characterized by, The method comprises: When it is detected that migration from a source database to a target database is required, backing up data in the source database and transmitting the backed-up data to the target database; the data comprises initial full data and incremental data generated in the process of backing up the initial full data; the source database is in a business read-write state; In the case where it is determined that the backed-up data is successfully transmitted to the target database, performing read-write service verification on the target database, and in the case where the verification is passed, migrating the read-write service to the target database.

2. The method of claim 1, wherein, The read-write service comprises read service and write service, and the read-write service verification on the target database and the migration of the read-write service to the target database in the case where the verification is passed comprise: Verifying a write operation of the target database, and in the case where the write operation verification is passed, switching the write service from the source database to the target database; After the write service is switched to the target database, verifying a read operation of the target database, and in the case where the read operation verification is passed, switching the read service from the source database to the target database.

3. The method of claim 2, wherein, The write operation verification on the target database comprises: Obtaining a first time when the incremental data is written into the source database and a second time when the incremental data is written into the target database; Comparing a time difference between the first time and the second time with a preset time threshold, and in the case where the time difference is less than or equal to the preset time threshold, determining that the write operation verification is passed, and in the case where the time difference is greater than the preset time threshold, determining that the write operation verification is not passed.

4. The method of claim 2, wherein, The method further comprises: After the write service is switched to the target database, downloading data generated by the write service of the target database to the source database until the read service is switched from the source database to the target database.

5. The method according to any one of claims 1 to 4, characterized in that, The backup of the data in the source database and the transmission of the backed-up data to the target database comprise: Full backup of initial full data in the source database and transmission of the full backup data to the target database; In the case where it is determined that the initial full data backup is successful, obtaining incremental data generated in the process of backing up the initial full data; Incremental backup of the incremental data and transmission of the incremental backup data to the target database.

6. The method of claim 1, wherein, The method further comprises: Uploading the data in the source database to a cloud object storage service; the cloud object storage service is independently set with the database; The transmission of the backed-up data to the target database comprises: Downloading the backed-up data from the object storage service to the target database.

7. A database migration apparatus characterized by comprising: The apparatus comprises: A data migration module is configured to back up data in a source database and transmit the backed-up data to a target database when it is detected that migration from the source database to the target database is required; the data includes initial full data and incremental data generated in the process of backing up the initial full data; and the source database is in a business read-write state. A service migration module is configured to perform read-write service verification on the target database when it is determined that the backed-up data is successfully transmitted to the target database, and migrate the read-write service to the target database when the verification is passed. 8.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-7. The processor, when executing the computer program, implements the steps of the method of any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 6.

Images