Database migration method, device, equipment and computer readable storage medium

Abstract

The application provides a database migration method, a database migration device, a database migration equipment and a computer readable storage medium; relates to network and routing technology of cloud technology; the method comprises the following steps: migrating data in a source database of an external network to a target database of an internal network, and keeping the source database and the target database synchronized; in the synchronization process, the following operations are performed: redirecting the address of the virtual private network of the target database to the address of the source database, and switching the access request to the address of the virtual private network of the target database, so as to redirect the access request to the source database based on the redirected address of the source database; stop the synchronization of the source database and the target database, and restore the address of the virtual private network of the target database to the address of the target database. Through the application, smooth migration of the database can be realized.

Description

Technical Field

[0001] This application relates to cloud technology and database technology, and in particular to a database migration method, database migration apparatus, database migration device and computer-readable storage medium. Background Technology

[0002] With the development of internet technology, the demand for data processing is constantly increasing. Compared with local data storage, cloud-based data storage can improve the security, stability, and flexibility of data storage. Therefore, more and more enterprises and individuals are choosing to migrate their data to the cloud.

[0003] However, due to the large volume of data access and the involvement of different business operations with complex dependencies between them, it is difficult to achieve a smooth, uninterrupted business transition when migrating data from an on-premises database to a cloud-based database. Summary of the Invention

[0004] This application provides a database migration method, a database migration apparatus, a database migration device, and a computer-readable storage medium, which can achieve smooth database migration.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a database migration method, including:

[0007] Migrate data from the source database on the external network to the destination database on the internal network, and keep the source database and the destination database synchronized;

[0008] The following operations are performed during the synchronization process: redirecting the address of the virtual private network of the destination database to the address of the source database, and

[0009] The access request is switched to the address of the virtual private network of the destination database, so that the access request is redirected to the source database based on the address of the source database to which it is redirected;

[0010] Stop the synchronization between the source database and the destination database, and restore the virtual private network address of the destination database to the address of the destination database.

[0011] This application provides a database migration apparatus, including:

[0012] The synchronization module is used to migrate data from the source database on the external network to the destination database on the internal network, and to keep the source database and the destination database synchronized.

[0013] The redirection module is used to perform the following operations during the synchronization process: redirecting the address of the virtual private network of the destination database to the address of the source database, and switching the access request to the address of the virtual private network of the destination database, so as to redirect the access request to the source database based on the redirected address of the source database;

[0014] The recovery module is used to stop the synchronization between the source database and the destination database, and restore the virtual private network address of the destination database to the address of the destination database.

[0015] In the above scheme, the redirection module is further configured to: modify the routing rules of the virtual private network to change the address of the virtual private network of the destination database from the address pointing to the destination database to the address pointing to the source database.

[0016] In the above scheme, the database migration device further includes a detection module, used to: detect the connectivity of the link from the virtual private network to the source database; when the connectivity is detected to be successful, determine that the redirection is successful; when the connectivity is detected to be unsuccessful, restore the address of the virtual private network of the destination database to the address of the destination database.

[0017] In the above scheme, the redirection module is further configured to: determine the switching order of services corresponding to multiple access requests to the source database; and, according to the switching order, modify the destination address of each access request corresponding to the service from the address of the source database to the address of the virtual private network of the destination database in batches.

[0018] In the above scheme, the redirection module is further configured to: allocate a switching time period for the services corresponding to multiple access requests; and during each switching time period, modify the destination address of the access request for the corresponding service from the address of the source database to the address of the virtual private network of the destination database.

[0019] In the above scheme, the recovery module is further configured to: when it is determined that all access requests to the source database have been successfully switched, set the source database to read-only and set the destination database to read-write.

[0020] In the above scheme, the synchronization module is further configured to: import all data from the source database into the destination database; import the logs of the source database into the relay log of the destination database; wherein the logs of the source database are binary logs generated by the source database during the process of importing the all data into the destination database; and execute events in the relay log to update the data in the destination database.

[0021] This application provides a database migration device, including:

[0022] Memory, used to store executable instructions;

[0023] The processor, when executing executable instructions stored in the memory, implements the database migration method provided in the embodiments of this application.

[0024] This application provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, implement the database migration method provided in this application.

[0025] The embodiments of this application have the following beneficial effects:

[0026] Migrating data from the source database to the destination database while maintaining data synchronization between the two ensures that the source database can provide services without interruption during the migration. Redirecting the virtual private network (VPN) address of the destination database to the address of the source database and switching access requests to the VPN address of the destination database allows access requests to be redirected to the source database based on the redirected address. After all access requests have been successfully switched, synchronization between the source and destination databases can be stopped, and the VPN address of the destination database can be restored to the address of the destination database, enabling access requests to access the destination database through the VPN, thus achieving a smooth and uninterrupted service switchover. Attached Figure Description

[0027] Figure 1 This is a diagram illustrating database migration provided by related technologies;

[0028] Figure 2 This is a schematic diagram of the architecture of the database migration system 200 provided in an embodiment of this application;

[0029] Figure 3 This is a flowchart illustrating the database migration method provided in an embodiment of this application;

[0030] Figure 4 This is a schematic diagram of redirection provided in an embodiment of this application;

[0031] Figure 5 This is a schematic diagram of the route recovery provided in an embodiment of this application;

[0032] Figure 6 This is a schematic diagram of the interaction flow of the database migration method provided in the embodiments of this application;

[0033] Figure 7 This is a time-period diagram illustrating the data migration provided in the embodiments of this application;

[0034] Figure 8This is a schematic diagram of the structure of the database migration device provided in the embodiments of this application. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0037] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0038] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0040] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0041] 1. Data Transmission Service (DTS) supports data transfer between relational databases and various heterogeneous data sources. It provides various data transfer capabilities, including data migration, real-time data subscription, and real-time data synchronization. For example, it can be used to migrate data from on-premises to the cloud.

[0042] 2. A Virtual Private Cloud (VPC) is a cloud-based network space that uses encryption protocols, tunneling protocols, and other security procedures to transmit data between users (such as enterprise users) and cloud service providers. Different VPCs are logically isolated from each other. VPCs can be built on public clouds, private clouds, or hybrid clouds, isolating a portion of cloud resources for specific users, providing them with a collection of resources for their private use. This ensures resource isolation between each user, and users are not affected by other users when using a VPC, experiencing it as if they are using their own private cloud.

[0043] 3. Redirection: redirects the network request to another specified address.

[0044] 4. A database is a repository for organizing, storing, and managing data according to a data structure. It is a collection of files, organized according to a certain data model and stored on various storage media.

[0045] 5. A database instance is a running database program. It is a layer of data management software located between the application and the operating system. All operations performed by the application on the data in the database, including data definition, data query, data maintenance, database operation control, etc., are performed in the database instance. The application can only interact with the database through the database instance. One database can correspond to one or more database instances.

[0046] 6. An intranet, also called a local area network (LAN) or private network, is a computer communication network that connects various hosts (such as various terminals and servers), external devices and databases within a local geographical area (such as a school, factory or government agency), generally within a radius of several kilometers.

[0047] 7. The external network, also called the public network or open internet, is where hosts have public Internet Protocol (IP) addresses. The public network is in contrast to the internal network. Hosts on an internal network use reserved IP addresses from the Internet; while hosts on the public network use public, non-reserved IP addresses from the Internet.

[0048] See Figure 1 , Figure 1 This is a diagram illustrating database migration provided by related technologies. Figure 1In this process, source database 103 is a local database, and data in source database 103 needs to be migrated to destination database 105. Because destination database 105 is located on the intranet, VPC 102 needs to access destination database 105 through the second proxy server 106. After receiving the database migration instruction from terminal 101, the DTS server calls DTS, which then actively pulls data from source database 103 through the first proxy server 104 and imports the pulled data into destination database 105 through the second proxy server 106. After the data migration is completed, the source database 103 and destination database 105 are kept synchronized; that is, writes made by the business end (e.g., terminal 101) to source database 103 will be immediately synchronized to destination database 105. During the synchronization phase between source database 103 and destination database 105, the access requests from the business side to source database 103 are switched to access destination database 105. The switching process is as follows: by setting source database 103 to read-only, all writes to source database 103 are stopped; the synchronization between source database 103 and destination database 105 is stopped via DTS; the access request is switched to the address of VPC 102, and then the access request is sent to destination database 105 through VPC 102 and the second proxy server 106.

[0049] The migrated data includes full data and incremental data. Full data is the data that already existed in the source database 103 before the migration, while incremental data is the new data generated by the source database 103 in response to access requests from the business side during the migration process.

[0050] The data migration process is as follows: DTS accesses the source database 103 through the first proxy server 104, exports all data from the source database 103, and imports the full data into the destination database 105. The starting position of the binary log in the source database 103 is recorded; the binary log records changes to the data in the source database 103. After the full data import is complete, the binary log (in the form of events) is imported into the relay log of the destination database 105 according to the starting position of the binary log. The destination database 105 replays (reads and applies) the contents of the relay log, ensuring that the data in the destination database 105 is consistent with the data in the source database 103.

[0051] It should be noted that the address in the embodiments of this application may include IP address and port. Figure 1 The arrows in the diagram indicate the direction of data flow.

[0052] In implementing this application, the applicant discovered that after stopping the synchronization between the source database 103 and the destination database 105, access requests to the source database 103 should be switched to access the destination database 105 simultaneously to achieve a smooth, uninterrupted switchover. However, in practical applications, many services access the source database 103, especially when the source database is a core database, and the access requests involve multiple business departments, making it impossible to switch all business access requests to the destination database 105 at the same time. Even if different businesses are coordinated to switch to accessing the destination database simultaneously, the recovery time after switching to accessing the destination database varies. Excessive recovery time may adversely affect complex business systems. For example, when the business system is an e-commerce transaction system, a long recovery time will prevent users from using the transaction system, resulting in a poor user experience and potentially causing a decline in transaction volume.

[0053] To at least solve the aforementioned technical problems in related technologies, embodiments of this application provide a database migration method, apparatus, electronic device, and computer-readable storage medium, which can achieve smooth and uninterrupted service switching.

[0054] See Figure 2 , Figure 2 This is a schematic diagram of the architecture of the database migration system 200 provided in this application embodiment. The database migration system 200 includes an electronic device 201, a VPC 102, a source database 103, a first proxy server 104, a destination database 105, and a second proxy server 106. The electronic device 201, VPC 102, and source database 103 are located on the external network, while the destination database 105 and second proxy server 106 are located on the internal network. The first proxy server 104 is located at the boundary between the internal and external networks; therefore, devices on both the internal and external networks can access the first proxy server 104. The source database 103 is located on the source database server, and the destination database 105 is located on the destination database server. Before the database migration, the destination instance in the destination database 105 is an empty instance without any data, and its access permission is read-only, meaning that only reading of the instance is allowed, and writing operations are not permitted.

[0055] DTS is powered by the DTS server ( Figure 2 (Not shown in the image) is running, VPC 102 is operated by the VPC server ( Figure 2(Not shown in the image) The aforementioned servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0056] Electronic device 201 is any device in the business end (i.e., the business party that needs data in the database to perform various Internet services) that needs to access the database to complete relevant business processing logic. It can be implemented as a host or desktop computer, laptop, smartphone, tablet computer and other types of user terminals, or it can be a server of various forms in the business end. Electronic device 201 can migrate data in source database 103 to destination database 105 by calling DTS in DTS server.

[0057] The database migration method provided in this application embodiment can be implemented through the following process: After receiving the database migration instruction from the electronic device 201, the DTS server migrates the data in the source database 103 to the destination database 105 through DTS, and keeps the source database 103 and the destination database 105 synchronized; During the synchronization process, the following operations are performed: The VPC server redirects the address of the VPC 102 of the destination database 105 to the address of the source database 103, and switches the access request from the business end to the address of the VPC 102 of the destination database 105, so as to redirect the access request to the source database 103 based on the redirected address of the source database 103 (that is, the address under the cloud); After the access request is successfully switched, the DTS server stops the synchronization of the source database 103 and the destination database 105, and the VPC server restores the address of the VPC 102 of the destination database 105 to the address of the destination database 105, that is, restores it to the address in VPC 102 used to access the destination database 105.

[0058] Therefore, the business side can conduct business access based on the address corresponding to the destination database 105 in VPC 102. In other words, cloud resources are specifically allocated for the destination database 105 in VPC 102, and the business side can initiate business access to the destination database 105 based on the cloud resources specifically allocated in VPC 102, thereby making full use of the security, stability and flexibility of cloud technology for storing data.

[0059] See Figure 3 , Figure 3 This is a flowchart illustrating the database migration method provided in this application embodiment, which will be combined with... Figure 3 The steps shown illustrate the database migration method provided in the embodiments of this application.

[0060] In step 301, the DTS server migrates data from the source database on the external network to the destination database on the internal network, while maintaining synchronization between the source and destination databases.

[0061] In some embodiments, compared to on-premises source databases, using cloud storage offers advantages such as remote access, reduced hardware costs, enhanced security, and lower file loss rates. Therefore, an increasing number of enterprises and individuals are choosing to migrate their source databases from on-premises to the cloud (destination database). Users can initiate database migration via electronic devices (such as terminals or servers) to migrate data to the cloud. Users can choose to migrate all data or only a portion of it. The electronic device monitors and reports the progress of the data migration in real time. Once the migration is complete, the DTS server notifies the electronic device that the migration is finished. Upon receiving this feedback, the electronic device will notify the user that the data migration is complete.

[0062] The source database can be a self-built database on an external network with a public network address, a self-built database connected to the cloud via a virtual private network or leased line, or a self-built database on a cloud server. For example, the source database can be a MySQL database (relational database management system), a NoSQL database (non-relational database), or similar types. The destination database can be the same as or different from the source database.

[0063] In some embodiments, at least one of the following needs to be verified before database migration: compatibility, network access permissions, source database configuration, source database inspection permissions, and destination database inspection permissions. For example, if verification determines that the source and destination databases are not compatible, another destination database that matches the source database compatibility can be selected for database migration, or the parameters of the source / destination databases can be modified to meet the compatibility requirements.

[0064] In one possible example, the DTS server can migrate data from a source database on the external network to a destination database on the internal network in the following way: The DTS server imports all data from the source database into the destination database; imports the logs from the source database into the relay logs of the destination database; wherein, the logs from the source database are binary logs generated by the source database during the process of importing all data into the destination database; and executes events in the relay logs to update the data in the destination database.

[0065] Because importing all data from the source database into the destination database takes a long time, and the source database remains readable and writable throughout the migration process, data continues to be written to the source database during the import. Therefore, newly generated incremental data needs to be synchronized to the destination database to maintain consistency between the two databases. The source database's binary log records its changes. The binary log of the source database can be read via the DTS server, and the starting position of the binary log can be used to determine where to begin importing the binary log from the destination database's relay log. The binary log is then imported into the relay log as events, and the events in the relay log are executed, thus ensuring data consistency between the source and destination databases. This allows for online hot migration of data without service interruption.

[0066] In some embodiments, after migrating data from the source database on the external network to the destination database on the internal network, the data consistency between the source and destination databases is checked using a full-scale inspection or a sampling inspection method. Once data consistency is confirmed, the source and destination databases are synchronized using Data Transmission System (DTS).

[0067] It should be noted that the database (including the source database and the target database) in this application embodiment can specifically be a database instance, including a source database instance and a target database instance. During the database migration process, the target database instance is an empty instance with read-only access permissions, and write operations cannot be performed on it.

[0068] In some embodiments, during the synchronization process, operations including steps 302 and 303 are performed.

[0069] In step 302, the VPC server redirects the address of the virtual private network of the destination database to the address of the source database.

[0070] In one possible example, the VPC server redirects the address of the destination database's Virtual Private Network (VPN) to the address of the source database. This can be achieved by modifying the VPN routing rules to redirect the VPN address from the address pointing to the destination database (e.g., the VPN routing rules for the VPN to the address pointing to the source database). Figure 2 The address used to access the destination database 105 via VPC 102 (i.e., the address within the VPC 102 network segment) is modified to point to the address of the source database (e.g., ...). Figure 2 (External network address of Zhongyuan Database 103).

[0071] See Figure 4 , Figure 4This is a schematic diagram of redirection provided in an embodiment of this application. In the original routing rules of the VPC, the VPC points to the address of the destination database, meaning the VPC server forwards access requests arriving at the VPC to the destination database on the internal network. However, the VPC server needs to go through at least one proxy server to forward the access request to the destination database. Figure 4 The example uses a second proxy server to represent at least one intermediate proxy server. After the VPC server modifies the routing rules, the VPC no longer points to the address of the destination database, but to the address of the source database. Therefore, access requests arriving at the VPC will be routed by the VPC server and at least one proxy server (in... Figure 4 (Example: The first proxy server represents at least one proxy server) forwards the data to the source database.

[0072] In some embodiments, the address of the destination database can be the address of a second proxy server on the intranet to which the destination database belongs, and the second proxy server can forward access requests arriving at the second proxy server on the intranet to which the destination database belongs to the destination database to the destination database; the address of the source database can be the address of a first proxy server on the external network to which the source database belongs, and the first proxy server can forward access requests arriving at the proxy server on the external network to which the source database belongs to the source database to the source database.

[0073] In some embodiments, after the VPC server redirects the address of the virtual private network of the destination database to the address of the source database, the VPC server checks the connectivity of the link from the virtual private network to the source database; if the connectivity check passes, the redirection is considered successful; if the connectivity check fails, the address of the virtual private network of the destination database is restored to the address of the destination database.

[0074] exist Figure 4In this process, it is necessary to check the network connectivity between the VPC -> the first proxy server -> the source database. In some possible examples, connectivity can be checked using a packet explorer (PING). That is, PING is used to determine whether the VPC can exchange (send and receive) data packets with the source database, and the returned information is used to determine whether the network between the two is connected. If connectivity is confirmed, the redirection is considered successful, and a redirection success message is sent to the electronic device. If connectivity is confirmed to be unavailable, the redirection may have failed, and the operation is rolled back to the state before step 302. That is, the destination address in the routing rules of the virtual private network is restored from the external address of the source database to the address of the virtual private network of the destination database (i.e., the address belonging to the virtual private network and used to access the destination database), thus forming a mapping from the address of the virtual private network to the address of the destination database. At the same time, a connectivity failure message is sent to the electronic device. After receiving the connectivity failure message, the electronic device prompts the user to initiate the redirection again. If the electronic device receives the user's redirection instruction, it will execute step 302 again through the VPC server.

[0075] As can be seen, in step 302, even if the redirection fails, the system can still restore the system to its state before the redirection by rolling back the operation, without affecting the access request, the source database, or the destination database.

[0076] In step 303, the VPC server switches the access request to the address of the virtual private network of the destination database, so as to redirect the access request to the source database based on the address of the redirected source database.

[0077] In some embodiments, after modifying the VPC routing rules, if an access request destined for the source database is detected, the access request is redirected to the VPC server. The VPC server then forwards the access request to the source database based on the modified routing rules. This allows for switching the access path of an access request without affecting access to the source database.

[0078] In one possible example, the VPC server can switch access requests to the address of the virtual private network of the destination database in the following way: the VPC server determines the switching order of multiple access requests to the source database; according to the switching order, the destination address of the multiple access requests is changed from the address of the source database to the address of the virtual private network in batches.

[0079] In one possible example, the order of service switching can be determined based on the dependencies between services accessing the source database, thus determining the switching order of access requests under different services. For instance, if service A depends on service B, and service C depends on service D, then the access requests corresponding to services A and C can be switched first, followed by the access requests corresponding to services B and D. This ensures that switching access requests for some services does not affect other services.

[0080] In another possible example, the order of service switching can be determined based on the recovery time required after a service switch, thereby determining the switching order of access requests under different services. For example, if the recovery time of service E is longer than that of service F, then the switching of access requests corresponding to service E should precede the switching of access requests corresponding to service F. Because in this embodiment, the routing rules of the VPC for the destination database are restored only after all access requests under the services accessing the source database have been switched, causing the VPC to redirect back to the destination database, prioritizing the switching of access requests under services with longer recovery times can reduce the overall access request switching time.

[0081] In one possible example, the VPC server can switch access requests to the address of the virtual private network of the destination database in the following way: The VPC server allocates a switching time period for the services corresponding to multiple access requests; during each switching time period, the destination address of the access request for the corresponding service is changed from the address of the source database to the address of the virtual private network of the destination database.

[0082] For example, if the peak usage times for services H and I are 9:00-11:30 AM, and for service J it's 7:00-9:00 PM, then we can choose to switch access requests for services H and I between 9:00-11:30 AM, and switch access requests for service J between 7:00-9:00 PM. Because the number of access requests is highest during peak periods, switching access requests during these periods allows for a smoother switchover of most services in the shortest possible time, reducing the overall access request switching time.

[0083] It should be noted that after migrating the data from the source database to the destination database, since the two are always kept in sync, it is only required that steps 302 and 303 be executed during the synchronization period. It is not required that step 302 be executed immediately after the migration is completed, nor is it required that step 303 be executed immediately after step 302 is completed.

[0084] In step 304, the DTS server stops synchronizing the source database and the destination database.

[0085] In some embodiments, when it is determined that all access requests for the source database have been successfully switched, the global parameters of the source database are changed from read-write to read-only via the source database server, and the global parameters of the destination database are changed from read-only to read-write via the destination database server. Then, the synchronization between the source and destination databases is stopped via the DTS server. Setting the source database to read-only ensures that there are no writes to the source database, and that the data in the source and destination databases is consistent before and after the access request switch.

[0086] In some possible examples, the destination database can be set to read-write after the synchronization between the source and destination databases is stopped, and before the address of the destination database's virtual private network is restored to the address of the destination database.

[0087] In some possible examples, because an account with super privileges can still access the source database without being affected even if the source database is set to read-only, the source database may need to revoke the account's super privileges. For example, the super privileges can be revoked using the REVOKE statement.

[0088] In step 305, the VPC server restores the address of the virtual private network of the destination database to the address of the destination database.

[0089] like Figure 5 As shown, Figure 5 This is a schematic diagram of the route recovery provided in the embodiments of this application. Figure 5 In this process, after stopping the synchronization between the source database and the destination database, the address of the virtual private network of the destination database is restored to the address of the destination database. In this way, after the access request arrives at the VPC, it will reach the destination database through the link of VPC --> second proxy server --> destination database.

[0090] In some embodiments, since access requests can continue to access the source database during redirection and access request switching, the VPC routing rules can be restored after all access requests have been switched to the VPC address, allowing access requests to be forwarded to the destination database through the VPC server. This achieves a smooth switching of access requests without impacting business operations.

[0091] In some embodiments, after the VPC server restores the address of the virtual private network of the destination database to the address of the destination database, the VPC server performs an access test on the source database and determines that the link from the virtual private network to the source database is invalid based on the test results; it then performs an access test on the destination database and determines that the link from the virtual private network to the destination database is valid based on the test results.

[0092] If it is determined that the link from the virtual private network to the source database is not invalid, settings can be configured on the first proxy server and / or the source database to ensure that the link is not invalid. For example, the port of the source database can be modified / disabled, the source database server where the source database resides can be hidden, or a firewall can be configured.

[0093] If it is determined that the link from the virtual private network to the destination database is not working, check the possible causes one by one (such as firewall settings, network connectivity, etc.) and select the corresponding solution strategy according to the causes found.

[0094] It should be noted that, Figure 2 , Figure 4 and Figure 5 The arrows in the diagram indicate the direction of data flow.

[0095] As can be seen, this embodiment of the application migrates data from the source database to the destination database while maintaining data synchronization between the two, ensuring that the source database can provide services without interruption during the migration. By redirecting the address of the virtual private network (VPN) of the destination database to the address of the source database and switching access requests to the address of the VPN of the destination database, access requests can be redirected to the source database based on the redirected address of the source database. After all access requests have been successfully switched, the synchronization between the source and destination databases can be stopped, and the address of the VPN of the destination database can be restored to the address of the destination database, allowing access requests to access the destination database through the VPN, thereby achieving a smooth and uninterrupted service switchover.

[0096] join Figure 6 , Figure 6 This is a schematic diagram of the interaction flow of the database migration method provided in the embodiments of this application. The following will combine... Figure 6 The steps shown illustrate the process by which electronic devices, DTS servers, and VPC servers collaboratively implement the database migration method provided in the embodiments of this application.

[0097] Step 601: The electronic device sends a database migration command to the DTS server.

[0098] Step 602: The DTS server calls DTS to migrate data from the source database on the external network to the destination database on the internal network, while maintaining synchronization between the source and destination databases.

[0099] During the synchronization process, perform steps 603-610.

[0100] Step 603: The electronic device sends a redirection command to the VPC server.

[0101] Step 604: The VPC server redirects the VPC address of the destination database to the address of the source database.

[0102] Step 605: The VPC server checks the connectivity of the link from the VPC to the source database; when the connectivity check passes, the VPC server determines that the redirection was successful.

[0103] Step 606: The VPC server sends a redirection success message to the electronic device.

[0104] Step 607: The electronic device sends an access request switching command to the VPC server.

[0105] Step 608: The VPC server switches the access request to the address of the VPC of the destination database, and redirects the access request to the source database based on the address of the source database.

[0106] Step 609: When the VPC server determines that all access requests have been successfully switched, it sends an access request switch success message to the electronic device.

[0107] Step 610: The electronic device sends a stop synchronization command to the DTS server.

[0108] Step 611: The DTS server stops synchronizing the source database and the destination database.

[0109] Step 612: After confirming that synchronization has stopped, the electronic device sends a recovery routing command to the VPC server.

[0110] Step 613: The VPC server restores the VPC address of the destination database to the address of the destination database.

[0111] Step 614: The VPC server receives the access request and forwards it to the destination database.

[0112] It should be noted that the above steps have been described in detail above, and will not be repeated here.

[0113] The following will describe specific examples of the database migration method provided in the embodiments of this application. The following will be combined with... Figure 4 and Figure 5 The data migration process provided in the embodiments of this application will be described.

[0114] The database migration process is as follows:

[0115] 1) Create a migration task based on the source and destination databases. The source database addresses are (SrcIp, SrcPort), and the destination database addresses are (DstVip, DstVport). The first proxy server is located at the boundary between the internal and external networks; therefore, it is accessible from both. The second proxy server and the destination database are both located within the internal network. Access to the source database is through the first proxy server, and access to the destination database is through the second proxy server.

[0116] 2) Migrate data from the source database to the destination database, and maintain synchronization between the source and destination databases. See also Figure 7 , Figure 7 This is a time-sharing diagram illustrating the data migration process provided in this application embodiment. In the first time period T1, all data from the source database is imported into the destination database via DTS. In the second time period T2, incremental data generated in the source database during T1 is imported into the destination database. Subsequently, during the synchronization period, i.e., the third time period T3, synchronization between the source and destination databases is maintained until synchronization stops.

[0117] 3) During synchronization, the destination address in the VPC routing rules is modified from the destination database address (DstVip, DstVport) to the source database address (SrcIp, SrcPort). In other words, the VPC address of the destination database is redirected to the address of the source database. Redirection is an atomic operation (an atomic operation is one that cannot be interrupted by the thread scheduling mechanism; once started, it runs continuously without switching to another thread). If the redirection succeeds, connectivity checks ensure that the redirected link is accessible, i.e., the network from VPC to the first proxy server to the source database is connected. If the redirection fails, all operations performed in between are rolled back, restoring the system to its state before the redirection.

[0118] 4) Switch all access requests to VPC, that is, change the destination address of access requests with destination addresses (SrcIp, SrcPort) to (DstVip, DstVport). In this way, after the access request arrives at VPC, it will be forwarded to the source database through the access link: VPC --> first proxy server --> source database.

[0119] 5) After confirming that all service access requests have been switched to VPC, set the source database to read-only. Setting it to read-only ensures that there are no writes to the source database, and that the data in the source and destination databases is consistent before and after the access request switch.

[0120] 6) Stop synchronizing the source and destination databases. Simultaneously, restore the destination address in the VPC's routing rules to the destination database's address (DstVip, DstVport) (i.e., restore the VPC's routing). At this point, all access requests will be sent to the VPC, which will forward the requests to the destination database via the access link: VPC --> second proxy server --> destination database.

[0121] In this step, it is necessary to ensure that the access link from VPC to the first proxy server to the source database is invalid, while ensuring that the access link from VPC to the second proxy server to the destination database is accessible. If this step fails, rollback is not possible.

[0122] As can be seen, the embodiments of this application have upgraded and modified the database migration process in the related technology, adding redirection and route recovery operations to the original process; the access request is switched before the source database and the destination database stop synchronizing, realizing the user business, especially the complex business, needs for time-sharing and batch switching; the whole method has no redundant operations, and does not introduce or rely on additional resources; it can switch access requests in seconds and the access link takes effect immediately, with almost no impact on the business, so that the impact of database migration to the cloud on services (especially complex businesses) is minimized.

[0123] The following describes an exemplary structure of the database migration device provided in the embodiments of this application. Taking the database migration device as a server as an example, based on the understanding above, the data migration device can be a server of the type such as the DTS server or VPC server mentioned above, which will be described in detail below.

[0124] See Figure 8 , Figure 8 This is a schematic diagram of the structure of the database migration device provided in the embodiments of this application. Figure 8 The database migration device 800 shown includes at least one processor 810, a memory 840, and at least one network interface 820. The various components in the database migration device 800 are coupled together via a bus system 830. It is understood that the bus system 830 is used to implement communication between these components. In addition to a data bus, the bus system 830 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 8 The general labeled all buses as Bus System 830.

[0125] The processor 810 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor.

[0126] The memory 840 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 840 may optionally include one or more storage devices physically located away from the processor 810.

[0127] The memory 840 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 840 described in this application embodiment is intended to include any suitable type of memory.

[0128] In some embodiments, memory 840 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0129] Operating system 841 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, for implementing various basic business functions and handling hardware-based tasks.

[0130] The network communication module 842 is used to reach other computing devices via one or more (wired or wireless) network interfaces 820, exemplary network interfaces 820 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0131] In some embodiments, the database migration apparatus provided in this application can be implemented in software. Figure 8 A database migration apparatus 843 stored in memory 840 is shown. This apparatus can be software in the form of programs and plug-ins, and includes the following software modules: a synchronization module 8431, a redirection module 8432, and a recovery module 8433. In some embodiments, the database migration apparatus 843 further includes a detection module 8434. These modules are logically linked and can therefore be arbitrarily combined or further separated depending on the functions they implement. The functions of each module are described below.

[0132] The synchronization module 8431 is used to migrate data from the source database on the external network to the destination database on the internal network, and to keep the source database and the destination database synchronized.

[0133] The redirection module 8432 is used to perform the following operations during the synchronization process: redirecting the address of the virtual private network of the destination database to the address of the source database, and switching the access request to the address of the virtual private network of the destination database, so as to redirect the access request to the source database based on the redirected address of the source database.

[0134] Recovery module 8433 is used to stop the synchronization between the source database and the destination database, and restore the virtual private network address of the destination database to the address of the destination database.

[0135] In some embodiments, the redirection module 8432 is further configured to: modify the routing rules of the virtual private network to change the address of the virtual private network of the destination database from the address pointing to the destination database to the address pointing to the source database.

[0136] In some embodiments, the database migration apparatus 843 further includes a detection module 8434, configured to: detect the connectivity of the link from the virtual private network to the source database; determine that the redirection is successful when the connectivity detection passes; and restore the address of the virtual private network of the destination database to the address of the destination database when the connectivity detection fails.

[0137] In some embodiments, the redirection module 8432 is further configured to: determine the switching order of services corresponding to multiple access requests to the source database; and, according to the switching order, modify the destination address of the access request corresponding to each service from the address of the source database to the address of the virtual private network of the destination database in batches.

[0138] In some embodiments, the redirection module 8432 is further configured to: allocate a switching time period for the services corresponding to multiple access requests; and during each switching time period, modify the destination address of the access request for the corresponding service from the address of the source database to the address of the virtual private network of the destination database.

[0139] In some embodiments, the recovery module 8433 is further configured to: when it is determined that all access requests to the source database have been successfully switched, set the source database to read-only and the destination database to read-write.

[0140] In some embodiments, the synchronization module 8431 is further configured to: import full data from the source database into the destination database; import the logs of the source database into the relay log of the destination database; wherein the logs of the source database are binary logs generated by the source database during the process of importing full data into the destination database; and execute events in the relay log to update the data in the destination database.

[0141] It should be noted that the database migration device 800 can implement all of the above modules, and some modules can be implemented selectively according to the actual deployment situation to realize the role of DTS server or VPC server.

[0142] For example, when the database migration device 800 implements the synchronization module 8431 and the recovery module 8433, it acts as a DTS server; when the database migration device 800 implements the redirection module 8432, the recovery module 8433, and the detection module 8434, it acts as a VPC server.

[0143] Of course, the server can also implement only the functional modules for implementing the DTS server or only the functional modules for implementing the VPC server, depending on the specific application scenario.

[0144] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the database migration method described above in this application.

[0145] This application provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored and, when executed by a processor, will cause the processor to execute the database migration method provided in this application.

[0146] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0147] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0148] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0149] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0150] In summary, this application embodiment migrates data from the source database to the destination database while maintaining data synchronization between the two, ensuring that the source database can provide services without interruption during the migration. By redirecting the virtual private network (VPN) address of the destination database to the address of the source database and switching access requests to the VPN address of the destination database, access requests are redirected to the source database based on the redirected address. After all access requests have been successfully switched, synchronization between the source and destination databases can be stopped, and the VPN address of the destination database can be restored to the address of the destination database, allowing access requests to access the destination database through the VPN, thus achieving a smooth, uninterrupted service switchover.

[0151] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A database migration method, characterized in that, The method includes: Import all data from the source database on the external network into the destination database on the internal network; import the logs of the source database into the relay log of the destination database; execute the events in the relay log to update the data in the destination database, and maintain synchronization between the source database and the destination database after the full data import is completed; The following operations are performed during the synchronization process: redirecting the address of the virtual private network of the destination database to the address of the source database, and The access request is switched to the address of the virtual private network of the destination database, so that the access request is redirected to the source database based on the address of the source database to which it is redirected; Once it is confirmed that all access requests to the source database have been successfully switched, the synchronization between the source database and the destination database is stopped, and the address of the virtual private network of the destination database is restored to the address of the destination database.

2. The method according to claim 1, characterized in that, The step of redirecting the address of the virtual private network of the destination database to the address of the source database includes: Modify the routing rules of the virtual private network to change the address of the virtual private network of the destination database from an address pointing to the destination database to an address pointing to the source database.

3. The method according to claim 1, characterized in that, After redirecting the address of the virtual private network of the destination database to the address of the source database, the method further includes: Detect the connectivity of the link from the virtual private network to the source database; When the connectivity is detected to be successful, the redirection is determined to be successful; When the connectivity failure is detected, the address of the virtual private network of the destination database is restored to the address of the destination database.

4. The method according to claim 1, characterized in that, The step of switching the access request to the address of the virtual private network of the destination database includes: Determine the switching order of services corresponding to multiple access requests to the source database; According to the switching order, the destination address of the access request corresponding to each service is changed from the address of the source database to the address of the virtual private network of the destination database in batches.

5. The method according to claim 1, characterized in that, The step of switching the access request to the address of the virtual private network of the destination database includes: Assign switching time periods to the services corresponding to multiple access requests; During each of the aforementioned switching time periods, the destination address of the access request for the corresponding service is changed from the address of the source database to the address of the virtual private network of the destination database.

6. The method according to claim 1, characterized in that, Before stopping the synchronization between the source database and the destination database, the method further includes: Once it is confirmed that all access requests to the source database have been successfully switched, the source database is set to read-only, and the destination database is set to read-write.

7. The method according to any one of claims 1 to 6, characterized in that, The source database log is a binary log generated by the source database during the process of importing the full data into the destination database.

8. A database migration device, characterized in that, include: The synchronization module is used to import all data from the source database on the external network into the destination database on the internal network. Import the logs of the source database into the relay log of the destination database; execute the events in the relay log to update the data in the destination database, and maintain synchronization between the source database and the destination database after the full data import is completed; The redirection module is used to perform the following operations during the synchronization process: redirecting the address of the virtual private network of the destination database to the address of the source database, and switching the access request to the address of the virtual private network of the destination database, so as to redirect the access request to the source database based on the redirected address of the source database; The recovery module is used to stop the synchronization between the source database and the destination database when it is determined that all access requests to the source database have been successfully switched, and to restore the virtual private network address of the destination database to the address of the destination database.

9. The apparatus according to claim 8, characterized in that, The redirection module is further configured to modify the routing rules of the virtual private network to change the address of the virtual private network of the destination database from an address pointing to the destination database to an address pointing to the source database.

10. The apparatus according to claim 8, characterized in that, The device further includes: The detection module is used to detect the connectivity of the link from the virtual private network to the source database; when the connectivity is detected to be successful, the redirection is determined to be successful; when the connectivity is detected to be unsuccessful, the address of the virtual private network of the destination database is restored to the address of the destination database.

11. The apparatus according to claim 8, characterized in that, The redirection module is further configured to determine the switching order of services corresponding to multiple access requests to the source database; and according to the switching order, modify the destination address of each access request corresponding to the service from the address of the source database to the address of the virtual private network of the destination database in batches.

12. The apparatus according to claim 8, characterized in that, The redirection module is also used to allocate a switching time period for the services corresponding to multiple access requests; during each switching time period, the destination address of the access request for the corresponding service is changed from the address of the source database to the address of the virtual private network of the destination database.

13. The apparatus according to claim 8, characterized in that, The recovery module is further configured to, when it is determined that all access requests to the source database have been successfully switched, set the source database to read-only and the destination database to read-write.

14. The apparatus according to any one of claims 8-13, characterized in that, The source database log is a binary log generated by the source database during the process of importing the full data into the destination database.

15. A database migration device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the database migration method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that, It stores executable instructions for causing a processor to perform the database migration method as described in any one of claims 1 to 7.

17. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the database migration method according to any one of claims 1 to 7.

Images