Database migration control method and device, equipment and medium

Abstract

The application relates to a database migration control method and device, equipment and a medium, and the method comprises the following steps: creating a load balancing agent, setting an access mode of e-commerce platform application access traffic to be directed to a source database via the load balancing agent; in response to a migration starting event, starting unidirectional incremental data synchronization after unidirectional full data synchronization of data in the source database to a target database through a data transmission service; in response to an elastic shrinkage instruction, switching the data transmission service from unidirectional incremental data synchronization to bidirectional incremental data synchronization; adding a back-end address of the target database to the load balancing agent, and removing a back-end address of the source database after verifying that the target database is available; adjusting the access mode of the application access traffic from being directed to the target database via the load balancing agent to being directly directed to the target database, and stopping the bidirectional incremental data synchronization.

Description

Technical Field

[0001] This application relates to the field of big data technology, and in particular to a database migration control method, apparatus, device, and medium thereof. Background Technology

[0003] With the rapid development of e-commerce platforms, large-scale promotional events have become commonplace. During these peak traffic periods, e-commerce platforms typically deploy high-performance, high-configuration database instances to handle peak access traffic, ensuring system stability and user experience. However, after the promotional event ends, business traffic quickly returns to normal levels. Continuing to use high-performance database instances at this time leads to a significant waste of computing and storage resources, increasing operating costs. Therefore, migrating databases from high-performance instances to low-cost instances during off-peak traffic periods, achieving elastic scaling, has become an important means for e-commerce platforms to reduce costs and increase efficiency.

[0004] Traditional database migration methods typically employ a combination of full data migration and incremental data synchronization. This involves copying data from the source database to the target database and then switching application traffic from the source to the target database during off-peak hours by modifying application configurations or DNS resolution. These methods require business interruption or a long switchover window, making them unsuitable for e-commerce platforms' zero-downtime migration needs. Furthermore, after the switchover is complete, traditional methods no longer maintain data synchronization between the source and target databases. If the target database experiences performance bottlenecks or data anomalies, a rapid switchback to the source database is not possible, posing significant business risks.

[0005] Another common migration solution involves using load balancing devices to gradually redirect application traffic from the source database to the target database, achieving a canary switchover. While this approach can reduce the impact of the switchover on business operations to some extent, the switchover process typically requires a monitoring period of several days or even weeks, and continuous monitoring of business metrics is necessary during the switchover, resulting in high operational complexity. For e-commerce platforms, database downsizing needs to be completed as soon as possible after promotional activities to save costs; a prolonged canary switchover does not meet the timeliness requirements of elastic scaling.

[0006] Therefore, how to quickly and seamlessly migrate high-performance databases that handle peak traffic to low-cost databases during off-peak hours on e-commerce platforms, maintain the ability to switch back after migration, and simplify the operation and maintenance process has become a pressing technical problem to be solved in this field. Summary of the Invention

[0007] The purpose of this application is to solve at least one of the above-mentioned problems by providing a database migration control method and corresponding apparatus, devices, non-volatile readable storage media, and computer program products.

[0008] According to one aspect of this application, a database migration control method is provided, comprising: Create a load balancing proxy and set the access method of the e-commerce platform application access traffic to be directed to the source database via the load balancing proxy. The source database is used to carry the peak access traffic during peak hours of the e-commerce platform traffic. In response to the migration start event triggered when the e-commerce platform enters a non-peak traffic period, the data in the source database is fully synchronized one-way to the target database through the data transmission service, and then one-way incremental data synchronization is started. The target database is used to carry the normal access traffic of the e-commerce platform during non-peak traffic periods. In response to the elastic contraction command, the data transmission service is switched from one-way incremental data synchronization to two-way incremental data synchronization, enabling the source database and the target database to synchronize incremental data with each other in real time. Add the backend address of the target database to the load balancing agent, and remove the backend address of the source database after verifying that the target database is available, so as to replace the backend of the load balancing agent from the source database to the target database; Change the access method of the application access traffic from passing through the load balancer to directly pointing to the target database, and stop the bidirectional incremental data synchronization.

[0009] According to another aspect of this application, a database migration control device is provided, comprising: The proxy configuration module is set to create a load balancing proxy, and the access method of the e-commerce platform application access traffic is set to be directed to the source database through the load balancing proxy. The source database is used to carry the peak access traffic during the peak period of e-commerce platform traffic. The migration startup module is configured to respond to a migration startup event triggered when the e-commerce platform enters a non-peak traffic period. After unidirectionally synchronizing the data in the source database to the target database through the data transmission service, it starts unidirectional incremental data synchronization. The target database is used to carry the normal access traffic of the e-commerce platform during non-peak traffic periods. The mode switching module is configured to switch the data transmission service from one-way incremental data synchronization to two-way incremental data synchronization in response to the elastic contraction command, so that the source database and the target database can synchronize incremental data with each other in real time. The address replacement module is configured to add the backend address of the target database to the load balancing agent, and remove the backend address of the source database after verifying that the target database is available, so as to replace the backend of the load balancing agent from the source database to the target database. The traffic switching module is configured to change the access method of the application access traffic from passing through the load balancing proxy to directly pointing to the target database, and stop the bidirectional incremental data synchronization.

[0010] According to another aspect of this application, an electronic device is provided, including a central processing unit and a memory, wherein the central processing unit is configured to invoke and run a computer program stored in the memory to perform the steps of the method described in this application.

[0011] According to another aspect of this application, a non-volatile readable storage medium is provided, which stores a computer program implemented according to the database migration control method in the form of computer-readable instructions, wherein the computer program, when invoked by a computer, executes the steps included in the method.

[0012] According to another aspect of this application, a computer program product is provided, comprising a computer program / instructions that, when executed by a processor, implement the steps of the method.

[0013] Compared to traditional technologies, this application achieves seamless application migration by creating a load-balanced proxy and directing application access traffic to the source database through this proxy. During off-peak traffic periods, full synchronization and one-way incremental synchronization are automatically triggered, completing the basic data migration during low-peak business hours and avoiding impact on peak-hour operations. By responding to elastic contraction commands, one-way incremental synchronization is switched to two-way incremental synchronization, enabling real-time synchronization of incremental data between the source and target databases. This ensures a switchback capability during subsequent backend replacements, significantly reducing migration risks. The load-balanced proxy adds the target backend address in one step, verifies its availability, and removes the source backend address, achieving second-level backend replacement without the days-long observation period of traditional canary rollouts. Finally, application access traffic is redirected directly to the target database, and two-way synchronization is stopped, eliminating the additional network jumps and single-point-of-failure risks associated with the proxy layer and allowing for the recovery of source database resources. The entire process is highly automated, enabling lossless elastic contraction of the e-commerce platform from peak to off-peak traffic periods, effectively reducing operating costs while ensuring business continuity. Attached Figure Description

[0014] Figure 1 This is an exemplary network architecture for this application; Figure 2 This is a flowchart illustrating one embodiment of the database migration control method of this application; Figure 3 This is a schematic block diagram of the database migration control device of this application; Figure 4 This is a schematic diagram of the structure of an electronic device used in this application. Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0016] This application's embodiments can be applied to various online business platforms employing distributed architectures, particularly in scenarios where business traffic exhibits significant tidal characteristics, such as e-commerce platforms, online ticketing platforms, social networking platforms, and cloud computing service platforms. In these platforms, the database, as the core infrastructure, carries critical business data such as user data, order information, product inventory, and transaction records. To cope with traffic surges caused by large-scale promotional activities or unexpected events, platforms typically temporarily deploy high-performance, high-configuration database instances to ensure system stability and user experience. However, after the traffic peak ends, business traffic quickly returns to normal levels. Continuing to use high-performance database instances at this time leads to a significant waste of computing and storage resources. Therefore, how to smoothly migrate the database from a high-performance instance to a low-cost instance after the traffic peak ends, achieving elastic scaling, becomes a key technical issue for reducing operating costs. This application's embodiments provide a technical solution for such application scenarios, enabling the rapid and lossless migration of a source database carrying peak access traffic to a target database carrying normal access traffic without interrupting business operations, while maintaining the ability to switch back after the migration.

[0017] Please see Figure 1 , Figure 1 This is a schematic diagram of the architecture of a database migration control system provided in an embodiment of this application. The system may include a client 80, an application server 81, and a proxy server 82. The client 80 may be a terminal device used by maintenance personnel to initiate elastic scaling commands to the migration control system and to receive and display the execution status of the migration process. The application server 81 is a server that hosts e-commerce platform business applications, running one or more application processes. These application processes are responsible for handling user access requests and interacting with the database to read and write business data. The proxy server 82 is a load balancing proxy deployed between the application server and the database, running load balancing software to distribute application access traffic to backend database instances according to a configured list of backend addresses. The client 80, application server 81, and proxy server 82 communicate via a network, which may be a wired or wireless network, such as a local area network (LAN), a wide area network (WAN), or the Internet.

[0018] In this embodiment, the source database refers to a high-performance database instance that carries peak access traffic during peak hours of the e-commerce platform. It is typically configured with a high CPU clock speed, large memory capacity, and storage media with higher I / O performance to meet the concurrent access demands during peak hours. The target database refers to a low-cost database instance that carries normal access traffic during off-peak hours of the e-commerce platform. Its configuration is relatively low, but sufficient to support normal business operations under normal traffic conditions. The data transmission service is a dedicated service used to achieve data synchronization between databases. It supports both full data migration and incremental data synchronization modes and can switch between one-way and two-way synchronization modes. In one-way synchronization mode, data flows only from the source database to the target database; in two-way synchronization mode, the source and target databases can synchronize incremental data with each other in real time, thereby maintaining eventual consistency between the two databases.

[0019] The database migration control method of this application can be implemented as a computer program. This computer program can be installed and run as an independent migration control unit on the proxy server 82, or it can be integrated into the load balancing software of the proxy server 82 as a functional module. The processor of the migration control unit executes the computer program to implement the various steps in the method. During operation, the migration control unit needs to communicate with the application process on the application server 81, the load balancing software on the proxy server 82, and the data transmission service to adjust the access method of application access traffic, update the backend address list of the load balancing proxy, and control the synchronization mode switching of the data transmission service. To support the reliable execution of the migration process, the migration control unit can also include a local storage module specifically for storing migration progress and status information. This local storage module can be an embedded database or file system, used to persistently record information such as the current migration stage, data synchronization status, and verification results. In actual deployment, the migration control unit can run on the same physical server or virtual server as the load balancing software, or it can run on a separate server; this application does not impose any restrictions on this.

[0020] Taking an e-commerce platform as an example, during large-scale promotional events, the platform pre-deploys high-performance database instances to handle surges in user traffic. After the promotion ends, business traffic gradually returns to normal levels. At this point, operations personnel can initiate an elastic scaling command to the migration control system via client 80, starting the migration process from the high-performance source database to the low-cost target database. The migration control unit first confirms whether it is currently a non-peak traffic period. If so, it uses the data transmission service to fully synchronize historical data from the source database to the target database, and then initiates one-way incremental synchronization to catch up on new data generated during the migration. After confirming that the data latency has dropped to a safe range, the data transmission service is switched to bidirectional incremental synchronization mode, enabling the source and target databases to synchronize incremental data with each other in real time. Next, the migration control unit sends a command to the load balancing software on proxy server 82 to add the target database's backend address to the backend address list, and verifies the availability of the target database by sending probe query statements. After successful verification, the source database's backend address is removed, thus seamlessly switching application access traffic to the target database. Finally, the migration control unit changes the application server's access method from via proxy server 82 to directly pointing to the target database and stops bidirectional incremental data synchronization, completing the entire migration process. Throughout the process, the application processes on application server 81 do not need to modify any connection configurations, and user access will not be affected in any way.

[0021] It should be noted that the promotional activity scenarios of the aforementioned e-commerce platforms are merely illustrative examples. In actual applications, different platforms can flexibly set the triggering conditions and migration strategies for traffic peaks according to their own business characteristics, and this application does not impose any restrictions on this. As long as there is a need for elastic scaling from high-performance databases to low-cost databases, the technical solution of this application can be applied.

[0022] The technical solution of this application will be further described in detail below with reference to specific embodiments.

[0023] Please see Figure 2 According to the database migration control method provided in this application, it can be implemented as a computer program product and run on electronic devices such as servers, specifically including the following steps: Step S5100: Create a load balancing proxy and set the access method of e-commerce platform application access traffic to point to the source database via the load balancing proxy. The source database is used to carry the peak access traffic during peak hours of e-commerce platform traffic. By creating a load balancing proxy, a unified entry point for application access traffic can be managed. This load balancing proxy can be deployed on a standalone server, such as the aforementioned proxy server, or as a software module within a gateway server or cloud platform's load balancing service. The load balancing proxy can employ Layer 4 or Layer 7 load balancing protocols, such as load balancing software based on Linux virtual server technology or HTTP protocol, to distribute access requests from the application server to backend database instances according to preset policies. When creating the load balancing proxy, a virtual IP address is assigned to it as the unified entry point for application access, and a mapping relationship is established between this virtual IP address and the backend address of the source database, which can be an internal or external network address. This ensures that all database access requests pointing to this virtual IP address are forwarded to the source database.

[0024] After the load balancer is created, the access method for e-commerce platform application traffic can be set to be routed through the load balancer to the source database. Specifically, in one implementation, the database connection configuration file on the application server can be modified to change the database address in the connection string from the original address of the source database (such as the internal or external address mentioned above) to the virtual IP address of the load balancer. Then, the application process can be restarted or the new connection configuration can be applied via hot reloading. In another implementation, the access method can be adjusted through domain name resolution. The domain name that originally resolved directly to the source database can be re-resolved to the virtual IP address of the load balancer, and the application traffic can be gradually redirected to the load balancer using the domain name resolution caching and expiration mechanism. In yet another implementation, if a gateway or proxy layer already exists between the application server and the database, the target address only needs to be set to the virtual IP address of the load balancer in the routing configuration of the gateway or proxy layer. Regardless of the specific implementation method used, the common effect is to ensure that all application traffic of the e-commerce platform is uniformly forwarded to the source database through the load balancer during the migration process, thus laying the foundation for subsequent backend address replacement.

[0025] The source database is the database instance used to handle peak access traffic during peak periods on e-commerce platforms. During large-scale promotional events on e-commerce platforms, to cope with the surge in user access and transaction requests, the platform typically deploys high-performance database instances, such as those configured with high CPU clock speeds, large memory capacities, and storage media with higher I / O performance, to ensure stable handling of massive concurrent requests during peak traffic periods. While these high-performance database instances are indispensable during peak periods, continuing to use them after the promotional event ends and business traffic returns to normal levels would lead to a waste of computing and storage resources. Therefore, in the technical solution of this application, the source database plays the role of the migration starting point, and the peak access traffic it carries will be gradually transferred to the target database in subsequent steps.

[0026] Step S5200: In response to the migration start event triggered when the e-commerce platform enters a non-peak traffic period, the data in the source database is fully synchronized unidirectionally to the target database through the data transmission service, and then unidirectional incremental data synchronization is started. The target database is used to carry the normal access traffic of the e-commerce platform during non-peak traffic periods. After a major promotional event on an e-commerce platform ends, business traffic gradually returns to normal levels. At this point, it can be automatically or manually determined that the e-commerce platform has entered a non-peak traffic period, and a migration start event can be triggered accordingly. There are several ways to trigger the migration start event. In one embodiment, it can be achieved by continuously monitoring the access traffic value of the source database. When the monitored access traffic value drops below a preset traffic threshold and remains stable for a preset duration, it is determined that the e-commerce platform has entered a non-peak traffic period, and a migration start event is automatically generated. In another embodiment, the migration start event can be manually issued by operations and maintenance personnel according to business arrangements. For example, at a designated time point after the end of the promotional event, operations and maintenance personnel send a migration start command to the migration control system through a client. In yet another embodiment, the migration start event can be triggered based on a scheduled task. A fixed time point after the end of the promotional event is preset as the migration start time, and the migration start event is automatically triggered when that time is reached.

[0027] In response to the migration start event, the data transfer service can unidirectionally and fully synchronize data from the source database to the target database. The data transfer service is a dedicated service for synchronizing data between databases, supporting both full data migration and incremental data synchronization modes. During the full synchronization phase, the data transfer service completely copies all historical data from the source database to the target database, including table structures, indexes, stored procedures, triggers, and all business data records. During full synchronization, the data transfer service can improve synchronization efficiency using batch reads and parallel writes, and its breakpoint resume mechanism ensures that synchronization can continue from the point of interruption without restarting in the event of network interruption or service failure. After full synchronization is complete, the target database has a data copy that is completely identical to the source database at a given point in time.

[0028] After full synchronization is complete, one-way incremental data synchronization is initiated immediately. In one-way incremental data synchronization mode, the data transmission service continuously captures all data change operations occurring on the source database, including data insertion, update, and deletion operations, and synchronizes these changes to the target database in real time. One-way incremental data synchronization is initiated immediately after full synchronization to ensure that no new data generated on the source database during full synchronization is missed. Through one-way incremental data synchronization, the data in the target database can continuously catch up with the real-time state of the source database, gradually reducing the data differences between the two databases.

[0029] In contrast to the source database, the target database is a database instance designed to handle the normal access traffic of the e-commerce platform during off-peak hours. Compared to the high-performance configuration of the source database, the target database typically uses lower-configuration computing and storage resources, such as lower CPU clock speeds, smaller memory capacities, and lower-cost storage media, aiming to meet the normal access needs during off-peak hours. In the elastic scaling scenarios of e-commerce platforms, the selection of the target database can be based on historical data analysis of business traffic during off-peak hours to determine its required configuration specifications, thereby optimizing costs while ensuring normal business operation.

[0030] Step S5300: In response to the elastic contraction command, the data transmission service is switched from one-way incremental data synchronization to two-way incremental data synchronization, so that the source database and the target database can synchronize incremental data with each other in real time. After unidirectional incremental data synchronization has been running stably for a period of time, the data differences between the source and target databases gradually decrease until they become consistent. At this point, an elastic contraction command can be triggered, allowing the data transmission service to be switched from unidirectional incremental data synchronization to bidirectional incremental data synchronization in response to the elastic contraction command.

[0031] The elastic shrinkage command is a control signal that triggers a switching action, and its source can be implemented in various ways. In one embodiment, the elastic shrinkage command can be automatically generated by the migration control unit when a preset condition is detected. For example, the migration control unit continuously monitors the data latency value between the source database and the target database. When the data latency value is less than a preset latency threshold, it indicates that the data in the two databases is basically consistent, and the elastic shrinkage command is automatically triggered at this time. In another embodiment, the elastic shrinkage command can be manually issued by maintenance personnel through a client. After confirming that the data synchronization status is normal, the maintenance personnel initiate the elastic shrinkage command through the operation interface. In yet another embodiment, the elastic shrinkage command can be triggered based on a scheduled task, and the elastic shrinkage command can be automatically generated at a specified time point after the unidirectional incremental data synchronization starts.

[0032] In response to the elastic contraction command, the data transmission service switches from a one-way incremental data synchronization mode to a two-way incremental data synchronization mode. In one-way incremental data synchronization mode, data flows only from the source database to the target database; changes to data on the target database are not synchronized back to the source database. In two-way incremental data synchronization mode, a real-time synchronization channel is established between the source and target databases, ensuring that data changes occurring on either database are synchronized to the other in real time.

[0033] Specifically, the working principle of bidirectional incremental data synchronization is as follows: The data transmission service deploys data capture components on both the source and target databases, continuously monitoring their respective transaction logs or change data capture logs. When a data change event is detected, it is encapsulated as a synchronization message and transmitted over the network to the peer database. Upon receiving the synchronization message, the peer database executes the corresponding data change operations locally. In this way, bidirectional real-time data synchronization is achieved between the two databases, ensuring that the data in the source and target databases always maintains eventual consistency.

[0034] This application's switch from unidirectional incremental data synchronization to bidirectional incremental data synchronization is of significant technical importance. During the subsequent load balancing proxy backend replacement process, application access traffic will switch from the source database to the target database. If the target database experiences performance bottlenecks or data anomalies after the switch, traffic needs to be switched back to the source database. In bidirectional incremental data synchronization mode, any data changes occurring on the target database are synchronized back to the source database in real time. Therefore, the source database possesses complete and up-to-date data during the switchback, preventing data loss. In other words, the timely initiation of bidirectional incremental data synchronization provides a reversible guarantee mechanism for the entire migration process, allowing for safe rollback at any stage and significantly reducing migration risks.

[0035] After the switchover is complete, bidirectional incremental data synchronization continues, preparing for the subsequent load balancing proxy backend replacement. At this point, the source and target databases are in a bidirectional real-time synchronization state, with consistent data between the two databases, laying the data foundation for subsequent traffic switching.

[0036] Step S5400: Add the backend address of the target database to the load balancing agent, and remove the backend address of the source database after verifying that the target database is available, so as to replace the backend of the load balancing agent from the source database to the target database. After bidirectional incremental data synchronization is established, the source and target databases can synchronize incremental data with each other in real time, maintaining eventual consistency. At this point, the backend address replacement operation of the load balancer proxy can be executed, seamlessly switching application access traffic from the source database to the target database.

[0037] During real-time backend address replacement, the target database's backend address is first added to the load balancing agent. The load balancing agent maintains a list of backend addresses, recording the address information of all backend database instances available for traffic distribution. Initially, this list only contains the source database's backend address. To achieve a smooth traffic switch, the target database's backend address is first added to the load balancing agent's backend address list. After the addition operation is complete, the load balancing agent's backend address list contains both the source and target database's backend addresses. At this point, the load balancing agent will distribute application access traffic to these two backend database instances according to its configured distribution strategy. In one embodiment, the load balancing agent can use a weighted round-robin algorithm to set a lower weight for the newly added target database, allowing a small amount of probe or test traffic to be distributed to the target database to verify its access capabilities without affecting mainstream business. In another embodiment, the load balancing agent can set the target database to a standby state, establishing a connection only when traffic is distributed to that address, thereby achieving on-demand verification.

[0038] After adding the backend address of the target database, it is necessary to verify its availability to ensure it can handle application traffic. There are several ways to verify the availability of the target database. In one embodiment, this can be achieved by sending a probe query to the target database. The probe query is a lightweight structured query language statement, such as executing a simple data query or status query. The availability is determined by whether the target database can return the query results normally. In another embodiment, this can be achieved by checking the current number of connections to the target database. If the current number of connections is less than a preset connection threshold, it indicates that the target database has sufficient connection resources to handle new access requests; otherwise, it indicates that the target database may be overloaded. In yet another embodiment, this can be achieved by measuring the current response time of the target database. If the current response time is less than a preset response time threshold, it indicates that the target database can respond to query requests quickly; otherwise, it indicates that the target database may have a performance bottleneck. In practical applications, these verification methods can be combined. For example, both the current number of connections and the current response time can be checked simultaneously. The target database is only considered available when both indicators meet preset conditions, thereby improving the reliability of the verification.

[0039] After verifying the availability of the target database, the source database's backend address is removed from the load balancer's backend address list. Once removed, the load balancer's backend address list contains only the target database's backend address, and all subsequent application traffic will be distributed to the target database. This sequence of operations—adding the target database's backend address, verifying its availability, and then removing the source database's backend address—achieves a smooth replacement of the load balancer's backend. During the replacement process, because the target database's backend address undergoes availability verification after being added, and is only removed after confirmation of availability, application traffic will not experience a window of inactivity due to the target database's unavailability. The entire replacement process is completed within seconds; the application process on the application server does not require any modification to its connection configuration, and users will not experience any interruption or delay.

[0040] If the target database is found to be unavailable during the verification process, the operation to remove the backend address of the source database will not be performed. The load balancer proxy will continue to point to the source database, thereby ensuring that application access traffic continues to be processed through the source database and does not affect the normal operation of the business. In one embodiment, after determining that the target database is unavailable, the availability verification can be re-initiated. If it is still determined to be unavailable after a preset number of retries, the entire migration process will be aborted and an alarm message will be generated to notify the operation and maintenance personnel to intervene.

[0041] Step S5500: Change the access method of the application access traffic from via the load balancing proxy to directly pointing to the target database, and stop the bidirectional incremental data synchronization.

[0042] After the load balancer proxy's backend was changed from the source database to the target database, all application traffic was distributed to the target database via the load balancer proxy. At this point, the load balancer proxy still exists as a traffic relay layer between the application server and the target database. Although the load balancer proxy played a crucial role in traffic scheduling during the migration process, continuing to retain this proxy layer after the migration introduces additional network jumps, increases request latency, and the proxy server itself becomes a potential point of failure. Therefore, it is necessary to change the way application traffic is accessed from via the load balancer proxy to directly pointing to the target database to eliminate the architectural redundancy introduced by the proxy layer.

[0043] There are several ways to adjust the access method of application traffic to directly point to the target database. In one embodiment, this can be achieved by modifying the database connection configuration file on the application server, changing the address in the database connection string from the virtual IP address of the load balancer proxy to the original address of the target database, and then restarting the application process or using hot reloading to make the new connection configuration effective. In another embodiment, this can be achieved through domain name resolution, resolving the domain name that originally resolved to the virtual IP address of the load balancer proxy to the original address of the target database, and gradually guiding application traffic to the target database using the domain name resolution caching and expiration mechanism. In yet another embodiment, if a gateway or proxy layer already exists between the application server and the database, it is only necessary to change the target address from the virtual IP address of the load balancer proxy to the original address of the target database in the routing configuration of the gateway or proxy layer. Regardless of the specific implementation method, the common effect is to enable the application process of the application server to directly connect to the target database, without going through the load balancer proxy for traffic relay.

[0044] After adjusting the application access traffic access method to directly point to the target database, it is necessary to stop bidirectional incremental data synchronization. The specific operation to stop bidirectional incremental data synchronization can be achieved by the migration control unit sending a stop command to the data transmission service. Upon receiving the stop command, the data transmission service shuts down the data capture components on both the source and target databases, disconnects the synchronization channel between the two databases, and clears relevant synchronization status information. The timing for stopping bidirectional incremental data synchronization is chosen after the application access traffic has been successfully switched to the target database and the direct connection architecture has been established. This ensures that before synchronization stops, all data changes on the target database caused by handling application access traffic have been synchronized to the source database through the bidirectional synchronization channel, maintaining eventual consistency between the two databases. After stopping bidirectional incremental data synchronization, the source and target databases no longer synchronize data, and each database operates independently. At this point, the entire database migration process is complete. The e-commerce platform's application access traffic has been smoothly migrated from the source database, which handled peak traffic during peak hours, to the target database, which handles normal traffic during off-peak hours. Furthermore, the application access architecture has been restored from indirect access via a load balancer to a simple architecture that directly accesses the target database.

[0045] After the migration process described above, the e-commerce platform's application access traffic has been smoothly migrated from the source database, which handled peak traffic during peak hours, to the target database, which handles normal traffic during off-peak hours. The source database, having fulfilled its mission during promotional activities, can be taken offline and its occupied computing and storage resources reclaimed, thus achieving the ultimate goal of elastic scaling. The target database stably supports the e-commerce platform's daily operations at a lower operating cost during off-peak hours. Furthermore, because bidirectional incremental data synchronization was established during the migration process and data consistency was maintained after the switch, even if performance or data issues are discovered in the target database some time after the migration, a safe rollback can be achieved by re-establishing bidirectional incremental data synchronization and restoring application access traffic to the source database. The entire migration process is uninterrupted, application-aware, and highly automated, effectively solving the technical challenges faced by e-commerce platforms when elastically scaling from high-performance databases to low-cost databases after large-scale promotional activities.

[0046] This application has undergone in-depth optimization of database migration technology to adapt to large-scale promotional activities on e-commerce platforms, achieving various beneficial effects, including but not limited to: First, this application achieves lossless elastic scaling of an e-commerce platform from peak traffic periods to off-peak periods. By creating a load-balanced proxy and configuring application access traffic to pass through this proxy to the source database, the application does not need to modify its connection configuration throughout the migration process, remaining completely unaware of changes to the backend database. During off-peak periods, full and one-way incremental synchronization are initiated, utilizing the low-traffic period to complete the basic data migration and avoiding impact on peak-hour business. Finally, the access method is changed from passing through the proxy to directly pointing to the target database, and synchronization is stopped, completing a smooth transition from a high-performance source database to a low-cost target database. The entire process is uninterrupted, achieving elastic scaling with zero downtime.

[0047] Secondly, this application significantly improves the security and reversibility of the migration process. By switching to bidirectional incremental synchronization in response to an elastic shrinkage command after unidirectional incremental synchronization is completed, a real-time synchronization capability is established between the source and target databases. This allows traffic to be switched back to the source database at any time during subsequent load balancing proxy backend replacement, even if the target database experiences an anomaly, without data loss. Compared to traditional migration methods that lose their ability to switch back after the switch is completed, this application maintains a switchable state throughout the entire migration process, greatly reducing migration risks.

[0048] Furthermore, this application significantly reduces the switching time and simplifies operation and maintenance. By adding the target backend address in one go through the load balancing agent and removing the source backend address after verification, the backend pointer is replaced within seconds, eliminating the need for the observation period of several days required for traditional canary rollouts. Simultaneously, the migration start event is automatically triggered when the e-commerce platform enters a non-peak traffic period, and the elastic scaling command responds after data synchronization catches up. The entire process is highly automated, reducing manual intervention and operational complexity.

[0049] Furthermore, this application simplifies the architecture and effectively reclaims resources after migration. By redirecting application access traffic from a load-balanced proxy to directly accessing the target database, the additional network jumps and single-point-of-failure risks associated with the proxy layer are eliminated, restoring a simple and efficient direct-connect architecture. After ceasing bidirectional incremental data synchronization, the source database can be taken offline, and its occupied computing and storage resources can be reclaimed, achieving the ultimate goal of elastic scaling—effectively reducing operating costs while ensuring e-commerce business continuity.

[0050] Based on any embodiment of the method in this application, after initiating unidirectional incremental data synchronization, the method includes: Step S5210: Continuously monitor the data latency value between the source database and the target database; After initiating unidirectional incremental data synchronization, the migration control unit continuously monitors the data latency between the source and target databases. Data latency is a quantitative indicator that measures how much data in the target database lags behind the data in the source database; it is typically expressed in units of time, such as seconds or milliseconds.

[0051] In one embodiment, the data latency value can be calculated by comparing the timestamps of the most recent data changes on the source database and the target database. Specifically, when the data transmission service synchronizes the data changes on the source database to the target database, it records the generation timestamp of each change record on the source database and the application completion timestamp of the change record on the target database. The difference between the two is the synchronization latency time of the change record. The maximum latency time among all change records that have not been synchronized is taken as the current data latency value.

[0052] In another embodiment, the data latency value can be directly obtained by querying the monitoring interface provided by the data transmission service. The data transmission service internally maintains the length of the synchronization queue and the consumption progress, and the current data latency value can be calculated using these internal indicators. The migration control unit can perform a data latency value collection and recording operation once according to a preset sampling period, such as every one second or every five seconds.

[0053] Step S5220: When the data delay value is less than a preset delay threshold, trigger the elastic contraction command; After obtaining the data latency value, the migration control unit compares this value with a preset latency threshold. The preset latency threshold is a pre-configured parameter used to define the acceptable latency range for data synchronization; its value can be set according to the e-commerce platform's data consistency requirements. In one embodiment, for core transaction scenarios with high data consistency requirements, the preset latency threshold can be set to a smaller value, such as 100 milliseconds or 500 milliseconds. In another embodiment, for non-core business scenarios with relatively low data consistency requirements, the preset latency threshold can be set to a larger value, such as 5 seconds or 10 seconds.

[0054] When the data latency value is less than the preset latency threshold, it indicates that the data difference between the source database and the target database has been reduced to an acceptable range, and the data in the two databases has basically reached a consistent state. At this point, the migration control unit automatically triggers an elastic shrinkage command to initiate the subsequent synchronization mode switching process. In this way, the triggering timing of the elastic shrinkage command is no longer based on a fixed point in time or manual judgment, but on the actual progress of data synchronization, thereby ensuring that the switching operation is only performed after the data has been truly caught up, improving the security of the switching.

[0055] Step S5230: When the data delay value is greater than or equal to the preset delay threshold, maintain the one-way incremental data synchronization until the data delay value is less than the preset delay threshold.

[0056] When the data latency value is greater than or equal to the preset latency threshold, it indicates that the data difference between the source database and the target database is still significant, and the conditions for switching to synchronization mode are not yet met. In this case, the migration control unit continues to operate unidirectional incremental data synchronization without triggering the elastic shrinkage command, and continues to monitor the data latency value according to the preset sampling period until the data latency value drops below the preset latency threshold. During the waiting process, unidirectional incremental data synchronization continues to run, constantly synchronizing newly generated data changes on the source database to the target database, and the data latency value gradually decreases accordingly. If the data latency value remains higher than the preset latency threshold for a period of time, the migration control unit can generate an alarm message to prompt maintenance personnel to pay attention to whether there are any abnormalities in the data synchronization status, such as insufficient network bandwidth or excessive load on the source database causing the data change generation rate to exceed the synchronization rate.

[0057] Through the above embodiments, after initiating unidirectional incremental data synchronization, this application continuously monitors the data latency value and determines whether to trigger an elastic contraction command based on the comparison result between this value and a preset latency threshold, thus achieving automated and intelligent triggering of the elastic contraction command. Compared to methods that rely on fixed time points or manual judgment to trigger the switch, this embodiment makes decisions based on the actual progress of data synchronization, ensuring that the subsequent synchronization mode switch is only performed after the data in the source database and the target database are indeed synchronized. This improves the automation level and security of the migration process while ensuring data consistency.

[0058] Based on any embodiment of the method in this application, after switching the data transmission service from one-way incremental data synchronization to two-way incremental data synchronization, the method includes: Step S5310: Send a probe query statement to the target database, and determine the cache preheating status of the target database based on the response time of the probe query statement; After switching the data transmission service from one-way incremental data synchronization to two-way incremental data synchronization, the migration control unit sends probe query statements to the target database and determines the cache warm-up status of the target database based on the response time of these probe query statements. The probe query statement is a lightweight structured query language statement, designed for fast execution and low resource consumption. For example, it can be a statement querying the current time or a statement querying a single record. The purpose of the probe query statement is not to obtain specific business data, but to test the responsiveness of the target database.

[0059] When a target database has just completed data synchronization but has not yet received actual business traffic, its internal caches, including data page caches, index caches, and query plan caches, may be in a cold state, meaning no business data has been loaded. In this cold state, even executing a simple query requires the target database to read data from disk and load it into the cache, resulting in a longer response time. By sending probe queries to the target database and measuring its response time, the cache warm-up level of the target database can be effectively assessed.

[0060] In one embodiment, the migration control unit can continuously send multiple identical probe query statements and record the response time for each one. If the subsequent response time is significantly lower than the first response time, it indicates that the target database cache is gradually warming up.

[0061] Step S5320: When the response time is less than the preset warm-up time threshold, it is determined that the target database has completed cache warm-up, and then the step of adding the backend address of the target database to the load balancing agent is executed. After obtaining the response time of the probe query, the migration control unit compares this response time with a preset warm-up time threshold. The preset warm-up time threshold is a pre-configured parameter used to define the standard for completing the target database cache warm-up. Its value can be set according to the target database's configuration specifications and business scenarios. In one embodiment, for a target database with a high configuration, the preset warm-up time threshold can be set to a smaller value, such as 10 milliseconds or 20 milliseconds. In another embodiment, for a target database with a low configuration, the preset warm-up time threshold can be set to a larger value, such as 50 milliseconds or 100 milliseconds. When the response time is less than the preset warm-up time threshold, it indicates that the target database's cache has completed sufficient warm-up and can respond to query requests with low latency. At this point, the migration control unit determines that the target database has completed cache warm-up and then executes the step of adding the target database's backend address to the load balancing proxy. By adding the backend address after the cache warm-up is complete, it ensures that the target database is in an optimal response state when application access traffic is distributed to it, avoiding slow request response issues caused by cache cold start.

[0062] Step S5330: When the response duration is greater than or equal to the preset warm-up duration threshold, continue to send the probe query statement to the target database until the response duration is less than the preset warm-up duration threshold.

[0063] When the response time is greater than or equal to a preset warm-up time threshold, it indicates that the target database cache has not been sufficiently warmed up. At this point, the migration control unit continues to send probe query statements to the target database and continuously monitors the trend of response time changes until the response time is less than the preset warm-up time threshold. During the continuous sending of probe query statements, the probe query statements themselves also continuously trigger data loading and cache filling operations in the target database. Therefore, sending probe query statements multiple times is itself a means of accelerating cache warm-up. In one embodiment, the migration control unit can repeatedly send probe query statements at fixed time intervals, such as every 200 milliseconds or every 500 milliseconds, and record the response time each time. If the response time does not decrease significantly after multiple sendings, or even shows an upward trend, the migration control unit can generate an alarm message to prompt maintenance personnel to check whether there are any abnormalities in the target database's operating status, such as excessive CPU load or bottlenecks in the input / output system.

[0064] Through the above embodiments, after switching the data transmission service to bidirectional incremental data synchronization, this application sends probe query statements to the target database and determines the cache preheating status based on the response time. The backend address addition operation is only performed after the cache preheating is complete, thus proactively verifying and preparing the target database's traffic handling capacity. Compared to directly adding a backend address to instantly introduce traffic into the target database, this embodiment, through proactive preheating and status judgment, ensures that the target database is in an optimal response state when handling application access traffic, avoiding slow request responses or timeouts caused by cache cold starts, and further improving the smoothness of the migration process and user experience.

[0065] Based on any embodiment of the method in this application, when verifying the availability of the target database, the method includes: Step S5410: Obtain the current number of connections and the current response time of the target database; When verifying the availability of a target database, it is necessary to first obtain the current number of connections and the current response time. The current number of connections refers to the number of active connections currently being processed by the target database, reflecting its connection resource usage. In one embodiment, the current number of connections can be obtained by sending a Structured Query Language (SCL) statement to the target database to query its connection status, for example, by executing a statement querying the number of currently active connections. The current response time refers to the average time required for the target database to respond to a query request or the most recent response time, reflecting the target database's processing capacity and load status. In one embodiment, the current response time can be obtained by sending a lightweight probe query to the target database and measuring the time elapsed from sending to receiving the complete result.

[0066] The current number of connections and the current response time are two complementary metrics. The current number of connections reflects the saturation of the target database's connection resources, while the current response time reflects the target database's processing performance and response speed. Combining the two can comprehensively assess whether the target database has the capacity to handle application access traffic.

[0067] Step S5420: When the current number of connections is less than a preset connection number threshold and the current response time is less than a preset response time threshold, the target database is determined to be available; After obtaining the current number of connections and the current response time, the current number of connections is compared with a preset connection number threshold, and the current response time is compared with a preset response time threshold. The preset connection number threshold is a pre-configured parameter used to define the upper limit of the acceptable number of connections for the target database. Its value can be set according to the maximum connection number configuration and safety margin of the target database. In one embodiment, if the maximum number of connections for the target database is configured to be 1000, the preset connection number threshold can be set to 800, that is, reserving 20% ​​of connection resources as a buffer. The preset response time threshold is also a pre-configured parameter used to define the upper limit of the acceptable response time for the target database. Its value can be set according to the service level agreement requirements of the e-commerce platform regarding database response time. In one embodiment, for online transaction scenarios sensitive to response time, the preset response time threshold can be set to 50 milliseconds or 100 milliseconds. When the current number of connections is less than the preset connection number threshold and the current response time is less than the preset response time threshold, it indicates that the target database has sufficient connection resources to handle new access requests and can quickly respond to query requests. At this time, it can be determined that the target database is available, and then the operation of removing the backend address of the source database is executed.

[0068] Step S5430: When the current number of connections is greater than or equal to the preset connection number threshold, or the current response time is greater than or equal to the preset response time threshold, the target database is determined to be unavailable, and the availability check of the target database is re-initiated. When the current number of connections is greater than or equal to a preset connection threshold, or the current response time is greater than or equal to a preset response time threshold, it indicates that the target database may have insufficient connection resources or degraded processing performance. In this case, the migration control unit determines that the target database is unavailable and re-initiates an availability check. When re-initiating the availability check, it returns to step S5410 to obtain the current number of connections and the current response time of the target database again, and then compares them with their respective preset thresholds, and so on. In one embodiment, a certain time interval can be set between two availability checks, such as a 3-second or 5-second interval, to avoid excessively frequent checks causing additional burden on the target database.

[0069] Step S5440: If the availability check is determined to be unavailable after a preset number of consecutive tests, the process is terminated and the backend of the load balancing agent is kept pointing to the source database.

[0070] If, after a preset number of availability checks, each check results in the target database being deemed unavailable, it indicates a persistent abnormal state in the target database, making it unsuitable for handling application access traffic. In this case, the migration control unit halts the entire database migration process while maintaining the load balancer's backend pointing to the source database. This ensures that application access traffic continues to be processed through the source database, preventing disruption to normal business operations. The preset number of checks can be set to 3 or 5, depending on the tolerance for migration success rate. Simultaneously with the process halt, the migration control unit generates an alarm and sends it to the operations personnel's client, notifying them to investigate and resolve the target database's anomaly.

[0071] Through the above embodiments, this application uses both the current number of connections and the current response time as dual indicators to comprehensively judge the availability of the target database when verifying its availability. It also provides retry and abort mechanisms when verification fails, achieving refined and reliable verification of the target database's availability. Compared to methods relying solely on a single indicator or a single verification, this embodiment improves the accuracy of availability judgment through cross-validation of dual indicators, avoids misjudgments caused by instantaneous fluctuations through the retry mechanism, and protects the continuity of e-commerce operations through the abort mechanism when the target database remains unavailable. Thus, it enhances the robustness of the migration process while ensuring switchover security.

[0072] Based on any embodiment of the method in this application, after adjusting the access method of the application access traffic to directly point to the target database, the method includes: Step S5511: Continuously monitor the performance metrics data of the target database, wherein the performance metrics data includes at least one of processor utilization, memory usage, and disk read / write bandwidth; After adjusting the application's access traffic to directly target the target database, the migration control unit continuously monitors the target database's performance metrics. Performance metrics are quantitative data reflecting the target database's operating status and load, and may include at least one of processor utilization, memory usage, and disk read / write bandwidth. Processor utilization refers to the percentage of the target database server's CPU that is occupied, reflecting the CPU's workload. Memory usage refers to the percentage of the target database server's memory that is used, reflecting memory resource usage. Disk read / write bandwidth refers to the amount of data read and written per second on the target database server's disk, reflecting the load on the disk I / O system.

[0073] In one embodiment, these performance metrics data can be obtained through the monitoring interface provided by the operating system of the server hosting the target database. For example, real-time values ​​of processor utilization, memory usage, and disk read / write bandwidth can be obtained by reading system files or calling system commands. The migration control unit can collect the performance metrics data once according to a preset sampling period, such as every 10 seconds or every 30 seconds, and compare the collected data with their respective preset thresholds.

[0074] Step S5512: When the performance index data exceeds its corresponding preset threshold and the corresponding preset duration, re-establish the bidirectional incremental data synchronization and restore the access method of the application access traffic to the source database via the load balancing proxy; otherwise, maintain the access method of the application access traffic as directly pointing to the target database.

[0075] After obtaining the performance metric data, it can be determined whether each performance metric exceeds its corresponding preset threshold and whether the exceeding state has lasted for the corresponding preset duration. Each performance metric corresponds to an independent preset threshold and an independent preset duration. The values ​​of these parameters can be set according to the configuration specifications of the target database and the business needs of the e-commerce platform during off-peak traffic periods. For example, for processor utilization, the preset threshold can be set to 80%, and the preset duration can be set to 60 consecutive seconds; for memory usage, the preset threshold can be set to 90%, and the preset duration can be set to 120 consecutive seconds; for disk read / write bandwidth, the preset threshold can be set to 70% of the maximum disk bandwidth, and the preset duration can be set to 30 consecutive seconds. Only when a performance metric exceeds its corresponding preset threshold and the duration exceeds its corresponding preset duration is the metric considered to have triggered an alert condition.

[0076] When any of the monitored performance metrics triggers its corresponding warning condition, it indicates that the overall operating status of the target database has deteriorated and may be unable to stably handle the current business traffic. In this case, a rollback operation can be performed. The rollback operation includes re-establishing bidirectional incremental data synchronization and restoring the access method of application traffic to the source database via the load balancing proxy. It should be noted that, to accommodate the rollback requirement, step S5400 may not require deleting the backend address of the source database, or, if the backend address has been deleted, it may be necessary to rewrite the backend address of the source database to the load balancing proxy.

[0077] When re-establishing bidirectional incremental data synchronization, the migration control unit sends a command to the data transmission service to restart the data capture components on both the source and target databases, establishing a bidirectional real-time data synchronization channel. Then, the application access traffic is restored to be routed through a load balancer to the source database. This can be achieved by modifying the database connection configuration file on the application server or through domain name resolution, ensuring that application access traffic is once again forwarded to the source database via the load balancer. Through this rollback operation, the e-commerce platform's business traffic is switched from the target database, which may have experienced performance bottlenecks, back to the stable source database, ensuring business continuity and stability.

[0078] When at least one of the performance metrics does not exceed its corresponding preset threshold, or exceeds the preset threshold but does not last for the corresponding preset duration, it indicates that the target database's operating status is still within an acceptable range. At this time, the migration control unit maintains the application access traffic access method as directly pointing to the target database and continues to monitor the changing trends of various performance metrics.

[0079] Through the above embodiments, this application, after adjusting application access traffic to directly target the target database, continuously monitors the target database's performance metrics and automatically performs a switchback operation when the metrics continuously exceed limits, thus achieving automated monitoring and protection of the target database's operational status. In the elastic scaling scenario of e-commerce platforms, low-cost target databases have limited performance margins and may experience performance bottlenecks after handling business traffic. This embodiment, by automatically switching traffic back to the source database when performance deteriorates, fully utilizes the cost advantage of the low-cost target database under normal conditions and promptly switches back to the high-performance source database when the target database's performance is insufficient to ensure business stability, thereby achieving a dynamic balance between cost reduction and performance assurance.

[0080] Based on any embodiment of the method in this application, after stopping the bidirectional incremental data synchronization, the method includes: Step S5521: Perform data integrity verification on the target database. The data integrity verification includes at least one of the following: total record count verification, key field verification, and comparison of business data node elements. After stopping bidirectional incremental data synchronization, data integrity verification can still be performed on the target database. Data integrity verification is a process used to verify whether the data in the target database is completely consistent with the data in the source database. Its purpose is to ensure that no data loss, data errors, or data inconsistencies have occurred during the migration process.

[0081] Data integrity verification can be implemented using at least one of several verification methods. Record count verification is a basic method. It involves counting the total number of records in a specified table in both the source and target databases and comparing the results. If the total number of records matches, it initially indicates that no data is missing. Key field checksum verification is a more precise method. It calculates the checksum of key fields in a specified table in both the source and target databases. For example, it calculates the sum or cyclic redundancy check value for the order amount field in an order table and compares the results. If the checksums match, it indicates that the data content of the key fields has not changed. Business data node element comparison is a more comprehensive method. It involves selecting several representative business data records and comparing their values ​​field-by-field in the source and target databases, including string content, numerical precision, and date / time formats, ensuring that each selected record is completely consistent across all fields.

[0082] In one embodiment, for order data from an e-commerce platform, all order records within the most recent hour can be selected, and the values ​​of core fields such as order number, user identifier, product identifier, order amount, order status, and creation time can be compared one by one in the source database and the target database. If every field of all selected records is completely consistent, the data integrity verification is deemed to have passed.

[0083] Step S5522: If the data integrity verification fails, re-establish the bidirectional incremental data synchronization and restore the access method of the application access traffic to the source database via the load balancing proxy.

[0084] When data integrity verification fails, it indicates a discrepancy between the data in the target database and the data in the source database. This discrepancy may be due to data omissions, synchronization delays, or data conflicts during the migration process. In this case, the migration control unit performs a rollback operation, including re-establishing bidirectional incremental data synchronization and restoring the application access traffic to the source database via the load balancing agent. It should be noted that, to accommodate the rollback, step S5400 may not require deleting the source database's backend address; alternatively, if the backend address has been deleted, it can be rewritten to the load balancing agent.

[0085] When re-establishing bidirectional incremental data synchronization, the migration control unit sends instructions to the data transmission service to restart the data capture components on both the source and target databases, establishing a bidirectional real-time data synchronization channel to bring the data between the two databases back to consistency. Then, the application access traffic is restored to be routed through a load balancer to the source database. This can be achieved by modifying the database connection configuration file on the application server or through domain name resolution, ensuring that application access traffic is forwarded to the source database via the load balancer. Through this rollback operation, the e-commerce platform's business traffic is switched from the inconsistent target database back to the complete and stable source database, preventing business errors caused by data inconsistency. After the rollback is complete, operations personnel can investigate and repair the cause of the data inconsistency, and then re-execute the migration process once the problem is resolved.

[0086] Through the above embodiments, this application achieves data security assurance in the final stage of migration by performing data integrity verification on the target database after stopping bidirectional incremental data synchronization and automatically executing a rollback operation when the verification fails. In the elastic scaling scenario of e-commerce platforms, data integrity is the cornerstone of business operations. Any data loss or error may lead to serious consequences such as order disorder, inventory discrepancies, or financial reconciliation failures. This embodiment, by adding a data integrity verification and automatic rollback mechanism at the final stage of the migration process, ensures that the migration is only completed when the data is complete and error-free; otherwise, it promptly rolls back to the source database, thereby safeguarding the bottom line of data security while pursuing cost reduction and efficiency improvement.

[0087] Based on any embodiment of the method in this application, after stopping the bidirectional incremental data synchronization, the method includes: Step S5531: Perform a high-concurrency stress test on the target database. The test scenario of the high-concurrency stress test includes simulating the expected concurrent access volume of the e-commerce platform during non-peak traffic periods. After stopping bidirectional incremental data synchronization, high-concurrency stress testing can be performed on the target database. High-concurrency stress testing is a verification method that simulates a large number of concurrent access requests to examine the stability and performance of the target database under high load conditions. Test scenarios for high-concurrency stress testing include simulating the expected concurrent access volume of an e-commerce platform during off-peak traffic periods. This means that the design of test scenarios should be based on the analysis of historical data on business traffic during off-peak periods to determine parameters such as the typical number of concurrent requests, the distribution of request types, and the request frequency during that period, and test cases should be constructed accordingly.

[0088] For example, if an e-commerce platform experiences an average of 2000 concurrent requests per second during off-peak hours, with 70% being read requests and 30% being write requests, a high-concurrency stress test can simulate the real business load during off-peak hours by sending requests to the target database at a ratio of 70% read requests and 30% write requests. This can be run continuously for a period of time, such as 10 or 30 minutes. During the test, the migration control unit can record various performance metrics of the target database, including average response time, maximum response time, request success rate, and throughput, as a basis for subsequent determination of whether the test meets the standards.

[0089] Step S5532: When the high-concurrency stress test is passed, the source database is taken offline and the computing and storage resources occupied by the source database are reclaimed; After the high-concurrency stress test is completed, the migration control unit determines whether the test has passed based on the test results. The criteria for passing the test can include a comprehensive evaluation of multiple indicators, such as an average response time less than the preset upper limit, a request success rate greater than the preset lower limit, and throughput greater than the preset lower limit. For example, the preset upper limit for response time can be set to 200 milliseconds, the preset lower limit for success rate can be set to 99.9%, and the preset lower limit for throughput can be set to 1800 requests per second. When the high-concurrency stress test passes, it indicates that the target database can run stably and meet performance requirements under simulated off-peak traffic loads. At this point, the migration control unit takes the source database offline and reclaims the computing and storage resources occupied by the source database.

[0090] Taking a source database offline can include stopping its service processes, removing its node information from the database cluster, and releasing the cloud or physical resources it occupies. The recovered computing and storage resources can include CPU resources, memory resources, disk space, and network bandwidth. These resources can be reallocated to other business systems, thereby optimizing resource allocation and reducing operating costs.

[0091] With the source database offline, the existence of the load balancing agent is no longer necessary. Therefore, in one embodiment, the load balancing agent can also be removed to release the corresponding computing and storage resources.

[0092] Step S5533: If the high-concurrency stress test fails to meet the standard, perform performance optimization operations on the target database and re-execute the high-concurrency stress test until the high-concurrency stress test meets the standard.

[0093] When the high-concurrency stress test fails to meet the requirements, it indicates that the target database cannot meet the performance requirements during off-peak hours under its current configuration. In this case, the migration control unit can perform performance optimization operations on the target database and re-run the high-concurrency stress test after optimization until the test meets the requirements. Performance optimization operations can include various specific measures, such as adjusting the target database's configuration parameters, such as increasing the buffer size, optimizing the query cache configuration, or adjusting the connection pool size; analyzing and optimizing slow queries in the target database, such as adding missing indexes or rewriting inefficient query statements; or adjusting the resource configuration of the server hosting the target database, such as increasing the number of CPU cores or expanding the memory capacity.

[0094] For example, if the average response time reaches 350 milliseconds in the first high-concurrency stress test, exceeding the preset response time limit of 200 milliseconds, the migration control unit can first try increasing the target database's buffer size and then re-execute the high-concurrency stress test. If the average response time drops to 250 milliseconds in the second test, but still fails to meet the standard, slow queries can be further optimized, and then a third test can be performed until the average response time drops below 200 milliseconds. Through this closed-loop process of iterative optimization and testing, it ensures that the target database reaches the expected performance level before officially handling business traffic.

[0095] Through the above embodiments, this application, after stopping bidirectional incremental data synchronization, performs high-concurrency stress testing on the target database and only takes the source database offline after the test meets the standards, thus achieving the final verification of the target database's ability to handle business traffic. In the elastic scaling scenario of e-commerce platforms, whether the performance of the low-cost target database is sufficient to support business operations during off-peak traffic periods is a key factor determining the success or failure of the migration. This embodiment, by adding high-concurrency stress testing and iterative optimization mechanisms in the final stage of migration, ensures that the target database has undergone sufficient performance verification before actually handling business traffic, avoiding the risk of business damage after going live due to insufficient target database performance, thereby achieving cost reduction and efficiency improvement while ensuring stable business operation.

[0096] Please see Figure 3According to one aspect of this application, a database migration control device includes a proxy configuration module 5100, a migration startup module 5200, a mode switching module 5300, an address replacement module 5400, and a traffic switching module 5500. The proxy configuration module 5100 is configured to create a load balancing proxy, setting the access method of e-commerce platform application access traffic to point to the source database via the load balancing proxy. The source database is used to handle peak access traffic during peak e-commerce platform traffic periods. The migration startup module 5200 is configured to respond to a migration startup event triggered when the e-commerce platform enters a non-peak traffic period, and then initiate unidirectional incremental data synchronization after unidirectionally synchronizing the data in the source database to the target database via a data transmission service. The target database is used for... The system handles normal access traffic to the e-commerce platform during off-peak hours. The mode switching module 5300 is configured to switch the data transmission service from one-way incremental data synchronization to two-way incremental data synchronization in response to an elastic contraction command, enabling real-time synchronization of incremental data between the source database and the target database. The address replacement module 5400 is configured to add the backend address of the target database to the load balancing proxy, and remove the backend address of the source database after verifying the availability of the target database, thereby replacing the backend pointing of the load balancing proxy from the source database to the target database. The traffic switching module 5500 is configured to change the access method of the application access traffic from via the load balancing proxy to directly pointing to the target database, and stop the two-way incremental data synchronization.

[0097] Based on any embodiment of the device in this application, the migration startup module 5200 includes: a latency monitoring module, configured to continuously monitor the data latency value between the source database and the target database; an automatic triggering module, configured to trigger the elastic contraction command when the data latency value is less than a preset latency threshold; and a mode maintenance module, configured to maintain the unidirectional incremental data synchronization until the data latency value is less than the preset latency threshold when the data latency value is greater than or equal to the preset latency threshold.

[0098] Based on any embodiment of the device in this application, the mode switching module 5300 includes: a preheating detection module, configured to send a probe query statement to the target database and determine the cache preheating status of the target database based on the response time of the probe query statement; a completion processing module, configured to determine that the target database has completed cache preheating when the response time is less than a preset preheating time threshold, and then execute the step of adding the backend address of the target database to the load balancing agent; and an incomplete iteration module, configured to continue sending the probe query statement to the target database until the response time is less than the preset preheating time threshold when the response time is greater than or equal to the preset preheating time threshold.

[0099] Based on any embodiment of the device in this application, the address replacement module 5400 includes: a data acquisition module, configured to acquire the current number of connections and the current response time of the target database; an availability determination module, configured to determine that the target database is available when the current number of connections is less than a preset connection number threshold and the current response time is less than a preset response time threshold; a verification restart module, configured to determine that the target database is unavailable and re-initiate the availability verification of the target database when the current number of connections is greater than or equal to the preset connection number threshold, or the current response time is greater than or equal to the preset response time threshold; and a stop-switch module, configured to stop the process and keep the backend of the load balancing agent pointing to the source database when the availability verification is determined to be unavailable for a preset number of consecutive times.

[0100] Based on any embodiment of the device in this application, the traffic switching module 5500 includes: an indicator monitoring module, configured to continuously monitor the performance indicator data of the target database, the performance indicator data including at least one of processor utilization, memory usage, and disk read / write bandwidth; and an indicator response module, configured to, when the performance indicator data exceeds its corresponding preset threshold and a corresponding preset duration, re-establish the bidirectional incremental data synchronization and restore the access method of the application access traffic to the source database via the load balancing proxy; otherwise, maintain the access method of the application access traffic as directly pointing to the target database.

[0101] Based on any embodiment of the device in this application, the traffic switching module 5500 includes: a complete verification module, configured to perform data integrity verification on the target database, wherein the data integrity verification includes at least one of record total count verification, key field verification and business data node element comparison; and an anomaly back-off module, configured to re-establish the bidirectional incremental data synchronization and restore the access method of the application access traffic to the source database via the load balancing proxy when the data integrity verification fails.

[0102] Based on any embodiment of the device in this application, the traffic switching module 5500 includes: a test initiation module, configured to perform a high-concurrency stress test on the target database, wherein the test scenario of the high-concurrency stress test includes simulating the expected concurrent access volume of the e-commerce platform during non-peak traffic periods; a pass-through offline module, configured to take the source database offline and reclaim the computing and storage resources occupied by the source database when the high-concurrency stress test passes the test; and an optimization retest module, configured to perform performance optimization operations on the target database and re-execute the high-concurrency stress test when the high-concurrency stress test fails the test, until the high-concurrency stress test passes the test.

[0103] Another embodiment of this application also provides an electronic device. For example... Figure 4 The diagram shows the internal structure of an electronic device. This electronic device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable, non-volatile storage medium stores an operating system, a database, and computer-readable instructions. The database can store information sequences, and when executed by the processor, the computer-readable instructions enable the processor to implement a database migration control method.

[0104] The processor of this electronic device provides computing and control capabilities to support the operation of the entire device. The memory of this electronic device can store computer-readable instructions, which, when executed by the processor, cause the processor to perform the database migration control method of this application. The network interface of this electronic device is used for communication with a terminal.

[0105] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0106] In this embodiment, the processor is used to execute... Figure 3 The specific functions of each module are described, and the memory stores the program code and various data required to execute the aforementioned modules or sub-modules. The network interface is used to enable data transmission between user terminals or the server. In this embodiment, the non-volatile readable storage medium stores the program code and data required to execute all modules in the database migration control device of this application. The server can call the server's program code and data to execute the functions of all modules.

[0107] This application also provides a non-volatile readable storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the database migration control method of any embodiment of this application.

[0108] This application also provides a computer program product, including a computer program / instructions that, when executed by one or more processors, implement the steps of the method described in any embodiment of this application.

[0109] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM).

Claims

1. A database migration control method, characterized in that, include: Create a load balancing proxy and set the access method of the e-commerce platform application access traffic to be directed to the source database via the load balancing proxy. The source database is used to carry the peak access traffic during peak hours of the e-commerce platform traffic. In response to the migration start event triggered when the e-commerce platform enters a non-peak traffic period, the data in the source database is fully synchronized one-way to the target database through the data transmission service, and then one-way incremental data synchronization is started. The target database is used to carry the normal access traffic of the e-commerce platform during non-peak traffic periods. In response to the elastic contraction command, the data transmission service is switched from one-way incremental data synchronization to two-way incremental data synchronization, enabling the source database and the target database to synchronize incremental data with each other in real time. Add the backend address of the target database to the load balancing agent, and remove the backend address of the source database after verifying that the target database is available, so as to replace the backend of the load balancing agent from the source database to the target database; Change the access method of the application access traffic from passing through the load balancer to directly pointing to the target database, and stop the bidirectional incremental data synchronization.

2. The database migration control method according to claim 1, characterized in that, After initiating unidirectional incremental data synchronization, the following is included: Continuously monitor the data latency value between the source database and the target database; When the data delay value is less than a preset delay threshold, the elastic contraction command is triggered; When the data delay value is greater than or equal to the preset delay threshold, the one-way incremental data synchronization is maintained until the data delay value is less than the preset delay threshold.

3. The database migration control method according to claim 1, characterized in that, After switching the data transmission service from one-way incremental data synchronization to two-way incremental data synchronization, the following is included: Send a probe query statement to the target database, and determine the cache preheating status of the target database based on the response time of the probe query statement; When the response time is less than the preset warm-up time threshold, it is determined that the target database has completed cache warm-up, and then the step of adding the backend address of the target database to the load balancing agent is executed. When the response duration is greater than or equal to the preset preheating duration threshold, the probe query statement continues to be sent to the target database until the response duration is less than the preset preheating duration threshold.

4. The database migration control method according to claim 1, characterized in that, When verifying the availability of the target database, the following are included: Obtain the current number of connections and the current response time of the target database; When the current number of connections is less than a preset connection threshold and the current response time is less than a preset response time threshold, the target database is determined to be available. When the current number of connections is greater than or equal to the preset connection number threshold, or the current response time is greater than or equal to the preset response time threshold, the target database is determined to be unavailable, and the availability check of the target database is re-initiated. If the availability check is determined to be unavailable after a preset number of consecutive tests, the process is terminated and the backend of the load balancing agent is kept pointing to the source database.

5. The database migration control method according to any one of claims 1 to 4, characterized in that, After adjusting the access method of the application's access traffic to directly point to the target database, the following steps are included: Continuously monitor the performance metrics of the target database, including at least one of processor utilization, memory usage, and disk read / write bandwidth. When the performance index data exceeds its corresponding preset threshold and the corresponding preset duration, the bidirectional incremental data synchronization is re-established and the access method of the application access traffic is restored to pointing to the source database via the load balancing proxy; otherwise, the access method of the application access traffic is maintained to directly point to the target database.

6. The database migration control method according to any one of claims 1 to 4, characterized in that, After stopping the bidirectional incremental data synchronization, the following is included: Perform data integrity verification on the target database. The data integrity verification includes at least one of the following: record count verification, key field verification, and comparison of business data node elements. If the data integrity verification fails, the bidirectional incremental data synchronization is re-established and the access method of the application access traffic is restored to pointing to the source database via the load balancing proxy.

7. The database migration control method according to any one of claims 1 to 4, characterized in that, After stopping the bidirectional incremental data synchronization, the following is included: Perform high-concurrency stress tests on the target database. The test scenarios for the high-concurrency stress tests include simulating the expected concurrent access volume of the e-commerce platform during non-peak traffic periods. When the high-concurrency stress test is passed, the source database will be taken offline and the computing and storage resources occupied by the source database will be reclaimed. If the high-concurrency stress test fails to meet the standard, performance optimization operations are performed on the target database and the high-concurrency stress test is re-executed until the high-concurrency stress test meets the standard.

8. A database migration control device, characterized in that, include: The proxy configuration module is set to create a load balancing proxy, and the access method of the e-commerce platform application access traffic is set to be directed to the source database through the load balancing proxy. The source database is used to carry the peak access traffic during the peak period of e-commerce platform traffic. The migration startup module is configured to respond to a migration startup event triggered when the e-commerce platform enters a non-peak traffic period. After unidirectionally synchronizing the data in the source database to the target database through the data transmission service, it starts unidirectional incremental data synchronization. The target database is used to carry the normal access traffic of the e-commerce platform during non-peak traffic periods. The mode switching module is configured to switch the data transmission service from one-way incremental data synchronization to two-way incremental data synchronization in response to the elastic contraction command, so that the source database and the target database can synchronize incremental data with each other in real time. The address replacement module is configured to add the backend address of the target database to the load balancing agent, and remove the backend address of the source database after verifying that the target database is available, so as to replace the backend of the load balancing agent from the source database to the target database. The traffic switching module is configured to change the access method of the application access traffic from passing through the load balancing proxy to directly pointing to the target database, and stop the bidirectional incremental data synchronization.

9. An electronic device comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 7.

10. A non-volatile readable storage medium, characterized in that, It stores, in the form of computer-readable instructions, a computer program implemented according to any one of claims 1 to 7, which, when invoked by a computer, executes the steps included in the corresponding method.

Images