A middleware-based database cluster switchover system and method

Abstract

The application discloses a middleware-based database cluster switching system, which comprises a middleware management server group, a database middleware and a database cluster, wherein the middleware management server group comprises a first middleware management server and a second middleware management server, the first middleware management server manages a first database middleware group, and the second middleware management server manages a second database middleware group; the database middleware is used for data processing of the database cluster; and the database cluster comprises a first database cluster and a second database cluster, and each database cluster comprises a master database and a slave database.

Description

Technical Field

[0001] This invention relates to the field of data storage technology, and in particular to a middleware-based database cluster switching system and method. Background Technology

[0002] Clustering technology is a relatively new data storage technology that offers relatively high performance, reliability, and flexibility at a lower cost. A cluster is a group of independent computers interconnected by a high-speed network, managed as a single system. When a client interacts with a cluster, the cluster acts like a single server. Cluster configuration is used to improve availability and scalability. Database clustering is an application of clustering in the database field. In practice, database clusters are not permanent. When encountering situations such as data center relocation, network zone isolation, or vertical splitting of business operations, database clusters may switch between clusters. Since data is stored in the database cluster, switching between clusters can affect upper-layer services' data writing and reading. Therefore, efficient, reliable, and stable switching between database clusters is essential. Furthermore, if unexpected problems occur after switching from database cluster A to database cluster B, a rollback operation is required, i.e., switching back from database cluster B to database cluster A. This process is called database cluster switchover rollback.

[0003] Currently, there are two main methods for switching between database clusters: those based on database middleware and those based on domain names. Both have drawbacks to varying degrees in practical applications: the former requires modification of the middleware configuration file to adapt to the overall cluster migration, and rollback operations require further modification of the configuration file, causing secondary impacts on upper-layer applications. Furthermore, for scenarios where some data is sharded within the cluster, this method amplifies the impact because upper-layer applications need to modify their database cluster access configurations during the switchover process, increasing the update configuration process for service deployment and the risk of missed configuration updates. The latter method suffers from similar drawbacks; it requires modification of the domain name configuration file to adapt to the overall cluster migration, and rollback operations require further modification of the configuration file, causing secondary impacts on upper-layer applications. Additionally, for scenarios where some data is sharded within the cluster, this method amplifies the impact because upper-layer applications need to modify their database cluster access configurations during the switchover process, increasing the update configuration process for service deployment and the risk of missed configuration updates. Summary of the Invention

[0004] To address this, the present invention proposes a middleware-based database cluster switching system and method. Through this invention, upper-layer services do not need to update configurations during the database cluster switching process, while eliminating the risk of missed service configuration updates. This achieves the effects of shortening switching time, improving switching efficiency, reducing influencing factors, and ensuring controllable impact.

[0005] To achieve the objectives of this invention, the following technical solution is adopted:

[0006] A middleware-based database cluster switching system includes a middleware management server group, database middleware, and a database cluster. The middleware management server group includes a first middleware management server and a second middleware management server. The first middleware management server manages a first database middleware group, and the second middleware management server manages a second database middleware group. The database middleware is used for data processing within the database cluster. The database cluster includes a first database cluster and a second database cluster, and each database cluster includes a master database and slave databases.

[0007] The database cluster switching system is described above, wherein the master database and the slave database are MySQL databases.

[0008] The database cluster switching system includes a first middleware management server and a second middleware server, which are used to receive upper-layer service requests and send them to their respective database middleware groups. The database middleware groups are used to receive service requests and send them to the lower-layer database cluster.

[0009] The database cluster switching system includes a database middleware group comprising at least one database middleware, which is used to configure the master and slave databases.

[0010] The database cluster switching system wherein data is synchronized in real time between the first database cluster and the second database cluster.

[0011] The database cluster switching system, wherein:

[0012] The first database middleware group has the same configuration, which is the master database and slave database of the first database cluster; the second database middleware group has the same configuration, which is the master database of the first database cluster and the slave database of the second database cluster.

[0013] The database cluster switching system, wherein:

[0014] The first database middleware group is entirely open to the public, while only a portion of the second database middleware group is open to the public.

[0015] The database cluster switching system, as described above, involves the following steps: First, the second database middleware group is completely disabled; the configuration of the second database middleware group is updated to include the master and slave databases of the second database cluster, and the master database of the second database cluster is set to a write-disabled state; then, the middleware management server group is updated to have only one member, the second middleware management server, and the second database middleware group is fully opened to provide services to the outside world; the write function of the master database of the second database cluster is enabled, and real-time data synchronization is performed by connecting upstream to the slave database of the second database cluster and downstream to the master database of the first database cluster through a data synchronization tool. At this point, the cluster switchover is complete.

[0016] The database cluster switching system described above includes the following steps: After the switch is completed, the operation of the database middleware logs is observed to check whether the read and write traffic is normal. If it is normal, the switch is successful; if it is abnormal, the system enters the rollback phase.

[0017] The database cluster switching system, during rollback, firstly disables the write function of the primary database of the second database cluster and disables data synchronization; secondly, updates the middleware management server group to have only one member, the first middleware management server, and simultaneously rolls back the database cluster architecture to the original architecture before the switchover, observing whether the traffic is normal.

[0018] A middleware-based database cluster switching method is applied to the database cluster switching system. The method includes: during switching, firstly, completely disabling the second database middleware group; updating the configuration of the second database middleware group to include the master and slave databases of the second database cluster, and setting the master database of the second database cluster to a write-disabled state; then updating the middleware management server group to have only one member (the second middleware management server), and fully opening the second database middleware group to provide services externally; enabling the write function of the master database of the second database cluster, and using a data synchronization tool to connect upstream to the slave database of the second database cluster and downstream to the master database of the first database cluster for real-time data synchronization, thus completing the cluster switching.

[0019] A computer-readable storage device is characterized by storing processor-executable instructions that, when executed by a processor, cause the processor to perform the method described above. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the middleware-based database cluster switching system architecture. Detailed Implementation

[0021] The following is in conjunction with the appendix Figure 1The specific embodiments of the present invention will be described in detail below. These embodiments are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. Obviously, the embodiments described in this invention are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0022] The terms "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include the specific features, structures, or characteristics described in connection with that embodiment. Therefore, the terms "comprising," "including," "having," and variations thereof in this specification mean "including but not limited to," unless otherwise specifically emphasized.

[0023] like Figure 1 As shown, the middleware-based database cluster switching system of the present invention includes a middleware management server group (group-bns), database middleware, and database cluster.

[0024] The middleware management server group includes multiple middleware management servers (bns) for receiving upper-layer service instructions and requests. In this embodiment, two middleware management servers are optionally shown: the first middleware management server dbproxy-bns-A and the second middleware management server dbproxy-bns-B. Each middleware management server (bns) manages a group of database middleware. In this embodiment, the database middleware is illustrated as dbproxy, but it is not limited to dbproxy and other database middleware such as Tomcat can also be selected. The database cluster includes database cluster A and database cluster B. Each database cluster includes multiple MySQL databases, including master databases and slave databases. One optional database configuration is 1 master database + 2 slave databases.

[0025] Middleware management server (bns) provides a mapping between service names and running service instances. It allows you to retrieve information about the running instances of a service based on its name, including the instance's hostname, IP address, running status, port, and custom instance information. A server group (group-bns) is a collection of middleware management server (bns) of the same type.

[0026] Database middleware provides functions such as read / write separation, access control, load balancing, and account management for databases. Database cluster information can be configured through configuration files, with one database cluster corresponding to one dbproxy.

[0027] In this implementation, the group-bns server group includes two members: DBproxy-BNS-A and DBproxy-BNS-B. DBproxy-BNS-A hosts one or more DBproxy instances, each acting as a database middleware, receiving service requests from the upper layer and forwarding them to the lower-layer database cluster A. Similarly, DBproxy-BNS-B functions identically to DBproxy-BNS-A, receiving service requests from the upper layer and forwarding them to the lower-layer database cluster B. The database middleware (dbproxy) supports various configuration functions, including account configuration and MySQL instance configuration, allowing for the configuration of MySQL master and MySQL slave databases. Real-time data synchronization is performed between the first database cluster A and the second database cluster B using a data synchronization tool.

[0028] The following section details the database cluster switching process, using the example of switching from database cluster A to database cluster B:

[0029] (1) Pre-switch phase

[0030] Initially, the group-bns server group has only one member, DBproxy-BNS-A. During the pre-cluster switchover preparations, DBproxy-BNS-B is added to the group-bns server group. At this point, the group-bns server group has two members: DBproxy-BNS-A and DBproxy-BNS-B. DBproxy-BNS-A has one or more DBproxy instances mounted on it, each with the same configuration, configured as the master and slave databases of the first database cluster A. DBproxy-BNS-B also has one or more DBproxy instances mounted on it, each with the same configuration, configured as the master database of the first database cluster A and a slave database of the second database cluster B.

[0031] After ensuring all instances mounted on DBproxy-BNS-A are exposed to the public, a subset of the instances mounted on DBproxy-BNS-B will also be exposed. For example, if DBproxy-BNS-B has 8 instances mounted, initially expose 1-2 instances. The advantage of this configuration is that it allows for small-scale testing of the connectivity between DBproxy-BNS-B and database clusters A and B. If unexpected anomalies occur, the exposed DBproxy instances can be blocked immediately for efficient damage mitigation. If normal operation is achieved, the scope of exposed instances can be gradually expanded until all instances mounted on DBproxy-BNS-B are exposed.

[0032] In this embodiment, the connectivity between DBproxy-BNS-B and database clusters A and B is verified as follows: The second middleware management server, DBproxy-BNS-B, sends read / write traffic access requests to the master database of the first database cluster A and read traffic access requests to the slave database of the second database cluster B. The system then determines whether read / write access can be performed normally, and whether the account, password, and permissions are correct and without error messages. If so, then DBproxy-BNS-B is connected to database clusters A and B.

[0033] This invention uses a data synchronization tool to connect upstream to the slave database of database cluster A and downstream to the master database of database cluster B, so as to achieve real-time data synchronization between database cluster A and database cluster B.

[0034] In each database cluster, the data between the master and slave databases should be consistent, and the data in the slave databases should be updated through the master database. The slave databases only provide read services to the outside world, while the master databases can provide both read and write services to the outside world.

[0035] (2) Switching phase

[0036] Before the switchover, DBproxy-BNS-B was configured as the master database of database cluster A and the slave database of database cluster B. During the switchover, this configuration needs to be changed to reflect the information of the database cluster B to be switched over. First, all dbproxy instances mounted on DBproxy-BNS-B are disabled. This step ensures that DBproxy-BNS-B does not provide external services to avoid unexpected issues. Then, the dbproxy middleware configuration of DBproxy-BNS-B is updated to reflect the master and slave databases of database cluster B, and the master database of database cluster B is set to write-disabled mode.

[0037] Pre-switch checks: 1. Check if the master database of database cluster B is in a write-prohibited state. If so, proceed; otherwise, set the master database of database cluster B to a write-prohibited state. This is to avoid dual-write behavior and ensure data consistency between database clusters A and B. 2. Check if there is data latency between the master and slave databases of database cluster A. If there is latency, wait for the latency issue to be resolved before switching clusters. This is to ensure real-time data synchronization between database clusters A and B, guaranteeing data consistency during switching, and eliminating the need to handle latency issues during switching, thus improving the efficiency of cluster switching. Similarly, check if there is data latency between the master and slave databases of database cluster B. If there is latency, wait for the latency issue to be resolved before switching clusters. This is to ensure that when upper-layer services request the master and slave databases of database cluster B during the switching process, they obtain the same data without discrepancies. 3. Check if the slave databases of database cluster B have permissions for data synchronization tools. If they do, proceed with the switch; otherwise, grant permissions before switching. This is to ensure the stable synchronization of data from database cluster B to database cluster A, and to ensure the feasibility of the rollback plan should unexpected issues arise after cluster switchover. Cluster switchover can only proceed after all the above checks meet the requirements.

[0038] Initiating the switchover: First, set the master database of database cluster A to a write-free state and begin the switchover. In scenarios involving partial data migration within the cluster, stopping the DBproxy middleware of DBproxy-BNS-A achieves the write-free effect. This ensures that no upper-layer services access database cluster A for writes during and after the switchover process, preventing the issue of services simultaneously writing to both database cluster A and database cluster B. Second, it is necessary to verify whether the data in the master database of database cluster A at the write-free moment and before that moment is fully synchronized to database cluster B. The database administrator manipulates the migration heartbeat table of database cluster A, writing a verification record to the migration heartbeat table. Database cluster B checks the migration heartbeat table to see if the written verification record has been synchronized to its own migration heartbeat table. If so, the data-level switchover is complete, and the middleware switchover continues; otherwise, wait for the record to be checked before proceeding with the middleware switchover. This ensures that the data in database cluster A is fully synchronized to database cluster B during the migration process, guaranteeing data integrity and consistency. Next, update the group-bns server group to have only one member, DBproxy-BNS-B (the final state of the group-bns server group after the switch is complete). Simultaneously, all instances mounted on DBproxy-BNS-B are made available to the public. Finally, enable write functionality on the master database of database cluster B and check if read / write traffic is normal. One example of checking read / write traffic is to determine if there are still services accessing database cluster A. If so, it indicates abnormal read / write traffic, the switch failed, and a rollback is needed. Another example is to check the logs for read / write error messages. If there are only brief errors within a predetermined period, ignore them. If there are continuous abnormal errors over a period of time, it indicates the switch failed, and a rollback is needed. At this point, the switch from dataset cluster A to database cluster B is complete. If, during any of the above processes, the MySQL service in database cluster A or database cluster B unexpectedly stops, the machine hosting the MySQL service unexpectedly crashes, the upper-layer service unexpectedly stops, or the machine hosting the upper-layer service unexpectedly crashes, in these cases, the operations performed need to be rolled back in the order they occurred to restore the architecture to the state before the switch.

[0039] (3) Post-switch phase

[0040] If the above operations are normal, reverse data synchronization is performed using a data synchronization tool. The upstream connection is to the slave database of database cluster B, and the downstream connection is to the master database of database cluster A. Real-time data synchronization is performed, and an observation period of 1-2 weeks is entered. After the observation period, if the full service is normal, database cluster A and database cluster B will be isolated and become two independent clusters.

[0041] (4) Rollback phase

[0042] During rollback, firstly, write functionality is disabled on the primary database of database cluster B. Disabling writes ensures data consistency during the rollback phase. Secondly, data synchronization from database cluster B to database cluster A is disabled. It's necessary to verify whether the data on the primary database of database cluster B before and after the write-disabling time is fully synchronized to database cluster A. The database administrator manipulates the migration heartbeat table of database cluster B, writing a verification record to it. Database cluster A checks the migration heartbeat table to see if the written verification record has been synchronized. If so, the data-level rollback is complete, and middleware rollback continues; otherwise, the middleware rollback waits until the record is seen before proceeding. This ensures that incremental data written to database cluster B is fully synchronized to database cluster A, protecting data integrity and consistency. Thirdly, the group-bns server group is updated to have only one member, DBproxy-BNS-A, rolling back to the initial architecture before the switch. Write functionality is restored for database cluster A. In scenarios involving partial data migration within the cluster, the DBproxy middleware of DBproxy-BNS-A is enabled to achieve the effect of restoring write functionality. At this point, database cluster B is isolated and no longer exposed to the outside world. Only database cluster A is exposed to provide services. One example of checking if the traffic is normal is to determine if there is still service access to database cluster B. If so, it indicates abnormal read / write traffic, rollback fails, and the problem needs to be analyzed to mitigate losses. Another example of checking if the read / write traffic is normal is to check the logs for read / write error messages. If it is an expected, short-lived error (within a predetermined timeframe), it is ignored. If abnormal errors persist over a period of time, rollback fails, and the problem needs to be analyzed to mitigate losses.

[0043] According to one embodiment of the present invention, a computer-readable storage medium is provided. Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. Therefore, embodiments of this application provide a storage medium storing multiple instructions that can be loaded by a processor to execute the steps of the middleware-based database cluster switching method provided in this application.

[0044] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc. More specifically, it may include specific categories such as static random-access memory (SRAM) and dynamic random-access memory (DRAM).

[0045] Since the instructions stored in the storage medium can execute the middleware-based database cluster switching method of the present invention, the beneficial effects that can be achieved by the embodiments of the present invention can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0046] This invention leverages the flexibility of changing group-bns server members to manage cluster entry point changes, ensuring that upper-layer services do not need to update database configurations. Furthermore, the combination of bns and dbproxy addresses the issues of not being able to verify read traffic in advance during cluster switching and controlling the open ratio of entry point traffic, thus reducing risk and improving switching success rate.

[0047] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A middleware-based database cluster switching system, comprising a middleware management server group, database middleware, and a database cluster, characterized in that: The middleware management server group includes a first middleware management server and a second middleware management server. The first middleware management server manages a first database middleware group, and the second middleware management server manages a second database middleware group. The database middleware is used for data processing of the database cluster. The database cluster includes a first database cluster and a second database cluster, and each database cluster includes a master database and a slave database. The database cluster switching system switches database clusters in the following manner: (1) Pre-switching stage The initial middleware management server group includes one first middleware management server. Members, in the preparatory work before cluster switching, add the second middleware management server to the server group; The first middleware management server is configured to be the master and slave databases of the first database cluster; The second middleware management server is configured as the master database of the first database cluster and the slave database of the second database cluster; In addition to making all instances mounted on the first middleware management server publicly accessible, a portion of the instances mounted on the second middleware management server will also be made publicly accessible. Verify the connectivity between the second middleware management server and the first and second database clusters; Through data synchronization tools, the upstream connection is made to the slave database of the first database cluster, and the downstream connection is made to the master database of the second database cluster, so as to achieve real-time synchronization between the first database cluster and the second database cluster. (2) Switching phase First, disable all the second middleware instances mounted on the second middleware management server. Then, update the configuration of the second database middleware group on the second middleware management server to the master and slave databases of the second database cluster. Finally, set the master database of the second database cluster to write-prohibited mode. Pre-switch checks:

1. Check if the master database of the second database cluster is in a write-prohibited state; 2. Check if there is a data delay between the master and slave databases of the first database cluster; 3. Check if the slave databases of the second database cluster have permissions to the data synchronization tool. Cluster switching can only be performed after the above checks meet the requirements. Initiating the switchover: First, set the primary database of the first database cluster to write-prohibited mode and begin the switchover. Second, confirm whether the data in the primary database of the first database cluster before and after the write-prohibited moment is fully synchronized to the second database cluster. Operate the migration heartbeat table of the first database cluster, write a verification data to the migration heartbeat table, and check the migration heartbeat table in the second database cluster to see if the written verification data has been synchronized to the migration heartbeat table of the second database cluster. If so, the data-level switchover is complete, and the middleware switchover continues; otherwise, wait for the data to be seen before proceeding with the middleware switchover. Third, update the middleware management server group to only have the second database middleware group, and simultaneously, open all instances mounted on the second database middleware group to provide services externally. Finally, enable the write function of the primary database of the second database cluster and check whether the read and write traffic is normal. (3) Post-switching stage If the above operation process is normal, reverse data synchronization is performed through the data synchronization tool. The upstream connects to the slave database of the second database cluster, and the downstream connects to the master database of the first database cluster to perform real-time data synchronization. After the observation period, the full service is normal, and the first database cluster and the second database cluster are isolated and become two independent clusters. (4) Rollback phase During rollback, firstly, the write function of the primary database of the second database cluster is disabled; secondly, data synchronization from the second database cluster to the first database cluster is disabled, and it is confirmed whether the data in the primary database of the second database cluster at the write-disabled time and before that time is fully synchronized to the second database cluster. The migration heartbeat table of the second database cluster is manipulated, and a verification data is written to the migration heartbeat table. The migration heartbeat table of the first database cluster is checked to see if the written verification data has been synchronized to the migration heartbeat table of the first database cluster. If so, the data-level rollback is completed, and the middleware rollback continues; otherwise, the middleware rollback is performed only after the data is seen.

2. The database cluster switching system according to claim 1, characterized in that: The master and slave databases are MySQL databases.

3. The database cluster switching system according to claim 1, characterized in that: The first middleware management server and the second middleware management server are used to receive service requests from the upper layer and send them to their respective database middleware groups. The database middleware groups are used to receive service requests and send them to the lower-level database cluster.

4. The database cluster switching system according to claim 3, characterized in that: The database middleware group includes at least one database middleware, which is used to configure the master and slave databases.

5. The database cluster switching system according to claim 3, characterized in that: Real-time data synchronization between the first and second database clusters.

6. The database cluster switching system according to claim 1, characterized in that: The first database middleware group has the same configuration, which configures the master and slave databases of the first database cluster; the second database middleware group has the same configuration, which configures the master database of the first database cluster and the slave database of the second database cluster.

7. The database cluster switching system according to claim 1, characterized in that: After the switch is complete, observe the database middleware logs to check if the read and write traffic is normal. If it is normal, the switch is successful; if it is abnormal, proceed to the rollback phase.

8. The database cluster switching system according to claim 7, characterized in that: During the rollback, first disable the write function of the primary database of the second database cluster and disable data synchronization. Second, update the middleware management server group to have only one member, the first middleware management server. At the same time, roll back the database cluster architecture to the original architecture before the switch and check if the traffic is normal.

9. A middleware-based database cluster switching method, said method being applied to the database cluster switching system as described in any one of claims 1-8, characterized in that... The method includes: during the switch, firstly, completely disabling the second database middleware group; updating the configuration of the second database middleware group to the master and slave databases of the second database cluster, and setting the master database of the second database cluster to a write-disabled state; then updating the middleware management server group to have only one member, the second middleware management server; enabling the write function of the master database of the second database cluster, and using a data synchronization tool to connect upstream to the slave database of the first database cluster and downstream to the master database of the second database cluster for real-time data synchronization.

10. A computer-readable storage device, characterized in that, The system stores processor-executable instructions that, when executed by the processor, cause the processor to perform the method according to claim 9.

Images