Database expansion methods, devices, computer equipment and storage media
By introducing auxiliary virtual IPs and rule mapping tables in database expansion, traffic isolation and high availability are achieved during the database expansion process. This solves the problems of interruption and complex modification caused by downtime migration in existing technologies and simplifies the expansion process.
Patent Information
- Application Number
- CN202611224654.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-12
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies for database expansion employ a shutdown migration solution, which suffers from long service interruption times and complex operation and maintenance modifications, making it difficult to meet the requirements of high-availability business scenarios.
By adding an auxiliary virtual IP that is different from the original virtual IP, traffic isolation is achieved, a rule mapping table is built to ensure compatibility of fragmented routing rules, and paired expansion database instances are formed into a high-availability group. Traffic switching is completed using the auxiliary virtual IP, simplifying the expansion process.
This achieved isolation between expansion traffic and existing business traffic, reduced the impact of expansion operations on online business, ensured high availability of database services, and simplified the workload of the transformation.
Smart Images

Figure CN123115082A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a database expansion method, apparatus, computer equipment, and storage medium. Background Technology
[0002] With the rapid development of internet services, the volume of business data and access volume are surging, making it difficult for traditional single-database architectures to handle the ever-increasing read and write pressure. Currently, horizontal database sharding is commonly used, distributing business data across multiple database instances according to sharding rules to improve the overall cluster processing capacity. When the existing database cluster capacity reaches its performance bottleneck and horizontal scaling is required, conventional solutions often employ a shutdown migration mode: stopping external business services, deploying new database instances, performing a full data migration, modifying the application-side sharding configuration, and then restarting the service. This approach not only results in a long business service interruption time, affecting business continuity, but also requires changes to the upper-layer application routing configuration during the scaling process, leading to complex operational steps, high maintenance risks, and difficulty in meeting the requirements of high-availability business scenarios. Summary of the Invention
[0003] This invention provides a database expansion method, apparatus, computer equipment, and storage medium to solve the technical problems of long service interruption time and complex operation and maintenance in the existing horizontal sharded database expansion scheme, which adopts a shutdown migration scheme.
[0004] Firstly, a database expansion method is provided, including: Associate the original database instance with a secondary virtual IP, wherein the secondary virtual IP refers to a virtual IP set in addition to the original virtual IP corresponding to the original database instance; The initial sharding routing rule corresponding to the original database instance is expanded into a target sharding routing rule, and a rule mapping table between the initial sharding routing rule and the target sharding routing rule is established; Load the target sharding routing rule, perform sharding addressing according to the target sharding routing rule and the rule mapping table, and forward the data request to the auxiliary virtual IP associated with the target database instance; the target database entity is the database instance in the original database instance that receives the data request; Obtain all expanded database instances, pair the expanded database instances with the original database instances to obtain the target high availability group; The auxiliary virtual IP is unbound from the original database instance and associated with the database instance in the target high availability group to complete the database expansion.
[0005] Secondly, a database expansion device is provided, comprising: An auxiliary association module is used to associate the original database instance with an auxiliary virtual IP, wherein the auxiliary virtual IP refers to a virtual IP set in addition to the original virtual IP corresponding to the original database instance. The rule mapping module is used to expand the initial sharding routing rule corresponding to the original database instance into a target sharding routing rule, and to establish a rule mapping relationship table between the initial sharding routing rule and the target sharding routing rule; The request forwarding module is used to load the target sharding routing rule, perform sharding addressing according to the target sharding routing rule and the rule mapping table, and forward the data request to the auxiliary virtual IP associated with the target database instance; the target database entity is the database instance in the original database instance that receives the data request; The instance pairing module is used to obtain all expanded database instances, pair the expanded database instances with the original database instances, and obtain the target high availability group; The database expansion module is used to unbind the auxiliary virtual IP from the original database instance and associate it with the database instance in the target high availability group to complete the database expansion.
[0006] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described database expansion method.
[0007] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described database expansion method.
[0008] The aforementioned database expansion method, apparatus, computer equipment, and storage medium, wherein the database expansion method, by adding an auxiliary virtual IP distinct from the original virtual IP, enables the auxiliary virtual IP to carry sharding expansion-related traffic, achieving isolation between expansion traffic and existing service traffic, and reducing the impact of expansion operations on online services. By constructing a rule mapping table, compatibility between new and old sharding routing rules is achieved, enabling a smooth transition of sharding addressing logic. Pairing the expansion database instance with the original database instance to form a target high-availability group ensures that fault-tolerant capabilities remain after expansion, guaranteeing high availability of the database service. By reusing the same auxiliary virtual IP to complete traffic switching, traffic migration can be completed by only changing the virtual IP binding object, simplifying the expansion implementation process. Traffic switching is achieved based on virtual IP unbinding and rebinding, reducing the workload of modification. Attached Figure Description
[0009] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0010] Figure 1 This is a schematic diagram of an application environment for a database expansion method according to an embodiment of the present invention; Figure 2 This is a schematic flowchart of a database expansion method in one embodiment of the present invention; Figure 3 This is a flowchart illustrating step S103 of the database expansion method in one embodiment of the present invention; Figure 4 This is a schematic diagram of a database expansion device in one embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. Detailed Implementation
[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0012] The database expansion method provided in this embodiment of the invention can be applied to, for example... Figure 1In this application environment, the client communicates with the server via a network. The server can associate the original database instance with an auxiliary virtual IP, which is a virtual IP set up in addition to the original virtual IP corresponding to the original database instance; expand the initial sharding routing rule corresponding to the original database instance into a target sharding routing rule, and establish a rule mapping table between the initial sharding routing rule and the target sharding routing rule; load the target sharding routing rule, perform sharding addressing according to the target sharding routing rule and the rule mapping table, and forward data requests to the auxiliary virtual IP associated with the target database instance; the target database entity is the database instance in the original database instance that receives the data request; obtain all expanded database instances, pair the expanded database instances with the original database instances to obtain a target high-availability group; unbind the auxiliary virtual IP from the original database instance and associate it with the database instance in the target high-availability group to complete the database expansion. In this invention, by adding an auxiliary virtual IP distinct from the original virtual IP, the auxiliary virtual IP carries traffic related to sharding expansion, isolating expansion traffic from existing service traffic and reducing the impact of expansion operations on online services. By constructing a rule mapping table, compatibility between old and new sharding routing rules is achieved, enabling a smooth transition of sharding addressing logic. Pairing the expansion database instance with the original database instance to form a target high-availability group ensures fault-tolerant capability after expansion, guaranteeing high availability of the database service. Reusing the same auxiliary virtual IP for traffic switching simplifies the expansion implementation process by requiring only changes to the virtual IP binding object. Traffic switching based on virtual IP unbinding and rebinding reduces the workload of modifications. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.
[0013] Please see Figure 2 As shown, Figure 2 A schematic flowchart of a database expansion method provided in an embodiment of the present invention includes the following steps: S101. Associate the original database instance with the auxiliary virtual IP, wherein the auxiliary virtual IP refers to a virtual IP set in addition to the original virtual IP corresponding to the original database instance.
[0014] Understandably, the auxiliary virtual IP refers to a virtual IP set up in addition to the original virtual IP corresponding to the original database instance. The original virtual IP refers to the virtual IP used for normal access to the original database instance. A virtual IP is a logical network address. The original database instance refers to the database instance currently carrying business read / write operations and bound to the original virtual IP.
[0015] Specifically, all original database instances are retrieved, and a secondary virtual IP is set in addition to the original virtual IP of each original database instance. Each original database instance is then associated with its corresponding secondary virtual IP. For example, database instance A is bound to the original virtual IP 0, and simultaneously bound to the secondary virtual IP 00.
[0016] S102. Expand the initial sharding routing rule corresponding to the original database instance into a target sharding routing rule, and establish a rule mapping table between the initial sharding routing rule and the target sharding routing rule.
[0017] Understandably, the initial fragmentation routing rule refers to the fragmentation routing algorithm used to calculate the data's attribution fragment before expansion. The target fragmentation routing rule refers to the new fragmentation routing algorithm used to calculate the data's attribution fragment after expansion. The rule mapping table is a data table that records the correspondence between the initial fragmentation routing rule and the target fragmentation routing rule, used to achieve compatible addressing between the old and new fragmentation rules during the expansion transition period.
[0018] Specifically, the initial sharding routing rules corresponding to each original database instance are obtained. These initial sharding routing rules are then expanded from the current mode to the target mode, thus obtaining the target sharding routing rules. In one embodiment, the current mode is uid % N, and the target mode is uid % 2N. Expanding uid % N to uid % 2N expands the initial sharding routing rules into target sharding routing rules. Then, a rule mapping table is established between the initial and target sharding routing rules. In one embodiment, the original route with uid % 2 = 0 is mapped to new routes with uid % 4 = 0 and uid % 4 = 2, and both new rules initially point to the secondary virtual IP00 of the original database instance. This ensures that requests initiated according to the new rules can correctly access the original database instance containing all its data.
[0019] S103. Load the target fragmentation routing rule, perform fragmentation addressing according to the target fragmentation routing rule and the rule mapping table, and forward the data request to the auxiliary virtual IP associated with the target database instance; the target database entity is the database instance in the original database instance that receives the data request.
[0020] In essence, a data request refers to a database read / write operation request initiated by a business application. The target database instance refers to the database instance that, during the scaling process, is determined by the target sharding routing rules to handle this data request. Shard addressing refers to the process of calculating and determining, based on sharding routing rules, which determines the specific database instance to which the data request needs to be routed.
[0021] Specifically, the target shard routing rules are loaded by restarting the service or dynamically pushing them through the configuration center. Then, shard addressing is performed according to the target shard routing rules and the rule mapping table, forwarding the data request to the secondary virtual IP associated with the target database instance. That is, after receiving the data request, the shard key is extracted from the data request, and the target shard routing rules are used to calculate the shard key to obtain the target shard identifier. The rule mapping table is then queried to match the database access entry corresponding to the target shard identifier. Based on the matching result, the target database instance that will receive the data request is determined, the secondary virtual IP associated with the target database instance is read, and the data request is forwarded to the secondary virtual IP. The data is then routed to the corresponding target database instance via the secondary virtual IP to perform read and write operations.
[0022] S104. Obtain all expanded database instances, pair the expanded database instances with the original database instances, and obtain the target high availability group.
[0023] In other words, scaling up a database instance refers to deploying a new database instance to pair with the original database instance. A target high-availability group is a high-availability logical management unit consisting of the paired original database instance and the scaled-up database instance.
[0024] Specifically, select the deployed expanded database instance, pair the expanded database instance with the original database instance, and determine the master and slave roles of the original database instance and the expanded database instance. Then, establish a data synchronization channel between the original database instance and the expanded database instance. The original database instance and the expanded database instance that have been paired and established a data synchronization channel are assigned to the same logical group to obtain the target high availability group.
[0025] S105. Unbind the auxiliary virtual IP from the original database instance and associate it with the database instance in the target high availability group to complete the database expansion.
[0026] Understandably, the database instances in the target high availability group include the original database instance and the scaled-up database instance.
[0027] Specifically, the auxiliary virtual IP is disconnected from its original network binding, and a new database instance is selected within the target high-availability group to host the service. This means the auxiliary virtual IP is then bound to the expanded database instance within the target high-availability group. Next, it is verified that business data requests can access the database instance in the target high-availability group normally through the auxiliary virtual IP, and the master-slave data synchronization status in the target high-availability group is checked for stability. This confirms that sharded data can be read and written normally, thus completing this database sharding expansion.
[0028] In one embodiment, within the financial sector, the original database instance carrying existing transaction data is first associated with an auxiliary virtual IP. This auxiliary virtual IP is a newly added, independent virtual IP, distinct from the original virtual IP used for original financial business access, and is used to carry expansion and migration traffic. For the initial sharding routing rules based on user ID modulo for financial user accounts and transaction flows, these rules are smoothly extended to target sharding routing rules, and a rule mapping table between the initial and target sharding routing rules is constructed. After the business system loads the target sharding routing rules, it performs precise sharding addressing based on the target sharding routing rules and the rule mapping table, accurately forwarding new financial transactions, account queries, fund changes, and other data requests to the target database instance bound to the auxiliary virtual IP. Furthermore, a pre-deployed expansion database instance is acquired, and paired with the original database instance carrying core financial data to form a target high-availability group, within which a real-time data synchronization mechanism is established. Finally, the auxiliary virtual IP is unbound from the original database instance and reassigned to the database instance in the target high-availability group, completing the sharding expansion of the financial business database.
[0029] In one embodiment, in a medical database expansion scenario within the healthcare field, the original database instance storing patient electronic medical records and treatment records is first associated with an auxiliary virtual IP. This auxiliary virtual IP is independent of the original virtual IP used for daily hospital business access and is used to carry patient data traffic during expansion migration. Based on the existing initial sharding routing rules based on patient IDs, expanded target sharding routing rules are obtained, and a mapping table between the old and new rules is established. The medical business system loads the target sharding routing rules and, according to the rule mapping table, completes precise sharding addressing of patient data, smoothly forwarding data requests such as patient registration, medical record updates, report queries, and treatment record writing to the target database instance bound to the auxiliary virtual IP. The deployed expansion database instance is retrieved and paired with the original database instance storing core medical data to construct a target high-availability group. Finally, the auxiliary virtual IP instance is unbound from the high-availability group and re-associated, completing the medical database sharding expansion.
[0030] The database expansion method in this embodiment of the invention achieves this by adding an auxiliary virtual IP, distinct from the original virtual IP, enabling the auxiliary virtual IP to carry sharding expansion-related traffic. This isolates expansion traffic from existing service traffic, reducing the impact of expansion operations on online services. By constructing a rule mapping table, compatibility between old and new sharding routing rules is achieved, ensuring a smooth transition of sharding addressing logic. Pairing the expansion database instance with the original database instance to form a target high-availability group ensures continued fault-tolerant capability after expansion, guaranteeing high availability of the database service. Reusing the same auxiliary virtual IP for traffic switching simplifies the expansion implementation process by requiring only a change in the virtual IP binding object. Traffic switching based on virtual IP unbinding and rebinding reduces the workload of the modification process.
[0031] In one embodiment, step S102, expanding the initial sharding routing rule corresponding to the original database instance into a target sharding routing rule, includes: S1021. Based on the initial fragmentation routing rules, determine the initial fragmentation judgment conditions, data coverage range, and initial routing association information.
[0032] S1022, according to the expansion and splitting requirements, the data coverage area corresponding to the initial sharding routing rule is divided into multiple non-overlapping sub-intervals, and the target sharding determination conditions are determined based on all the sub-intervals.
[0033] S1023, determine the target route association information according to the fragmentation identifier corresponding to each of the sub-intervals, and construct the target fragmentation routing rules according to the target fragmentation determination conditions and the target route association information.
[0034] Understandably, the initial fragmentation determination condition refers to the computational conditions used to determine which fragmentation key belongs to which fragmentation key when using initial fragmentation routing rules. Data coverage refers to the range of values for the fragmentation key corresponding to a single fragment; all data within this range is carried by this fragment. Initial routing association information refers to the configuration information recording the binding relationship between the initial fragmentation identifier and the corresponding database instance and virtual IP. Expansion / splitting requirements refer to the business expansion requirement of splitting the original fragments into multiple new fragments. Target fragmentation determination condition refers to the new fragmentation computational conditions used after expansion. Fragmentation identifier refers to the identifier calculated through fragmentation routing rules, used to uniquely distinguish different fragments. Target routing association information refers to the configuration information recording the binding relationship between the target fragmentation identifier and the corresponding database instance and virtual IP.
[0035] Specifically, the initial sharding routing rules are parsed to determine the initial sharding criteria, the data coverage of each shard, and the initial routing association information. Then, based on the scaling requirements, the data coverage of the initial sharding routing rules is divided into intervals, creating multiple independent and non-overlapping sub-data intervals. Next, based on all the sub-data intervals obtained from the division, the target sharding criteria for matching the scaling scenario are redefined. Finally, based on the unique sharding identifier corresponding to each sub-data interval, the binding relationship between the scaled-up shards and the database instance is matched and configured to generate target routing association information. Furthermore, based on the determined target sharding criteria and target routing association information, the target sharding routing rules are constructed.
[0036] In this embodiment, the data coverage area is divided into multiple non-overlapping sub-intervals, achieving accurate splitting of the initial single data fragments. By matching the fragment identifiers of the sub-intervals to generate target routing association information, precise adaptation of the new fragment routing rules to the expanded database instance is achieved, ensuring the accuracy of fragment addressing. A progressive expansion logic is adopted to achieve a smooth transition between the old and new fragment rules, ensuring the continuity and stability of the database service.
[0037] In one embodiment, such as Figure 3 As shown, in step S102, establishing a rule mapping table between the initial sharding routing rule and the target sharding routing rule includes: S1024, Determine the initial fragment identifier according to the initial fragmentation routing rules.
[0038] S1025, determine the identifier of each target fragment according to the target fragment routing rules.
[0039] S1026, Based on the initial fragment identifier in the initial fragment routing rule and the target fragment identifier in the target fragment routing rule, determine the identifier correspondence between the initial fragment identifier and the target fragment identifier.
[0040] S1027, construct a rule mapping relationship table based on all the identification correspondences corresponding to the same user identifier.
[0041] Understandably, the initial shard identifier is a tag used to uniquely identify the shard before expansion. The target shard identifier is a tag used to uniquely identify the new shard after expansion. The identifier mapping relationship refers to the mapping information that records the correspondence between the initial shard identifier and one or more target shard identifiers generated by the split.
[0042] Specifically, all user identifiers are traversed, and the initial sharding routing rules are applied sequentially to obtain the initial sharding identifier for each user identifier. Similarly, for all identical user identifiers, the target sharding routing rules are applied sequentially to obtain the target sharding identifier for each user identifier. Then, for any given user identifier, its calculated initial sharding identifier is bound to the target sharding identifier, establishing an identifier mapping relationship between the initial and target sharding identifiers. All identifier mapping relationships for all user identifiers are collected, and duplicate mapping relationships are deduplicated to obtain a rule mapping relationship table.
[0043] In this embodiment, the shard identifiers before and after the expansion are associated with the same user identifier, an identifier correspondence is established and a rule mapping table is generated, which realizes the accurate conversion between the initial shard routing rules and the target shard routing rules, ensures the accuracy of shard addressing during the expansion process, and realizes uninterrupted expansion of database shards.
[0044] In one embodiment, step S103, which involves performing fragmentation addressing based on the target fragmentation routing rules and the rule mapping table, and forwarding the data request to the auxiliary virtual IP associated with the target database instance, includes: S1031, perform fragmentation addressing according to the target fragmentation routing rules and the rule mapping relationship table to determine the logical database address.
[0045] S1032, Determine the corresponding target database instance based on the logical database address.
[0046] S1033, based on the auxiliary virtual IP associated with the target database instance, forward the data request to the auxiliary virtual IP.
[0047] Understandably, a logical database address refers to a logical addressing identifier used to identify the logical access unit corresponding to a shard and to establish the association between the shard identifier and the database instance.
[0048] Specifically, upon receiving a data request, the sharding key is first extracted from the request. Then, the sharding key is processed according to the target sharding routing rules to obtain the sharding information to which the data request belongs. This information is then matched and verified using a rule mapping table to obtain the corresponding logical database address. Next, based on the parsed logical database address, the target database instance corresponding to that address is matched. Then, the auxiliary virtual IP associated with the target database instance is retrieved, and the data request is forwarded to this auxiliary virtual IP. The auxiliary virtual IP then performs network scheduling, distributing the request to the corresponding target database instance for read / write operations.
[0049] In this embodiment, the logical database address is used as the intermediate addressing carrier, eliminating the need to modify business code and access configuration, thus reducing the difficulty of database expansion. The auxiliary virtual IP handles the fragmented requests after expansion, completely isolating them from the original virtual IP traffic before expansion, preventing interference between old and new business traffic and ensuring business continuity.
[0050] In one embodiment, step S104, which involves obtaining all expanded database instances and pairing the expanded database instances with the original database instances to obtain the target high-availability group, includes: S1041 Select the deployed expansion database instance and establish a pairing relationship between the expansion database instance and the original database instance.
[0051] S1042, Establish a data synchronization channel between the original database instance and the expanded database instance.
[0052] S1043, the original database instance and the expanded database instance that have been paired and established a data synchronization channel are assigned to the same logical group to obtain the target high availability group.
[0053] In essence, a pairing relationship refers to a one-to-one association between the original database instance and the expanded database instance. A data synchronization channel is a data transmission link between the master and slave instances, used for real-time synchronization of incremental data from the master instance to the slave instance. It specifies one party in the pairing relationship as the master instance and the other as the slave instance.
[0054] Specifically, the deployed and expanded database instances are selected, and a one-to-one pairing relationship is established between the selected expanded database instances and the original database instances carrying existing services. One instance is selected as the master instance, responsible for handling data read and write requests; the other instance is set as a slave instance, used to synchronize data with the master instance and prepare for failover. Then, a data synchronization channel is established between the original and expanded database instances. This involves configuring communication parameters between the master and slave instances to establish a bidirectional data synchronization channel, configuring synchronization rules, enabling change log collection for the master instance, and having the slave instance pull data changes from the master instance. Finally, the paired and synchronized database instances are incorporated into the same logical management unit and assigned a unique group identifier to obtain the target high-availability group.
[0055] In this embodiment, by pairing the expanded database instance with the original database instance as a master-slave and establishing a data synchronization channel, data consistency between the two sets of instances is ensured, avoiding data inconsistency issues during the expansion process. By dividing logical groups to form target high-availability groups, the paired master and slave instances are managed in a unified manner, enabling support for the switching of auxiliary virtual IPs within the group.
[0056] In one embodiment, after step S105, after unbinding the auxiliary virtual IP from the original database instance and associating it with the database instance in the target high availability group, the method further includes: S1051, based on the configuration information of the initial high availability group, determine the initial master instance and initial slave instance in each initial high availability group, and stop data replication from the initial master instance to the initial slave instance.
[0057] S1052, based on the target sharding routing rules, determine the pairing combination of the update master instance and the update slave instance, and configure the synchronization output parameters of the update master instance and the synchronization access parameters of the update slave instance.
[0058] S1053, Based on the synchronization output parameters and the synchronization access parameters, establish a data synchronization channel and start real-time data synchronization from the update master instance to the paired update slave instance.
[0059] Understandably, the initial high-availability group refers to the logically high-availability group that provides business access before scaling. Configuration information refers to a set of parameters recording high-availability groups, database instance roles, virtual IP associations, etc. The initial master instance refers to the database instance that handles business read and write requests before scaling. The initial slave instance refers to the database instance that synchronizes data with the initial master instance before scaling. The updated master instance refers to the newly designated database instance that handles read and write services after scaling. The updated slave instance refers to the database instance that synchronizes data with the updated master instance after scaling. Synchronization output parameters refer to the external communication parameters deployed on the master instance side for slave instances to establish synchronization connections. Synchronization access parameters refer to the connection configuration parameters deployed on the slave instance side for connecting to the master instance.
[0060] Specifically, the configuration information corresponding to the initial high-availability group is obtained, and the initial master instance and initial slave instance corresponding to each initial high-availability group are identified based on the configuration information. Then, a synchronization stop command is issued to shut down data replication from the initial master instance to the initial slave instance and to release the master-slave binding relationship between the original instances. Next, based on the sharding data distribution and instance resource deployment in the target sharding routing rules, the pairing combination of update master instances and update slave instances is replanned and determined, completing the allocation of new instance roles after expansion. Then, for the determined update master instance, the corresponding synchronization output parameters are configured; simultaneously, matching synchronization access parameters are configured on the paired update slave instance side to ensure that the synchronization parameters of the update master instance and the update slave instance are compatible. Finally, after the synchronization output parameters and synchronization access parameters are configured and verified to be correct, a data synchronization channel from the update master instance to the update slave instance is established through the synchronization output parameters and synchronization access parameters, completing the initialization of the master-slave communication link. A real-time data synchronization mechanism is started, with the update master instance pushing incremental data changes in real time, and the update slave instance continuously pulling and replaying data.
[0061] In this embodiment, accurate identification of the initial master instance and initial slave instance is achieved through configuration information, thereby stopping data replication and avoiding data duplication caused by the coexistence of old and new synchronization links. By re-matching the pairing of the update master instance and update slave instance, master-slave pairing adaptation for expanded sharded data is achieved. By configuring the synchronization output parameters of the update master instance and the synchronization access parameters of the update slave instance respectively, master-slave synchronization parameter adaptation is achieved, ensuring the stability and effectiveness of the data synchronization channel.
[0062] In one embodiment, after step S1053, after initiating real-time data synchronization from the primary update instance to the paired secondary update instance, the method further includes: S1054, Traverse the data records of all data tables in the target database instance, where the target database instance includes the original database instance and the expanded database instance.
[0063] S1055, extract the user identifier field from each data record, and perform sharding determination according to the target sharding routing rules of each target database instance, and filter out and delete data rows whose determination results do not belong to the sharding range corresponding to the target database instance.
[0064] Understandably, the target database instance includes both the original database instance and the expanded database instance. A data record refers to a complete piece of business data stored in the database, composed of multiple fields. The user identifier field is a field in the data table that stores a unique identifier for the user, used for sharding routing calculation. The sharding range refers to the range of user identifier values defined by the sharding routing rules; all data within this range belongs to the same shard. A data row refers to a single row of data stored in the data table; one data row corresponds to one data record.
[0065] Specifically, the target database instance to be cleaned is determined, including both the original and expanded database instances. Then, all data tables within the target database instance are accessed, and all data records in each table are traversed. For each data record obtained, the user identifier field, which serves as the sharding key, is extracted. Using the target sharding routing rules corresponding to the current target database instance, a sharding determination operation is performed on the extracted user identifier field to obtain the corresponding sharding information. Next, the sharding determination result is compared with the legal sharding range corresponding to the target database instance, filtering out data rows whose sharding belongs to a shard outside the current instance's sharding range. Finally, data rows that do not belong to the instance's sharding range are cleaned up, and the above extraction, determination, filtering, and deletion steps are repeated until all data records in all tables have been cleaned up.
[0066] In one embodiment, the instance corresponding to the original virtual IP0 retains only the data with uid % 4 = 0, and the new instance corresponding to the auxiliary virtual IP00 retains only the data with uid % 4 = 2.
[0067] In this embodiment, by traversing the data tables in each target database instance, the data records of each target database instance are obtained, thereby achieving accurate identification of the user identification field and thus enabling the deletion of data rows that do not belong to the target database instance.
[0068] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0069] In one embodiment, a database expansion device is provided, which corresponds one-to-one with the database expansion methods described in the above embodiments. For example... Figure 4 As shown, the database expansion device includes an auxiliary association module 101, a rule mapping module 102, a request forwarding module 103, an instance pairing module 104, and a database expansion module 105. Detailed descriptions of each functional module are as follows: The auxiliary association module 101 is used to associate the original database instance with an auxiliary virtual IP, wherein the auxiliary virtual IP refers to a virtual IP set in addition to the original virtual IP corresponding to the original database instance. The rule mapping module 102 is used to expand the initial sharding routing rule corresponding to the original database instance into a target sharding routing rule, and to establish a rule mapping relationship table between the initial sharding routing rule and the target sharding routing rule; The request forwarding module 103 is used to load the target fragmentation routing rule, perform fragmentation addressing according to the target fragmentation routing rule and the rule mapping relationship table, and forward the data request to the auxiliary virtual IP associated with the target database instance; the target database entity is the database instance in the original database instance that receives the data request; The instance pairing module 104 is used to obtain all expanded database instances, pair the expanded database instances with the original database instances, and obtain the target high availability group. The database expansion module 105 is used to unbind the auxiliary virtual IP from the original database instance and associate it with the database instance in the target high availability group to complete the database expansion.
[0070] In one embodiment, the rule mapping module 102 includes: The rule parsing unit is used to determine the initial fragmentation judgment conditions, data coverage range, and initial route association information based on the initial fragmentation routing rules. The interval partitioning unit is used to divide the data coverage area corresponding to the initial fragmentation routing rule into multiple non-overlapping sub-intervals according to the expansion and splitting requirements, and to determine the target fragmentation judgment conditions based on all the sub-intervals. The rule construction unit is used to determine the target route association information based on the fragmentation identifier corresponding to each of the sub-intervals, and to construct the target fragmentation routing rules based on the target fragmentation determination conditions and the target route association information.
[0071] In one embodiment, the rule mapping module 102 further includes: An initial fragment identifier unit is used to determine each initial fragment identifier according to the initial fragment routing rules; The target fragment identifier unit is used to determine the identifier of each target fragment according to the target fragment routing rules; The identification correspondence unit is used to determine the identification correspondence between the initial fragment identifier and the target fragment identifier based on the initial fragment identifier in the initial fragment routing rule and the target fragment identifier in the target fragment routing rule, which are the same user identifiers; The relationship table construction unit is used to construct a rule mapping relationship table based on all the identification correspondences corresponding to the same user identifier.
[0072] In one embodiment, the request forwarding module 103 includes: The logical database address unit is used to perform fragmentation addressing based on the target fragmentation routing rules and the rule mapping table, and to determine the logical database address. The target database instance unit is used to determine the corresponding target database instance based on the logical database address; The data request forwarding unit is used to forward data requests to the auxiliary virtual IP associated with the target database instance.
[0073] In one embodiment, the instance pairing module 104 includes: The pairing relationship unit is used to select the deployed expanded database instance and establish a pairing relationship between the expanded database instance and the original database instance; The data synchronization channel unit is used to establish a data synchronization channel between the original database instance and the expanded database instance. The logical grouping unit is used to group the original database instance and the expanded database instance that have been paired and established a data synchronization channel into the same logical group to obtain the target high availability group.
[0074] In one embodiment, the device further includes: The data replication unit is used to determine the initial master instance and initial slave instance in each initial high availability group based on the configuration information of the initial high availability group, and to stop data replication from the initial master instance to the initial slave instance; The parameter configuration unit is used to determine the pairing combination of the update master instance and the update slave instance according to the target sharding routing rules, and to configure the synchronization output parameters of the update master instance and the synchronization access parameters of the update slave instance. The real-time data synchronization unit is used to establish a data synchronization channel based on the synchronization output parameters and the synchronization access parameters, and to initiate real-time data synchronization from the update master instance to the paired update slave instance.
[0075] In one embodiment, the device further includes: A data recording unit is used to traverse the data records of all data tables in the target database instance, which includes the original database instance and the expanded database instance. The data row deletion unit is used to extract the user identifier field from the data record one by one, and perform sharding determination according to the target sharding routing rules of each target database instance, and filter out and delete the data rows whose determination results do not belong to the sharding range corresponding to the target database instance.
[0076] Specific limitations regarding the database expansion device can be found in the limitations on the database expansion method described above, and will not be repeated here. Each module in the aforementioned database expansion device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.
[0077] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the client-side functions or steps of a database expansion method.
[0078] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Associate the original database instance with a secondary virtual IP, wherein the secondary virtual IP refers to a virtual IP set in addition to the original virtual IP corresponding to the original database instance; The initial sharding routing rule corresponding to the original database instance is expanded into a target sharding routing rule, and a rule mapping table between the initial sharding routing rule and the target sharding routing rule is established; Load the target fragmentation routing rule, perform fragmentation addressing according to the target fragmentation routing rule and the rule mapping table, and forward the data request to the auxiliary virtual IP associated with the target database instance; Obtain all expanded database instances, pair the expanded database instances with the original database instances to obtain the target high availability group; The auxiliary virtual IP is unbound from the original database instance and associated with the database instance in the target high availability group to complete the database expansion.
[0079] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Associate the original database instance with a secondary virtual IP, wherein the secondary virtual IP refers to a virtual IP set in addition to the original virtual IP corresponding to the original database instance; The initial sharding routing rule corresponding to the original database instance is expanded into a target sharding routing rule, and a rule mapping table between the initial sharding routing rule and the target sharding routing rule is established; Load the target fragmentation routing rule, perform fragmentation addressing according to the target fragmentation routing rule and the rule mapping table, and forward the data request to the auxiliary virtual IP associated with the target database instance; Obtain all expanded database instances, pair the expanded database instances with the original database instances to obtain the target high availability group; The auxiliary virtual IP is unbound from the original database instance and associated with the database instance in the target high availability group to complete the database expansion.
[0080] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.
[0081] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0082] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0083] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been authorized (with the knowledge and consent) by the relevant parties or has been fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.
[0084] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.
Claims
1. A database expansion method, characterized in that, include: Associate the original database instance with a secondary virtual IP, wherein the secondary virtual IP refers to a virtual IP set in addition to the original virtual IP corresponding to the original database instance; The initial sharding routing rule corresponding to the original database instance is expanded into a target sharding routing rule, and a rule mapping table between the initial sharding routing rule and the target sharding routing rule is established; Load the target sharding routing rule, perform sharding addressing according to the target sharding routing rule and the rule mapping table, and forward the data request to the auxiliary virtual IP associated with the target database instance; the target database entity is the database instance in the original database instance that receives the data request; Obtain all expanded database instances, pair the expanded database instances with the original database instances to obtain the target high availability group; The auxiliary virtual IP is unbound from the original database instance and associated with the database instance in the target high availability group to complete the database expansion.
2. The database expansion method as described in claim 1, characterized in that, The step of establishing a rule mapping table between the initial shard routing rule and the target shard routing rule includes: Based on the initial fragmentation routing rules, determine the initial fragmentation identifier; Based on the target fragment routing rules, determine the identifier of each target fragment; Based on the initial fragment identifier in the initial fragment routing rule and the target fragment identifier in the target fragment routing rule, determine the identifier correspondence between the initial fragment identifier and the target fragment identifier. A rule mapping table is constructed based on all the corresponding identifiers for the same user identifier.
3. The database expansion method as described in claim 1, characterized in that, The step of expanding the initial sharding routing rule corresponding to the original database instance into a target sharding routing rule includes: Based on the initial fragmentation routing rules, determine the initial fragmentation judgment conditions, data coverage range, and initial route association information; Based on the expansion and splitting requirements, the data coverage area corresponding to the initial sharding routing rule is divided into multiple non-overlapping sub-intervals, and the target sharding determination conditions are determined based on all the sub-intervals. The target route association information is determined based on the fragmentation identifier corresponding to each of the sub-intervals, and the target fragmentation routing rules are constructed based on the target fragmentation determination conditions and the target route association information.
4. The database expansion method as described in claim 1, characterized in that, The step of performing fragmentation addressing according to the target fragmentation routing rules and the rule mapping table, and forwarding the data request to the auxiliary virtual IP associated with the target database instance, includes: Fragmentation addressing is performed according to the target fragmentation routing rules and the rule mapping table to determine the logical database address; Based on the logical database address, determine the target database instance in the original database example; The data request is forwarded to the auxiliary virtual IP associated with the target database instance.
5. The database expansion method as described in claim 1, characterized in that, After unbinding the auxiliary virtual IP from the original database instance and associating it with the database instance in the target high availability group, the process further includes: Based on the configuration information of the initial high availability group, determine the initial master instance and initial slave instance in each initial high availability group, and stop data replication from the initial master instance to the initial slave instance; Based on the target sharding routing rules, determine the pairing combination of update master instance and update slave instance, and configure the synchronization output parameters of update master instance and the synchronization access parameters of update slave instance; Based on the synchronization output parameters and the synchronization access parameters, a data synchronization channel is established, and real-time data synchronization from the update master instance to the paired update slave instance is initiated.
6. The database expansion method as described in claim 5, characterized in that, After initiating the real-time data synchronization from the primary update instance to the paired update slave instance, the process also includes: Iterate through the data records of all tables in the target database instance, which includes the original database instance and the expanded database instance; Extract the user identifier field from each data record, and perform sharding determination according to the target sharding routing rules of each target database instance. Filter out and delete data rows whose determination results do not belong to the sharding range corresponding to the target database instance.
7. The database expansion method as described in claim 1, characterized in that, The process of obtaining all expanded database instances, pairing the expanded database instances with the original database instances to obtain the target high-availability group includes: Select the deployed expansion database instance and establish a pairing relationship between the expansion database instance and the original database instance; Establish a data synchronization channel between the original database instance and the expanded database instance; The original database instance and the expanded database instance that have been paired and established a data synchronization channel are grouped into the same logical group to obtain the target high availability group.
8. A database expansion device, characterized in that, include: An auxiliary association module is used to associate the original database instance with an auxiliary virtual IP, wherein the auxiliary virtual IP refers to a virtual IP set in addition to the original virtual IP corresponding to the original database instance. The rule mapping module is used to expand the initial sharding routing rule corresponding to the original database instance into a target sharding routing rule, and to establish a rule mapping relationship table between the initial sharding routing rule and the target sharding routing rule; The request forwarding module is used to load the target sharding routing rule, perform sharding addressing according to the target sharding routing rule and the rule mapping table, and forward the data request to the auxiliary virtual IP associated with the target database instance; the target database entity is the database instance in the original database instance that receives the data request; The instance pairing module is used to obtain all expanded database instances, pair the expanded database instances with the original database instances, and obtain the target high availability group; The database expansion module is used to unbind the auxiliary virtual IP from the original database instance and associate it with the database instance in the target high availability group to complete the database expansion.
9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the database expansion method as described in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the database expansion method as described in any one of claims 1 to 7.
