Partition splitting management method and apparatus applied to distributed database
By constructing the target partition within the consensus protocol group and sharing the log stream, the partitioning process of the distributed database is simplified, efficiency is improved, the complexity of the consensus protocol is reduced, and data consistency is guaranteed.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-09-05
- Publication Date
- 2026-03-12
AI Technical Summary
In distributed databases, partition splitting operations are complex and increase the complexity of consensus protocols, especially since partition splitting requires splitting the log stream of the partition, leading to inefficiency.
Multiple destination partitions are constructed within the consensus protocol group, and the data from the source partition is divided into these destination partitions while sharing the same log stream, thus avoiding log stream classification and simplifying the partition splitting process.
It improves the efficiency of partition splitting, reduces the complexity of consensus protocol groups, and ensures data consistency.
Smart Images

Figure CN2025119392_12032026_PF_FP_ABST
Abstract
Description
Partition splitting management method and device applied to distributed database TECHNICAL FIELD
[0001] The one or more embodiments of the present specification relate to the technical field of database, and particularly relate to a partition splitting management method and device applied to distributed database. BACKGROUND
[0002] In today's high-speed development of the Internet and informatization, the generation of data is growing explosively, so the requirements for database and its management are becoming higher and higher. The data tables in the database can be divided into different partitions to improve the efficiency of data management, such as the efficiency of data query; and as the continuous service of the business, the data volume of each partition becomes uneven, and a single hot partition can cause performance bottleneck of reading and writing, and a large number of small partitions can cause space fragmentation, at which time partition splitting and partition merging are usually performed to make the partitions on the table more uniform, thereby providing higher performance.
[0003] However, in the related art, the partition splitting also needs to split the log stream of the partition, that is, to construct a log stream for each target partition, and to transfer the logs in the log stream of the source partition that belong to different target partitions to the corresponding log stream, which makes the operation of partition splitting more complex, especially in the distributed database scenario, which also increases the complexity of the consensus protocol. SUMMARY
[0004] Therefore, the one or more embodiments of the present specification provide a partition splitting management method and device applied to distributed database.
[0005] To achieve the above object, the one or more embodiments of the present specification provide the technical solutions as follows.
[0006] According to a first aspect of the one or more embodiments of the present specification, a partition splitting management method applied to distributed database is provided, the method comprising:
[0007] receiving a splitting instruction for a source partition, and constructing a plurality of target partitions in a consensus protocol group in which the source partition is located, wherein the consensus protocol group comprises at least one database instance, each database instance has a log stream for recording data operation logs of all partitions in the database instance, and the data of the plurality of database instances in the consensus protocol group is the same and shares the same log stream;
[0008] dividing the data in the source partition to the plurality of target partitions according to the splitting instruction;
[0009] if all the data in the source partition is completed, deleting the source partition.
[0010] In one or more embodiments of the present specification, the receiving the split instruction for the source partition comprises:
[0011] The receiving the split instruction for the source partition comprises:
[0012] In one or more embodiments of the present specification, the receiving the split instruction for the source partition comprises:
[0013] The receiving the split instruction for the source partition comprises:
[0014] The receiving the split instruction for the source partition comprises:
[0015] In one or more embodiments of the present specification, the receiving the split instruction for the source partition comprises:
[0016] The receiving the split instruction for the source partition comprises:
[0017] In one or more embodiments of the present specification, the dividing the data in the source partition into the plurality of destination partitions according to the split instruction comprises:
[0018] The dividing the data in the source partition into the plurality of destination partitions according to the split instruction comprises:
[0019] In one or more embodiments of the present specification, the dividing the data in the source partition into the plurality of destination partitions according to the split instruction comprises:
[0020] The dividing the data in the source partition into the plurality of destination partitions according to the split instruction comprises:
[0021] In one or more embodiments of the present specification, the method further comprises:
[0022] receiving a data write instruction for the source partition, and writing data indicated by the data write instruction into a corresponding destination partition according to a mapping relationship between the data and the destination partition indicated in the split instruction.
[0023] In one or more embodiments of the present specification, the receiving a data write instruction for the source partition, and writing data indicated by the data write instruction into a corresponding destination partition according to a mapping relationship between the data and the destination partition indicated in the split instruction comprises:
[0024] receiving a data write instruction for the source partition, and writing data indicated by the data write instruction into a corresponding destination partition according to a mapping relationship between the data and the destination partition indicated in the split instruction.
[0025] In one or more embodiments of the present specification, the method further comprises:
[0026] if the data in the source partition is not completely divided, and a partition migration operation for the source partition is received, prohibiting the partition migration operation;
[0027] if the data in the source partition is not completely divided, and a partition migration operation for any destination partition is received, delaying the partition migration operation until the source partition is deleted.
[0028] According to a first aspect of one or more embodiments of the present specification, a partition split management device applied to a distributed database is provided, and the device comprises:
[0029] a construction module configured to receive a split instruction for a source partition, and construct a plurality of destination partitions in a consensus protocol group in which the source partition is located, wherein the consensus protocol group comprises a plurality of database instances, each of which has a log stream recording all transactions in the partitions, and the plurality of database instances in the consensus protocol group share the same log stream;
[0030] a division module configured to divide data in the source partition to the plurality of destination partitions according to the split instruction;
[0031] a deletion module configured to delete the source partition if all data in the source partition is completely divided.
[0032] In one or more embodiments of the present specification, the construction module is configured to:
[0033] receive a split instruction for a source partition, and construct a plurality of destination partitions in a consensus protocol group in which the source partition is located according to a number of destination partitions indicated in the split instruction.
[0034] In one or more embodiments of the present specification, the construction module is configured to:
[0035] receiving a split instruction for a source partition, constructing a plurality of destination partitions in a leader instance in a consensus protocol group in which the source partition is located, and writing an operation of constructing the plurality of destination partitions into a log stream;
[0036] sharing the log stream to other database instances in the consensus protocol group according to a consensus protocol of the consensus protocol group, so that the other database instances construct the plurality of destination partitions locally by replaying the log stream.
[0037] In one or more embodiments of the present specification, the constructing module is configured to:
[0038] if a split instruction for a source partition is received and there is no partition migration operation on the source partition, constructing a plurality of destination partitions in a consensus protocol group in which the source partition is located.
[0039] In one or more embodiments of the present specification, the dividing module is configured to:
[0040] According to the mapping relationship between the data and the destination partitions indicated in the split instruction, each data in the source partition is divided into the corresponding destination partition.
[0041] In one or more embodiments of the present specification, the dividing module is configured to:
[0042] flushing the data in the in-memory table of the source partition to the ordered string table of the source partition, and dividing the data in the ordered string table of the source partition to the plurality of destination partitions.
[0043] In one or more embodiments of the present specification, the apparatus further comprises a writing module configured to:
[0044] receiving a data write instruction for the source partition, and writing the data indicated by the data write instruction into the corresponding destination partition.
[0045] In one or more embodiments of the present specification, the writing module is configured to:
[0046] receiving a data write instruction for the source partition, and writing the data indicated by the data write instruction into the corresponding destination partition according to the mapping relationship between the data and the destination partitions indicated in the split instruction.
[0047] In one or more embodiments of the present specification, the apparatus further comprises a migration processing module configured to:
[0048] if the data in the source partition is not completely divided and a partition migration operation for the source partition is received, prohibiting the partition migration operation;
[0049] If the data in the source partition is not completely divided, and a partition migration operation for any target partition is received, the partition migration operation is delayed until the source partition is deleted.
[0050] According to a third aspect of one or more embodiments of the present specification, a computer program product is provided, comprising computer programs / instructions which, when executed by a processor, implement the steps of the method according to the first aspect.
[0051] According to a fourth aspect of one or more embodiments of the present specification, an electronic device is provided, comprising:
[0052] a processor;
[0053] a memory for storing processor-executable instructions;
[0054] wherein the processor implements the method according to the first aspect by executing the executable instructions.
[0055] According to a fifth aspect of one or more embodiments of the present specification, a computer-readable storage medium is provided, having stored thereon computer instructions which, when executed by a processor, implement the steps of the method according to the first aspect.
[0056] The technical solutions provided by the embodiments of the present specification can include the following beneficial effects:
[0057] In the partition splitting management method for a distributed database provided by the embodiments of the present specification, the consensus protocol group of the distributed database is composed of database instances rather than partitions, and each database instance in the consensus protocol group has a log stream for recording the transaction logs of all partitions therein, and the data of different database instances in the consensus protocol group is the same and shares the same log stream. When splitting a partition, the target partition is constructed in the same consensus protocol group, so that the source partition and the target partition belong to the same consensus protocol group, and the log streams of the source partition and the target partition belong to the same log stream, i.e. the log stream shared in the consensus protocol group. Therefore, the partition splitting no longer involves the classification of log streams, thereby improving the efficiency of partition splitting and reducing the complexity of the consensus protocol group regarding partition splitting. BRIEF DESCRIPTION OF DRAWINGS
[0058] FIG. 1 is a structure diagram of a distributed database according to an example embodiment.
[0059] FIG. 2 is a flowchart of a partition splitting management method for a distributed database according to an example embodiment.
[0060] FIG. 3 is a distributed database architecture diagram according to an example embodiment.
[0061] FIG. 4 is a structural schematic diagram of an apparatus according to an example embodiment.
[0062] FIG. 5 is a block diagram of a partition split management apparatus applied to a distributed database according to an example embodiment. DETAILED DESCRIPTION
[0063] The example embodiments will be described in detail herein with reference to the attached drawings. In the following description, unless otherwise indicated, like numbers in the attached drawings refer to the same or similar elements. The embodiments described in the following example embodiments are not representative of all embodiments consistent with one or more embodiments of the present description. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of the present description as detailed in the appended claims.
[0064] It should be noted that the steps of the corresponding method are not necessarily performed in the order shown and described in the present description in other embodiments. In some other embodiments, the steps included in the method can be more or less than described in the present description. In addition, a single step described in the present description can be divided into multiple steps for description in other embodiments, and multiple steps described in the present description can be combined into a single step for description in other embodiments.
[0065] In a distributed database in the related art, a data table has multiple partitions, and each partition has a consensus protocol group composed of different replicas of the partition. The consensus protocol group of the partition shares a log stream for recording all operations in the partition. The sharing of the log stream in the partition relies on the consensus protocol of the consensus protocol group of the partition. When a partition is split, the source partition is replaced by multiple target partitions, and the data in the source partition is transferred to the multiple target partitions for storage. The log stream of the source partition is split into multiple log streams corresponding to the multiple target partitions, which makes the process of partition splitting more complex. In addition, the consensus protocol needs to ensure data consistency during the process of partition splitting, which undoubtedly makes the consensus protocol very complex.
[0066] Based on this, at least one embodiment of the present description provides a partition split management method applied to a distributed database. The method can simplify the process of partition splitting in the distributed database, while avoiding the increase in complexity of the consensus protocol of the consensus protocol group due to partition splitting, thereby improving the efficiency of partition splitting and ensuring data consistency during the process of partition splitting.
[0067] The method can be applied to a distributed database based on an LSM-Tree storage engine as shown in FIG. 1, for example, to a leader instance of the distributed database. The distributed database includes at least one database instance.
[0068] Please refer to FIG. 2, which exemplarily shows the flow of the partition split management method applied to the distributed database, including steps S201 to S203.
[0069] In step S201, a split instruction for a source partition is received, and a plurality of destination partitions are constructed in a consensus protocol group in which the source partition is located, wherein the consensus protocol group includes at least one database instance, each database instance has a log stream for recording operation logs of all partitions in the database instance, and the data of the plurality of database instances in the consensus protocol group are the same and share the same log stream.
[0070] The consensus protocol group provides services such as electing a leader instance, sharing a log, etc. through a consensus protocol such as Paxos or Raft. The data of different database instances in the consensus protocol group are the same, and the log streams are shared. In a database instance, the data in a data table is divided into different partitions for storage, but the operation logs on all data in the data table are recorded in the same log stream, i.e. the log stream for recording operation logs of all partitions in the database instance. For example, the data table with a key range of [1, 200) in the database instance is divided into two partitions with a key range of [1, 100) and a key range of [101, 200) for storage, and the operation logs on all data of the data table are recorded in the log stream of the database instance with the key as an identifier, but the partition where the data is located is not identified.
[0071] Please refer to FIG. 3, which exemplarily shows the architecture of the distributed database to more intuitively show the relationship between the concepts of database instance, consensus protocol group, log stream, partition, etc. The distributed database in FIG. 3 has three availability zones, Zone1, Zone2 and Zone3; availability zone Zone1 has two database instances, instance 1 and instance 2, availability zone Zone2 has two database instances, instance 3 and instance 4, and availability zone Zone3 has two database instances, instance 5 and instance 6; instance 1, instance 3 and instance 5 form a Paxos group, each database instance in the Paxos group has partition A, partition B, and log stream 1 for recording operation logs on all data in partition A and partition B; instance 2, instance 4 and instance 6 form a Paxos group, each database instance in the Paxos group has partition C, partition D, and log stream 2 for recording operation logs on all data in partition C and partition D.
[0072] For example, the step can be performed in the following manner: receiving a split instruction for a source partition, constructing a plurality of destination partitions in a consensus protocol group in which the source partition is located according to a number of destination partitions indicated in the split instruction. For example, the split instruction indicates that the source partition is split into two destination partitions, and then two destination partitions are constructed in the consensus protocol group in which the source partition is located.
[0073] For example, the step can be performed in the following manner: receiving a split instruction for a source partition, constructing a plurality of destination partitions in a leader instance in a consensus protocol group in which the source partition is located, and writing an operation of constructing the plurality of destination partitions into a log stream; and then sharing the log stream to other database instances in the consensus protocol group according to a consensus protocol of the consensus protocol group, so that the other database instances construct the plurality of destination partitions locally by replaying the log stream.
[0074] For example, in the distributed database architecture shown in FIG. 3, instance 1 is elected as the leader instance in the Paxos group composed of instance 1, instance 3 and instance 5, and if a split instruction indicating that partition A is split into two partitions is received, two destination partitions can be constructed on instance 1, and instance 3 and instance 5 can also construct two destination partitions when replaying the log by the log sharing manner, and the naming of the two destination partitions is the same as that of the two destination partitions on instance 1.
[0075] For example, in the distributed database architecture shown in FIG. 3, instance 2 is elected as the leader instance in the Paxos group composed of instance 2, instance 4 and instance 6, and if a split instruction indicating that partition C is split into two partitions is received, two destination partitions can be constructed on instance 2, and instance 4 and instance 6 can also construct two destination partitions when replaying the log by the log sharing manner, and the naming of the two destination partitions is the same as that of the two destination partitions on instance 2.
[0076] In step S202, the data in the source partition is divided into the plurality of destination partitions according to the split instruction.
[0077] For example, according to the mapping relationship between the data and the destination partitions indicated in the split instruction, each data in the source partition is divided into a corresponding destination partition, for example, each data in the source partition is copied (replicated) into a corresponding destination partition. For example, a source partition with a key range of [1, 100) is split into a destination partition 1 with a key range of [1, 50) and a destination partition 2 with a key range of [51, 100), and then the data in the source partition within the key range [1, 50) is divided into the destination partition 1, and the data in the source partition within the key range [51, 100) is divided into the destination partition 2.
[0078] Further exemplarily, data in the MemTable of the source partition is flushed to the SSTable of the source partition, and the data in the SSTable of the source partition is divided into the plurality of destination partitions. For example, in the process of flushing the data in the MemTable of the source partition to the SSTable of the source partition, the source partition can be stopped from writing, i.e., the write operation is rejected.
[0079] It should be further understood that, in the process of dividing the data in the source partition into the plurality of destination partitions, a data write instruction for the source partition is received, and the data indicated by the data write instruction is written into the corresponding destination partition according to the mapping relationship between the data and the destination partition indicated in the split instruction.
[0080] For example, in the process of splitting the source partition with the key range [1, 100) into the destination partition 1 with the key range [1, 50) and the destination partition 2 with the key range [51, 100), if the source partition receives a data write instruction for the key range [1, 50), the data write instruction can be forwarded to the destination partition 1, so that the destination partition 1 writes the data indicated by the data write instruction into the MemTable of the destination partition 1.
[0081] In step 203, if all the data in the source partition is divided, the source partition is deleted.
[0082] In the partition split management method for a distributed database provided by the embodiments of the present disclosure, the consensus protocol group of the distributed database is composed of database instances rather than partitions, and each database instance in the consensus protocol group has a log stream for recording the transaction log of all the partitions in the database instance, and the data of different database instances in the consensus protocol group is the same and shares the same log stream. The method constructs the destination partition in the same consensus protocol group when the partition is split, so that the source partition and the destination partition belong to the same consensus protocol group, and the log streams of the source partition and the destination partition belong to the same log stream, i.e., the log stream shared in the consensus protocol group. Therefore, the partition split no longer involves the classification of the log stream, thereby improving the efficiency of the partition split and reducing the complexity of the consensus protocol group regarding the partition split.
[0083] In some embodiments of the present disclosure, step S201 shown in FIG. 2 can include: if a split instruction for the source partition is received and there is no partition migration operation on the source partition, constructing a plurality of destination partitions in the consensus protocol group where the source partition is located.
[0084] In other words, if a split instruction for a source partition is received, it is determined whether there is a partition migration operation on the source partition before the destination partition is built, and if not, multiple destination partitions are built in the consensus protocol group in which the source partition is located, so that the partition migration operation on the source partition and the partition split do not interfere with each other, and data consistency is further ensured.
[0085] In some embodiments of the present disclosure, the partition split management method applied to a distributed database further comprises: if the data in the source partition is not completely divided and a partition migration operation for the source partition is received, the partition migration operation is prohibited; and if the data in the source partition is not completely divided and a partition migration operation for any destination partition is received, the partition migration operation is delayed until the source partition is deleted.
[0086] Thus, the partition migration operation on the source partition and the partition split do not interfere with each other, and data consistency is further ensured.
[0087] FIG. 4 is a schematic structural diagram of an apparatus according to an example embodiment. Referring to FIG. 4, at the hardware level, the apparatus includes a processor 402, an internal bus 404, a network interface 406, a memory 408, and a non-volatile memory 410, and can further include other hardware required by tasks. One or more embodiments of the present disclosure can be implemented in a software manner, for example, by reading a corresponding computer program from the non-volatile memory 410 into the memory 408 by the processor 402 and then running. Of course, in addition to the software implementation, one or more embodiments of the present disclosure do not exclude other implementation manners, such as a logic device or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or a logic device.
[0088] Referring to FIG. 5, the partition split management apparatus applied to a distributed database can be applied to the apparatus shown in FIG. 4 to implement the technical solutions of the present disclosure. The partition split management apparatus applied to a distributed database can include:
[0089] A building module 501 is configured to receive a split instruction for a source partition, and build multiple destination partitions in a consensus protocol group in which the source partition is located, wherein the consensus protocol group includes multiple database instances, each of which has a log stream recording all transactions in the partitions, and the multiple database instances in the consensus protocol group share the same log stream;
[0090] A division module 502 is configured to divide the data in the source partition to the multiple destination partitions according to the split instruction;
[0091] The deletion module 503 is configured to delete the source partition if all data in the source partition is completely divided.
[0092] In one or more embodiments of the present specification, the construction module is configured to:
[0093] receive a split instruction for a source partition, and construct a plurality of destination partitions in a consensus protocol group in which the source partition is located according to a number of destination partitions indicated in the split instruction.
[0094] In one or more embodiments of the present specification, the construction module is configured to:
[0095] receive a split instruction for a source partition, construct a plurality of destination partitions in a leader instance in a consensus protocol group in which the source partition is located, and write an operation of constructing the plurality of destination partitions to a log stream;
[0096] share the log stream to other database instances in the consensus protocol group according to a consensus protocol of the consensus protocol group, so that the other database instances construct the plurality of destination partitions locally by replaying the log stream.
[0097] In one or more embodiments of the present specification, the construction module is configured to:
[0098] if a split instruction for a source partition is received and there is no partition migration operation on the source partition, construct a plurality of destination partitions in a consensus protocol group in which the source partition is located.
[0099] In one or more embodiments of the present specification, the division module is configured to:
[0100] divide each data in the source partition to a corresponding destination partition according to a mapping relationship between the data and the destination partitions indicated in the split instruction.
[0101] In one or more embodiments of the present specification, the division module is configured to:
[0102] flush the data in a memory table of the source partition to an ordered string table of the source partition, and divide the data in the ordered string table of the source partition to the plurality of destination partitions.
[0103] In one or more embodiments of the present specification, the apparatus further comprises a writing module configured to:
[0104] receive a data writing instruction for the source partition, and write data indicated by the data writing instruction to a corresponding destination partition.
[0105] In one or more embodiments of the present specification, the writing module is configured to:
[0106] receiving a data write instruction for the source partition, and writing data indicated by the data write instruction into a corresponding destination partition according to the mapping relationship between the data and the destination partition indicated in the split instruction.
[0107] In one or more embodiments of the present specification, the apparatus further comprises a migration processing module configured to:
[0108] if the data in the source partition is not completely divided, and a partition migration operation for the source partition is received, the partition migration operation is prohibited;
[0109] if the data in the source partition is not completely divided, and a partition migration operation for any destination partition is received, the partition migration operation is delayed until the source partition is deleted.
[0110] The system, apparatus, module or unit illustrated in the above embodiments can be implemented by a computer chip or entity, or by a product with certain functions. A typical implementation device is a computer, and the specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
[0111] In a typical configuration, a computer includes one or more processors (CPU), input / output interface, network interface, and memory.
[0112] The memory can include non-persistent memory in computer readable medium, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer readable media.
[0113] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, disk storage, quantum memory, graphene-based storage medium or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.
[0114] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but also other elements not explicitly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising the element.
[0115] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from the order in the embodiments and still achieve the desired result. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.
[0116] The terms used in one or more embodiments of the present specification are only for the purpose of describing specific embodiments and are not intended to limit one or more embodiments of the present specification. The singular forms "a", "an" and "the" used in one or more embodiments of the present specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein means and includes any or all possible combinations of one or more associated listed items.
[0117] The user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the specification are information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.
[0118] It should be understood that, although one or more embodiments of the specification can use the terms first, second, third, etc. to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the specification, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "when" or "when" or "in response to determining".
[0119] The above only describes the preferred embodiments of one or more embodiments of the specification, and does not limit one or more embodiments of the specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of the specification shall be included in the protection scope of one or more embodiments of the specification.
Claims
1. A method for managing partition split in a distributed database, the method comprising: receiving a split instruction for a source partition, and building a plurality of destination partitions in a consensus protocol group in which the source partition is located, wherein the consensus protocol group comprises at least one database instance, each database instance has a log stream for recording data operation logs of all partitions in the database instance, and the database instances in the consensus protocol group have the same data and share the same log stream; dividing data in the source partition into the plurality of destination partitions according to the split instruction; deleting the source partition if all data in the source partition is divided. 2.The method for managing partition split in a distributed database according to claim 1, wherein the receiving a split instruction for a source partition, and building a plurality of destination partitions in a consensus protocol group in which the source partition is located, comprises: receiving a split instruction for a source partition, and building a plurality of destination partitions in a consensus protocol group in which the source partition is located according to a number of destination partitions indicated in the split instruction. 3.The method for managing partition split in a distributed database according to claim 1 or 2, wherein the receiving a split instruction for a source partition, and building a plurality of destination partitions in a consensus protocol group in which the source partition is located, comprises: receiving a split instruction for a source partition, and building a plurality of destination partitions in a leader instance in a consensus protocol group in which the source partition is located, and writing an operation of building the plurality of destination partitions into a log stream; sharing the log stream to other database instances in the consensus protocol group according to a consensus protocol of the consensus protocol group, so that the other database instances build the plurality of destination partitions locally by replaying the log stream. 4.The method for managing partition split in a distributed database according to claim 1, wherein the receiving a split instruction for a source partition, and building a plurality of destination partitions in a consensus protocol group in which the source partition is located, comprises: if a split instruction for a source partition is received and there is no partition migration operation on the source partition, building a plurality of destination partitions in a consensus protocol group in which the source partition is located. 5.The method for managing partition split in a distributed database according to claim 1, wherein the dividing data in the source partition into the plurality of destination partitions according to the split instruction comprises: dividing each data in the source partition into a corresponding destination partition according to a mapping relationship between the data and the destination partitions indicated in the split instruction. 6.The method for managing partition split in a distributed database according to claim 1 or 5, wherein the dividing data in the source partition into the plurality of destination partitions according to the split instruction comprises: flushing data in a memory table of the source partition to an ordered string table of the source partition, and dividing data in the ordered string table of the source partition into the plurality of destination partitions. 7.The method for managing partition split in a distributed database according to claim 1, further comprising: receiving a data write instruction for the source partition, and writing data indicated by the data write instruction into a corresponding destination partition according to a mapping relationship between the data and the destination partition indicated in the split instruction.
8. The partition split management method for a distributed database according to claim 7, wherein the receiving a data write instruction for the source partition, and writing data indicated by the data write instruction into a corresponding destination partition comprises: receiving a data write instruction for the source partition, and writing data indicated by the data write instruction into a corresponding destination partition according to a mapping relationship between the data and the destination partition indicated in the split instruction.
9. The partition split management method for a distributed database according to claim 1, further comprising: if the data in the source partition is not completely divided, and a partition migration operation for the source partition is received, prohibiting the partition migration operation; if the data in the source partition is not completely divided, and a partition migration operation for any destination partition is received, delaying the partition migration operation until the source partition is deleted.
10. A partition split management apparatus for a distributed database, comprising: a construction module configured to receive a split instruction for a source partition, and construct a plurality of destination partitions in a consensus protocol group in which the source partition is located, wherein the consensus protocol group comprises a plurality of database instances, each of which has a log stream recording all transactions in the partitions, and the plurality of database instances in the consensus protocol group share the same log stream; a division module configured to divide data in the source partition to the plurality of destination partitions according to the split instruction; a deletion module configured to delete the source partition if all data in the source partition is completely divided.
11. The partition split management apparatus for a distributed database according to claim 10, wherein the construction module is configured to: receive a split instruction for a source partition, construct a plurality of destination partitions in a leader instance in the consensus protocol group in which the source partition is located, and write the operation of constructing a plurality of destination partitions into a log stream; share the log stream to other database instances in the consensus protocol group according to a consensus protocol of the consensus protocol group, so that the other database instances construct the plurality of destination partitions locally by replaying the log stream.
12. The partition split management apparatus for a distributed database according to claim 10, wherein the construction module is configured to: if a split instruction for a source partition is received, and there is no partition migration operation on the source partition, construct a plurality of destination partitions in the consensus protocol group in which the source partition is located.
13. The partition split management apparatus for a distributed database according to claim 10, wherein the division module is configured to: flush data in a memory table of the source partition to an ordered string table of the source partition, and divide data in the ordered string table of the source partition to the plurality of destination partitions.
14. The partition split management apparatus for a distributed database according to claim 10, further comprising a write module configured to: receiving a data write instruction for the source partition, and writing data indicated by the data write instruction into the corresponding destination partition. 15.The partition split management device for distributed database of claim 10, further comprising a migration processing module configured to: if the data in the source partition is not completely divided and a partition migration operation for the source partition is received, prohibiting the partition migration operation; if the data in the source partition is not completely divided and a partition migration operation for any destination partition is received, delaying the partition migration operation until the source partition is deleted. 16.A computer program product comprising computer programs / instructions which, when executed by a processor, implement the steps of the method of any one of claims 1 to 9. 17.An electronic device comprising: a processor; a memory for storing processor-executable instructions; wherein the processor, by running the executable instructions, implements the method of any one of claims 1 to 9. 18.A computer-readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the steps of the method of any one of claims 1 to 9.
Citation Information
Patent Citations
Method for partitioning database and system thereof
CN101876983A
Database management
CN112889040A
Partition splitting management method and device applied to distributed database
CN119293121A
Partition splitting in a distributed database
US11263184B1