Partition self-adaptive equalization method of database table and related device

Abstract

The invention discloses a partition self-adaptive balancing method of a database table and a related device, and relates to the technical field of distributed databases, and the method comprises the following steps: determining whether a partition meeting a partition segmentation condition and a partition merging condition exists in each partition according to load data of each partition where a target table is located, and determining whether a node meeting a partition equalization condition exists in the partition management nodes corresponding to each partition, performing segmentation equalization processing on the first partition meeting the partition segmentation condition, performing merging equalization processing on the second partition meeting the partition merging condition, and after the segmentation equalization processing and the merging equalization processing are completed, performing the segmentation equalization processing on the first partition and performing the merging equalization processing on the second partition. And randomly selecting a partition from the partitions corresponding to the target partition management node meeting the partition balance condition, and migrating the selected partition to other partition management nodes. According to the method, load balance distribution of the target table in the partitions and the partition management nodes can be realized through merging balance and segmentation balance processing of the partitions and partition balance processing of the partition management nodes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of distributed database, and particularly relates to a partition adaptive balancing method of a database table and a related device. BACKGROUND

[0002] In a big data cluster, load balancing of partitions is crucial to the performance and stability of the cluster. Unbalanced load distribution can affect the overall performance of the database. In particular, in a single-table high-concurrency read-write scenario, the performance bottleneck can be concentrated on a specific partition management node, resulting in increased latency, reduced throughput, and even node crashes.

[0003] Current partition load balancing methods are performed on the dimension of partition management nodes. However, this method cannot effectively balance the load of a single database table, which can cause the performance bottleneck to be concentrated on a specific partition management node in a single-table high-concurrency read-write scenario, resulting in increased latency, reduced throughput, and even node crashes. SUMMARY

[0004] In view of the above problems, the present application provides a partition adaptive balancing method of a database table and a related device to achieve the purpose of balancing the load of partitions in the dimension of a database table. The specific scheme is as follows:

[0005] The first aspect of the present application provides a partition adaptive balancing method of a database table, comprising:

[0006] obtaining load data of each partition in which a target table to be balanced in a target database is located;

[0007] determining whether there is a partition that meets preset partition splitting conditions and partition merging conditions in the each partition according to the load data of the each partition, and determining whether there is a node that meets a preset partition balancing condition in a partition management node corresponding to the each partition, the partition splitting conditions and the partition merging conditions being used to identify a partition with unbalanced load distribution, and the partition balancing condition being used to identify a partition management node with unbalanced load distribution;

[0008] if there is a first partition that meets the partition splitting conditions, performing splitting balancing treatment on the first partition, the splitting balancing treatment including splitting the first partition and / or migrating the split sub-partitions to other partition management nodes;

[0009] if there is a second partition that meets the partition merging conditions, performing merging balancing treatment on the second partition;

[0010] If there is a target partition management node satisfying the partition balance condition, at least one partition is randomly selected from the partition corresponding to the target partition management node after the split balance treatment and the merge balance treatment are completed, and the selected partition is migrated to other partition management nodes.

[0011] In a possible implementation, the partition split condition comprises one or more of the following conditions:

[0012] The size of the partition is greater than a preset split threshold value;

[0013] The size of the partition is greater than a first proportion threshold value, and the first proportion threshold value is an average value of the sizes of the respective partitions multiplied by a preset split proportion;

[0014] The read-write request increment of the partition satisfies a preset increment condition;

[0015] And / or, the partition merge condition comprises one or more of the following conditions:

[0016] The sizes of two partitions adjacent in physical address are both less than a preset merge threshold value;

[0017] The sizes of two partitions adjacent in physical address are both less than a second proportion threshold value, and the second proportion threshold value is an average value of the sizes of the respective partitions multiplied by a preset merge proportion.

[0018] In a possible implementation, the process of determining whether there is a node satisfying a preset partition balance condition in the respective partition management nodes corresponding to the respective partitions comprises:

[0019] According to the load data of the respective partitions, determining the respective load data of all partition management nodes corresponding to the respective partitions;

[0020] According to the respective load data of all partition management nodes, determining a load standard deviation and a load mean value;

[0021] According to the respective load data of all partition management nodes, the load standard deviation and the load mean value, determining a total standardization value of the respective partition management nodes;

[0022] Determining whether there is a node with a total standardization value greater than a preset load balance threshold value in the all partition management nodes, wherein the node with the total standardization value greater than the load balance threshold value is the node satisfying the preset partition balance condition.

[0023] In a possible implementation, the split balance treatment on the first partition comprises:

[0024] Splitting the first partition to obtain split sub-partitions;

[0025] In a case where the preset split balancing strategy is enabled, load data of each partition management node in the other partition management nodes and a localization rate of the sub-partition migrated to the each partition management node are determined;

[0026] According to the load data of the each partition management node and the localization rate of the sub-partition migrated to the each partition management node, a total generation value of the sub-partition migrated to the each partition management node is calculated;

[0027] A partition management node with a minimum total generation value is determined from the other partition management nodes as a to-be-migrated node;

[0028] The sub-partition is migrated to the to-be-migrated node.

[0029] In a possible implementation, the load data includes a load value of at least one load indicator;

[0030] The calculation of the total generation value of the sub-partition migrated to the each partition management node according to the load data of the each partition management node and the localization rate of the sub-partition migrated to the each partition management node includes:

[0031] According to the load data of the each partition management node, a load generation value corresponding to the at least one load indicator on the each partition management node is determined respectively;

[0032] According to the localization rate of the sub-partition migrated to the each partition management node, a localization rate generation value on the each partition management node is determined;

[0033] According to the load generation value corresponding to the at least one load indicator on the each partition management node and the localization rate generation value on the each partition management node, the total generation value of the sub-partition migrated to the each partition management node is obtained.

[0034] In a possible implementation, the calculation of the total generation value of the sub-partition migrated to the each partition management node according to the load generation value corresponding to the at least one load indicator on the each partition management node and the localization rate generation value on the each partition management node includes:

[0035] According to the load generation value corresponding to the at least one load indicator on the each partition management node and the localization rate generation value on the each partition management node, information entropy corresponding to the at least one load indicator and information entropy corresponding to a localization rate indicator are determined;

[0036] determine a cost weight corresponding to each of the at least one load indicator according to information entropy corresponding to the at least one load indicator respectively;

[0037] determine a cost weight corresponding to the localization rate indicator according to information entropy corresponding to the localization rate indicator;

[0038] perform weighted summation on a load cost value corresponding to each of the at least one load indicator on each partition management node and a localization rate cost value on each partition management node according to the cost weight corresponding to each of the at least one load indicator and the cost weight corresponding to the localization rate indicator, to obtain a total cost value of the sub-partition migrated to each partition management node.

[0039] The second aspect of the present application provides a device for adaptive balancing of a database table partition, comprising a monitoring unit and an execution unit, wherein the monitoring unit comprises a collector and an analyzer, and the execution unit comprises a splitter, a merger and a balancer;

[0040] The collector is configured to acquire load data of each partition of a target table to be balanced in a target database.

[0041] The analyzer is configured to determine whether there is a partition satisfying a preset partition splitting condition and a partition merging condition in each partition according to the load data of each partition, and determine whether there is a node satisfying a preset partition balancing condition in a partition management node corresponding to each partition, wherein the partition splitting condition and the partition merging condition are used to identify a partition with unbalanced load distribution, and the partition balancing condition is used to identify a partition management node with unbalanced load distribution.

[0042] The splitter is configured to, if there is a first partition satisfying the partition splitting condition, perform a splitting balancing treatment on the first partition, wherein the splitting balancing treatment comprises splitting the first partition, and / or migrating a split sub-partition to another partition management node.

[0043] The merger is configured to, if there is a second partition satisfying the partition merging condition, perform a merging balancing treatment on the second partition.

[0044] The balancer is configured to, if there is a target partition management node satisfying the partition balancing condition, randomly select at least one partition from a partition corresponding to the target partition management node after the splitting balancing treatment and the merging balancing treatment are completed, and migrate the selected partition to another partition management node.

[0045] In a possible implementation, the device further comprises a configuration unit, wherein the configuration unit comprises a parameter configurator and a strategy configurator.

[0046] The parameter configurator is configured to configure parameters in the partition splitting condition, the partition merging condition, and the partition balancing condition.

[0047] The policy configurator is configured to configure a stock balancing policy and / or a splitting balancing policy, the splitting balancing policy being used for splitting balancing treatment on the first partition, and the stock balancing policy being used for randomly selecting at least one partition from the partitions corresponding to the target partition management node after the splitting balancing treatment and the merging balancing treatment are completed, and migrating the selected partition to other partition management nodes.

[0048] The third aspect of the present application provides a computer program product, comprising computer readable instructions, when the computer readable instructions are run on an electronic device, the electronic device implements the database table partition adaptive balancing method of the first aspect or any implementation manner of the first aspect.

[0049] The fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected with the processor, wherein:

[0050] The memory is configured to store a computer program;

[0051] The processor is configured to execute the computer program, so that the electronic device can implement the guarantee credit evaluation method of the first aspect or any implementation manner of the first aspect.

[0052] The fifth aspect of the present application provides a computer storage medium, the storage medium carries one or more computer programs, when the one or more computer programs are executed by an electronic device, the electronic device can implement the database table partition adaptive balancing method of the first aspect or any implementation manner of the first aspect.

[0053] By the technical scheme, the database table partition adaptive balancing method provided by the application obtains load data of each partition where a target table to be balanced in a target database, determines whether there is a partition satisfying preset partition split conditions and partition merging conditions in each partition according to the load data of each partition, and determines whether there is a node satisfying a preset partition balancing condition in a partition management node corresponding to each partition. If there is a first partition satisfying the partition split condition, split balancing processing is performed on the first partition. If there is a second partition satisfying the partition merging condition, merging balancing processing is performed on the second partition. If there is a target partition management node satisfying the partition balancing condition, at least one partition is randomly selected from a partition corresponding to the target partition management node after the split balancing processing and the merging balancing processing are completed, and the selected partition is migrated to another partition management node. As can be seen, the application can realize load balancing distribution of the partition and the partition management node where the target table is located through the merging balancing processing and the split balancing processing of the partition and the partition balancing processing of the partition management node, so that the read-write request can be relatively uniformly dispersed to multiple partitions and partition management nodes even if the target table has a high concurrent read-write problem, thereby avoiding the problems of increased delay, reduced throughput, and even node crash. BRIEF DESCRIPTION OF DRAWINGS

[0054] The above and other features, advantages, and aspects of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings in which:

[0055] Figure 1 A flowchart of a database table partition adaptive balancing method provided by the application;

[0056] Figure 2 A structural diagram of a database table partition adaptive balancing device provided by the application;

[0057] Figure 3 A structural diagram of another database table partition adaptive balancing device provided by the application;

[0058] Figure 4 A general flowchart of a monitoring unit provided by the application;

[0059] Figure 5 A general flowchart of a splitter provided by the application;

[0060] Figure 6 A general flowchart of a merger provided by the application;

[0061] Figure 7A general flowchart of an equalizer provided for the present application;

[0062] Figure 8 A structural schematic diagram of an electronic device provided for the present application. DETAILED DESCRIPTION

[0063] The embodiments of the present application are described below in conjunction with the drawings of the embodiments of the present application. The terms used in the implementation part of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0064] The embodiments of the present application are described below in conjunction with the drawings. It is known to those of ordinary skill in the art that, as technology develops and new scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0065] The terms “first”, “second”, and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, and this is only a distinguishing way used in the description of the embodiments of the present application to describe the objects with the same attributes. In addition, the terms “include” and “have” and any variations thereof are intended to cover non-exclusive inclusion, so that the processes, methods, systems, products or devices containing a series of units do not have to be limited to those units, but can include other units not clearly listed or inherent to these processes, methods, products or devices.

[0066] In order for those skilled in the art to better understand the present application, the related terms applied in the present application are explained as follows.

[0067] Partition: the basic unit of expansion and load balancing in a distributed database table, which is essentially an interval of continuous storage sorted by row keys. A table is composed of multiple partitions. When the data volume of the table grows, the database will divide the data into different partitions according to the row key range. If the partitions are too large, the system will dynamically split them, and conversely, multiple partitions will be merged.

[0068] Partition management node: a server node in a big data cluster responsible for managing partitions and processing related read and write requests.

[0069] Partition splitting: the process of splitting a large partition into two partitions.

[0070] Partition merging: the process of merging two small partitions into one partition.

[0071] Partition balancing: adjusting the distribution of partition load by migrating partitions from one partition management node to another. When migrating, partitions only migrate read and write requests, and do not migrate actual data.

[0072] Localisation rate: represents the degree of matching between the data in the partition and the partition management node to which it is assigned in physical storage, a value between 0 and 1.

[0073] It can be understood that, before using the technical solutions disclosed in the embodiments of the present application, the type, use range, use scenario, etc. of the personal information involved in the present application should be informed to the user and the authorization of the user should be obtained according to relevant laws and regulations through appropriate means.

[0074] It can be understood that the data involved in the present technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of relevant laws and regulations and relevant provisions.

[0075] Referring to Figure 1 , a flowchart of a database table partition adaptive balancing method provided by the present application is shown, and the database table partition adaptive balancing method can include: Figure 1

[0076] Step S101, obtaining the load data of each partition where the target table in the target database to be balanced is located.

[0077] The above-mentioned target database refers to the database to be balanced, and the target table refers to a single database table in the target database to be balanced. It can be understood that all database tables in the target database may need to be balanced, and then all database tables can be taken as target tables respectively.

[0078] In order to realize the load balanced distribution of the target table on the partitions and the partition management nodes, the present application can obtain the load data of each partition where the target table is located.

[0079] It can be understood that, with the increase and deletion of the written data, the load distribution and size of each partition where the target table is located will change, in order to ensure that the partitions and the partition management nodes where the target table is located can be in a relatively balanced state at each moment, optionally, the present embodiment can preset the periodic time of data collection (for example, 5 minutes), so that the load data of each partition where the target table is located can be obtained periodically, and the partition splitting, partition splitting balancing, partition merging, partition load balancing of the partition management nodes, etc. are handled according to the following steps.

[0080] Step S102, determining whether there is a partition in each partition that meets the preset partition splitting condition and partition merging condition according to the load data of each partition, and determining whether there is a node in the corresponding partition management node of each partition that meets the preset partition balancing condition.

[0081] ​Here, the partition splitting condition and the partition merging condition are used to identify the partitions with unbalanced load distribution, and the partition balancing condition is used to identify the partition management node with unbalanced load distribution.

[0082] In step S103, if there is a first partition satisfying the partition splitting condition, the first partition is subjected to the splitting balancing treatment, if there is a second partition satisfying the partition merging condition, the second partition is subjected to the merging balancing treatment, and if there is a target partition management node satisfying the partition balancing condition, at least one partition is randomly selected from the partition corresponding to the target partition management node after the splitting balancing treatment and the merging balancing treatment are completed, and the selected partition is migrated to another partition management node.

[0083] Optionally, the splitting balancing treatment includes splitting the first partition, and / or migrating the sub-partition split from the first partition to another partition management node. Here, the another partition management node refers to the partition management node other than the partition management node corresponding to the first partition in the cluster.

[0084] For example, the cluster includes partition management node 1 to partition management node 10, the first partition is managed by the partition management node 1, and the another partition management node refers to the partition management node 2 to the partition management node 10.

[0085] Optionally, the embodiment can preset a splitting balancing strategy and set a switch for the splitting balancing strategy. In the case where the splitting balancing strategy is not turned on, the splitting balancing treatment only includes splitting the first partition. In the case where the splitting balancing strategy is turned on, the splitting balancing treatment includes splitting the first partition and migrating the sub-partition split from the first partition to another partition management node.

[0086] In a possible implementation, in order to avoid too many times of daily splitting and merging, causing a series of problems such as resource waste, performance loss, stability risk, cost increase, etc., the embodiment can preset an upper limit value of the number of partitions performing splitting per day and an upper limit value of the number of partitions performing merging per day. Then, the first partition satisfying the partition splitting condition can be subjected to the splitting balancing treatment when the number of splitting times per day does not reach the upper limit value of the number of partitions performing splitting per day, and the second partition satisfying the partition merging condition can be subjected to the merging balancing treatment when the number of merging times per day does not reach the upper limit value of the number of partitions performing merging per day.

[0087] Optionally, the time for performing the splitting and the time for performing the merging can also be set. In this case, when the relevant conditions for the splitting are met, the first partition is split only when the time for performing the splitting is reached (when the time for performing the splitting is not reached, the first partition is stored in the list of partitions to be split, and the sub-partitions split from the first partition can be stored in the list of partitions to be balanced); and when the relevant conditions for the merging are met, the second partition is merged and balanced only when the time for performing the merging is reached (when the time for performing the merging is not reached, the second partition is stored in the list of partitions to be merged).

[0088] Similarly, optionally, the preset inventory balancing strategy can also be set, and a switch for the inventory balancing strategy can also be set. When the inventory balancing strategy is not started, even if there is a target partition management node that meets the partition balancing condition, the above-mentioned "after the splitting balancing processing and the merging balancing processing are completed, at least one partition is randomly selected from the partition corresponding to the target partition management node, and the selected partition is migrated to other partition management nodes" is not performed. When the inventory balancing strategy is started, after the target partition management node is determined, the "after the splitting balancing processing and the merging balancing processing are completed, at least one partition is randomly selected from the partition corresponding to the target partition management node, and the selected partition is migrated to other partition management nodes" is performed. Here, the other partition management nodes refer to the partition management nodes other than the target partition management node in which the target table is located.

[0089] Optionally, the time for performing the load balancing can also be set. In this case, the sub-partitions in the list of partitions to be balanced (i.e., the sub-partitions split from the first partition) are balanced only when the time for performing the load balancing is reached, and after the splitting balancing processing and the merging balancing processing are completed, at least one partition is randomly selected from the partition corresponding to the target partition management node, and the selected partition is migrated to other partition management nodes.

[0090] The partition adaptive balancing method of the database table provided in the application obtains load data of each partition where a target table to be balanced in a target database, determines whether there is a partition satisfying preset partition split conditions and partition merge conditions in each partition according to the load data of each partition, and determines whether there is a node satisfying a preset partition balancing condition in a partition management node corresponding to each partition. If there is a first partition satisfying the partition split conditions, split balancing processing is performed on the first partition. If there is a second partition satisfying the partition merge conditions, merge balancing processing is performed on the second partition. If there is a target partition management node satisfying the partition balancing conditions, at least one partition is randomly selected from the partition corresponding to the target partition management node after the split balancing processing and the merge balancing processing are completed, and the selected partition is migrated to another partition management node. As can be seen, the application can realize load balancing distribution of the partition where the target table is located and the partition management node through the merge balancing processing and the split balancing processing of the partition and the partition balancing processing of the partition management node, so that the read-write request can be relatively uniformly dispersed to multiple partitions and partition management nodes even if the target table has a high concurrent read-write problem, thereby avoiding the problems of increased delay, reduced throughput, and even node crash.

[0091] In some embodiments of the application, the process of "determining whether there is a partition satisfying preset partition split conditions and partition merge conditions in each partition according to the load data of each partition, and determining whether there is a node satisfying a preset partition balancing condition in a partition management node corresponding to each partition" in the foregoing step S102 is introduced.

[0092] In a possible implementation, the partition split conditions can include one or more of the following conditions: the partition size is greater than a preset split threshold, the partition size is greater than a first proportion threshold, and the read-write request increment of the partition satisfies a preset increment condition. The first proportion threshold is the average value of the partition sizes of each partition where the target table is located multiplied by a preset split proportion.

[0093] Specifically, in this embodiment, the load data of each partition includes the partition size, and the partition whose partition size is greater than the preset split threshold can be selected from each partition as the partition satisfying the partition split conditions, that is, the first partition.

[0094] Meanwhile, the average value can be calculated according to the partition sizes of each partition where the target table is located, and then the average value is multiplied by the preset split proportion to obtain the first proportion threshold. In this way, the partition whose partition size is greater than the first proportion threshold can be selected from each partition as the partition satisfying the partition split conditions, that is, the first partition.

[0095] In addition, considering that the table data stored in the partition may have hot cycles and cold cycles, the number of read and write requests for the partition may suddenly increase during the hot cycle. If the partitions in the hot cycle are not split and balanced, it may cause request delays and node crashes when facing sudden increases in read and write requests. Therefore, it is also necessary to identify the partitions with sudden increases in read and write requests.

[0096] Based on this, the load data of each partition can also include the number of read requests and the number of write requests. As mentioned above, this embodiment can periodically collect the load data of each partition, thereby obtaining the read and write request increments of each partition at the same time in the latest statistical periods, and calculating the mean and standard deviation of the read and write request increments of each partition. If the difference between the read and write request increments of a partition and the mean exceeds a certain multiple of the standard deviation, then the partition is determined to be a partition that meets the increment condition, that is, the first partition that meets the partition splitting condition.

[0097] For example, starting from midnight every day, load data of each partition is collected every 5 minutes, and the statistical period is 24 hours (1 day). Then, the number of read requests and write requests of each partition collected at 7:55 and 8:00 every day in the recent days can be obtained. From this, the incremental read and write requests of each partition at 8:00 every day in the recent days can be obtained. For example, the incremental read and write requests of each partition at 8:00 = (the number of read requests of the partition at 8:00 - the number of read requests of the partition at 7:55) + (the number of write requests of the partition at 8:00 - the number of write requests of the partition at 7:55).

[0098] Next, the mean and standard deviation of the read and write request increments for all partitions of the target table are calculated, recorded as the increment mean and increment standard deviation. For each partition, the difference between the read and write request increment and the increment mean is calculated, recorded as the increment difference. The product of the increment standard deviation and a preset multiple is calculated, recorded as the increment product. If the increment difference is greater than the increment product, the partition is determined to meet the increment condition.

[0099] It should be noted that the above-mentioned incremental conditions are only examples. In addition, the incremental conditions can also be other, and this application does not make specific limitations.

[0100] In one possible implementation, the partition merging condition may include one or more of the following conditions: the sizes of two partitions with adjacent physical addresses are both smaller than a preset merging threshold, and the sizes of two partitions with adjacent physical addresses are both smaller than a second ratio threshold. The second ratio threshold is the average size of each partition containing the target table multiplied by a preset merging ratio.

[0101] Specifically, in this embodiment, the partition size is included in the load data of each partition, and two partitions with adjacent physical addresses and both smaller than a preset merging threshold in partition size can be selected from the partitions as two partitions satisfying the partition merging condition, i.e., two second partitions.

[0102] Meanwhile, an average value can be calculated according to the partition sizes of the partitions where the target table is located, and the average value is multiplied by a preset merging ratio to obtain a second ratio threshold. Thus, two partitions with adjacent physical addresses and both smaller than the second ratio threshold in partition size can be selected from the partitions as two partitions satisfying the partition merging condition, i.e., two second partitions.

[0103] In a possible implementation, the process of determining whether there is a node satisfying the preset partition balancing condition in the partition management nodes corresponding to each partition can include: determining the load data of all the partition management nodes corresponding to each partition according to the load data of each partition; determining a load standard deviation and a load average according to the load data of all the partition management nodes; determining a total standardization value of all the partition management nodes according to the load data of all the partition management nodes, the load standard deviation, and the load average; and determining whether there is a node with a total standardization value greater than a preset load balancing threshold in all the partition management nodes, where the node with the total standardization value greater than the load balancing threshold is the node satisfying the preset partition balancing condition.

[0104] Optionally, the load data of each partition includes a load value of at least one load indicator, and optionally, the at least one load indicator includes but is not limited to the following indicators: the number of partitions, the partition size, the read request increment, and the write request increment.

[0105] It should be noted that the number of read requests and the number of write requests will be cleared after the partition migration, and therefore, when calculating the read request increment and the write request increment, the difference between the number of read requests (or the number of write requests) of a time period of the current day and the number of read requests (or the number of write requests) of the same time period of the previous day needs to be calculated.

[0106] For any load indicator, the load value of the load indicator on all the partition management nodes can be determined according to the load values of the load indicator on the partitions, and specifically, the load values of the load indicator on all the partitions corresponding to the same partition management node are added, and the sum is taken as the load value of the load indicator on the same partition management node.

[0107] For example, the partitions 1 to 5 correspond to the partition management node 1, the partition management node 1, the partition management node 2, the partition management node 2, and the partition management node 2 respectively, and the read request increments of the partitions 1 to 5 are 100, 120, 110, 90, and 150 respectively, then the read request increment of the partition management node 1 is 100+120=220, and the read request increment of the partition management node 2 is 110+90+150=350.

[0108] Then, according to the load values of the load indicators on all the partition management nodes, the load standard deviation and the load mean corresponding to the load indicators are obtained. Specifically, for the jth load indicator in the at least one load indicator, the load standard deviation corresponding to the jth load indicator is obtained by using the following formula (1).

[0109] Formula (1);

[0110] wherein, denotes the load standard deviation corresponding to the jth load indicator, denotes the load value of the jth load indicator on the ith partition management node, denotes the load mean corresponding to the jth load indicator (obtained by averaging the load values of the jth load indicator on all the partition management nodes), denotes the total number of all the partition management nodes corresponding to each partition.

[0111] Further, according to the load value of the jth load indicator on each partition management node, the load standard deviation and the load mean corresponding to the jth load indicator, the Z-Score standardized value of the jth load indicator on each partition management node is determined. Here, Z-Score refers to a standard score, which is an index used in statistics to measure the relative position of a data point to the average value of a data set. By converting the original data into a value in units of standard deviation, the standardization and cross-data set comparison of data are realized.

[0112] Formula (2);

[0113] wherein, denotes the Z-Score standardized value of the jth load indicator on the ith partition management node.

[0114] Finally, the Z-Score standardized values of all the load indicators on each partition management node are weighted and summed according to the preset standard score weight, that is, the total standardized value of each partition management node can be obtained, and thus it can be further determined whether there is a node with a total standardized value greater than a preset load balancing threshold in all the partition management nodes.

[0115] Therefore, by setting the partition splitting condition, the partition merging condition and the partition balancing condition, the target table can be accurately screened out from the partition and the partition management node with uneven load distribution, and a balancing plan can be generated based on the screened partition and the partition management node. Compared with the prior art which needs to generate a balancing plan through a large number of random simulations, the calculation process of the present application is more concise and the overhead is smaller.

[0116] The process of performing the splitting balancing treatment on the first partition in step S103 is described below.

[0117] First, the first partition is split to obtain a split sub-partition. In the case where the preset splitting balancing strategy is turned on, the load data of each partition management node in the other partition management nodes can be determined according to the foregoing embodiments, that is, the load data of a partition management node is obtained by summing the load data of all partitions managed by the partition management node.

[0118] In addition, the localization rate of the split sub-partition migrated to each partition management node in the other partition management nodes can also be determined, where the localization rate of the sub-partition migrated to each partition management node refers to the matching degree of the data in the sub-partition and the physical storage of each partition management node.

[0119] Next, according to the load data of each partition management node (hereinafter referred to as each partition management node) in the other partition management nodes and the localization rate of the sub-partition migrated to each partition management node, the total value of the sub-partition migrated to each partition management node is calculated.

[0120] Optionally, the total value can be determined according to the following process.

[0121] As described above, the load data includes the load value of at least one load indicator, and then the present embodiment can first determine the load value of at least one load indicator corresponding to each partition management node according to the load data of each partition management node. Optionally, the calculation formula of the load value is as follows.

[0122] Formula (3);

[0123] wherein, represents the load value of the jth load indicator on the kth partition management node, represents the load value of the jth load indicator on the kth partition management node, represents the minimum load value of the jth load indicator on each partition management node included in the other partition management nodes, represents the maximum load value of the jth load indicator on each partition management node included in the other partition management nodes, .

[0124] The embodiment can also determine the localization rate value on each partition management node according to the localization rate of the sub-partition migrated to each partition management node. Optionally, the calculation formula of the localization rate value is as follows.

[0125] Formula (4);

[0126] wherein, represents the localization rate value on the kth partition management node, represents the localization rate of the sub-partition migrated to the kth partition management node.

[0127] Finally, the total value of the sub-partition migrated to each partition management node is obtained according to the load value corresponding to each load index on each partition management node and the localization rate value on each partition management node.

[0128] Optionally, the embodiment can obtain the cost weight corresponding to each load index and the cost weight corresponding to the localization rate index, and then combine the cost weight corresponding to each load index and the cost weight corresponding to the localization rate index to weight and sum the load value corresponding to each load index on each partition management node and the localization rate value on each partition management node, to obtain the total value of the sub-partition migrated to each partition management node. Specifically, the formula of the weighted sum to obtain the total value is as follows.

[0129] Formula (5);

[0130] wherein, represents the total value of the sub-partition migrated to the kth partition management node, represents the cost weight corresponding to the jth load index, represents the cost weight corresponding to the localization rate index.

[0131] Optionally, the above and may be preset cost weights.

[0132] In a more preferred implementation, the above and may also be adaptive weight values set for different value reference factors (here, the value reference factor refers to at least one load index and the localization rate index).

[0133] Specifically, the entropy value method can be used to calculate and According to the load value corresponding to each load index and the localization rate value corresponding to each partition management node, information entropy corresponding to each load index and information entropy corresponding to the localization rate index are determined; according to the information entropy corresponding to each load index, a cost weight corresponding to each load index is determined; and according to the information entropy corresponding to the localization rate index, a cost weight corresponding to the localization rate index is determined.

[0134] Here, the calculation formula of the information entropy is as follows:

[0135] Formula (6);

[0136] wherein, denotes information entropy corresponding to the i-th index (including the load index and the localization rate index); if the i-th index is the load index, then is , and if the i-th index is the localization rate index, then is .

[0137] Optionally, the process of "determining the cost weight corresponding to each load index according to the information entropy corresponding to each load index, and determining the cost weight corresponding to the localization rate index according to the information entropy corresponding to the localization rate index" can adopt the following formula (7).

[0138] Formula (7);

[0139] wherein, if the i-th index is the load index, i.e. denotes information entropy corresponding to the i-th load index, then is , and if the i-th index is the localization rate index, i.e. denotes information entropy corresponding to the localization rate index, then is .

[0140] ​​​​​​Therefore, in this embodiment, after calculating the total cost of migrating the sub-partition to each partition management node using the above formula (5), the partition management node with the smallest total cost is determined from other partition management nodes as the node to be migrated. The sub-partition is migrated to the node to be migrated, thereby completing the migration process of the sub-partition split from the first partition. The sub-partitions split from all the first partitions (or the first partitions within the upper limit of the number of partitions split per day) are migrated according to the above process, thus completing the entire migration process, thereby achieving partition splitting balance.

[0141] Optionally, as described above, the partition splitting conditions may include that the read and write request increment of the partition meets the preset increment condition. In this case, the first partition where the application detects a sudden increase in read and write requests can be split and migrated first to improve the overall read and write performance of the target database.

[0142] Similarly, in an embodiment of the present application, the process of "randomly selecting at least one partition from the partition corresponding to the target partition management node and migrating the selected partition to other partition management nodes" in the previous step S103 may include: randomly selecting at least one partition from the partition corresponding to the target partition management node, migrating the at least one partition to other partition management nodes according to the previous embodiment, calculating the load data of the target partition management node after the partition migration, and determining whether the load data of the target partition management node after the partition migration is less than the load balancing threshold. If not, return to randomly selecting at least one partition from the partition corresponding to the target partition management node until the load data of the target partition management node after the partition migration is less than the load balancing threshold.

[0143] Compared with the traditional weight calculation method based on normal distribution, this embodiment adopts the cost weight calculation method based on information entropy, which is more adaptable when facing load and dynamically changing load scenarios, can handle extreme values ​​and load concentration, has a wider application scenario and higher practicality.

[0144] The above describes a partition adaptive balancing method for a database table provided by an embodiment of the present application. The following describes an apparatus for executing the partition adaptive balancing method for a database table.

[0145] See also Figure 2 , Figure 2 This is a schematic diagram of the structure of a partition adaptive balancing device for a database table provided in an embodiment of the present application. Figure 2 As shown, the partition adaptive balancing device for the database table may include: a monitoring unit 21 and an execution unit 22 , wherein the monitoring unit includes a collector 211 and an analyzer 212 , and the execution unit includes a splitter 221 , a merger 222 and an equalizer 223 .

[0146] The collector 211 is configured to acquire load data of each partition where the target table to be balanced in the target database is located;

[0147] The analyzer 212 is configured to determine whether there is a partition that meets a preset partition split condition and / or a partition merge condition in each partition according to the load data of the target table in each partition, and determine whether there is a node that meets a preset partition balancing condition in the partition management node corresponding to each partition, the partition split condition and the partition merge condition being used to identify a partition with unbalanced load distribution, and the partition balancing condition being used to identify a partition management node with unbalanced load distribution;

[0148] The splitter 221 is configured to perform split balancing processing on a first partition if the first partition meets the partition split condition, the split balancing processing including splitting the first partition, and / or migrating the split sub-partitions to other partition management nodes;

[0149] The merger 222 is configured to perform merge balancing processing on a second partition if the second partition meets the partition merge condition;

[0150] The balancer 223 is configured to randomly select at least one partition from the partition corresponding to the target partition management node after the split balancing processing and the merge balancing processing are completed if the target partition management node meets the partition balancing condition, and migrate the selected partition to other partition management nodes.

[0151] In a possible implementation, as Figure 3 The database table partition adaptive balancing apparatus provided in the present application can further include a configuration unit 20, wherein the configuration unit includes a parameter configurator 201 and a strategy configurator 202.

[0152] The parameter configurator 201 is configured to configure parameters in the partition split condition, the partition merge condition and the partition balancing condition.

[0153] The strategy configurator 202 is configured to configure a stock balancing strategy and / or a split balancing strategy, the split balancing strategy being used to perform split balancing processing on a first partition, and the stock balancing strategy being used to randomly select at least one partition from the partition corresponding to the target partition management node after the split balancing processing and the merge balancing processing are completed, and migrate the selected partition to other partition management nodes.

[0154] To make the skilled in the art more understand the above-mentioned database table partition adaptive balancing apparatus, the following is introduced in detail.

[0155] The parameter configurator 201 is responsible for configuring the parameters related to the execution operation, including but not limited to the following parameters:

[0156] 1) For partition splitting operation, the following parameters are mainly configured: threshold value that needs to be reached to trigger splitting operation (i.e. preset splitting threshold value, unit: MB, and preset splitting ratio or first ratio threshold value), upper limit value of the number of partitions that perform splitting per day, time of performing splitting.

[0157] 2) For partition merging operation, the following parameters are mainly configured: threshold value that needs to be reached to trigger merging operation (i.e. preset merging threshold value, unit: MB, and preset merging ratio or second ratio threshold value), upper limit value of the number of partitions that perform merging per day, time of performing merging.

[0158] 3) For partition balancing operation, the following parameters are mainly configured: time of performing load balancing, load balancing threshold value.

[0159] Of course, the above parameters are only examples, and in addition to this, there can be other parameters, such as policy switch.

[0160] It should be noted that the operations of splitting, merging and balancing performed by the embodiment can be controlled by a timing task, and the start and stop of the task, the execution time, etc. can be flexibly configured in the parameter configurator.

[0161] The policy configurator 202 is responsible for configuring the related policies of performing load balancing, which in the embodiment includes but is not limited to the following two policies:

[0162] 1) Split balancing policy: for the sub-partitions split out by the first partition, migration is performed immediately after splitting is completed.

[0163] 2) Inventory balancing policy: for the case that the distribution of inventory table partitions is uneven (i.e. there is a target partition management node), balancing adjustment can be performed after other operations (splitting, merging, split balancing) are completed.

[0164] The above two policies do not conflict with each other and can be started at the same time.

[0165] It should be noted that the policy configurator and the parameter configurator can simultaneously configure all database tables in the target database, or can individually configure each database table.

[0166] The collector 211 is responsible for collecting partition information in a timely manner, including: table name, partition management node name, load data (such as partition size, partition number, read request number, write request number, etc.), localization rate, etc.

[0167] The analyzer 212 is responsible for detecting the situations such as request surge and uneven distribution of partition load in a timely manner according to the relevant load data collected by the collector, and when detecting the relevant situations, it is timely handed over to the execution unit for disposal.

[0168] Referring toFigure 4 The overall flowchart of the monitoring unit provided in the present application is shown in FIG. 1.

[0169] Step S401, collect partition load data in time.

[0170] Step S402a, determine whether there is a first partition satisfying the partition split condition according to the collected load data of each partition where the target table is located, if yes, execute step S403a.

[0171] Step S402b, determine whether there is a second partition satisfying the partition merge condition according to the collected load data of each partition where the target table is located, if yes, execute step S403b.

[0172] Step S402c, determine whether there is a target partition management node satisfying the partition balancing condition in the partition management node corresponding to each partition according to the collected load data of each partition where the target table is located, if yes, execute step S403c.

[0173] Step S403a, add the first partition to the to-be-split list, and send the to-be-split list to the execution unit.

[0174] Step S403b, add the second partition to the to-be-merged list, and send the to-be-merged list to the execution unit.

[0175] Step S403c, generate information triggering the partition balancing operation of the target partition management node, and send the generated information to the execution unit.

[0176] The execution unit makes corresponding disposal according to the information obtained from the monitoring unit, including the disposal of the splitter, the merger and the balancer respectively.

[0177] The splitter 221 is responsible for splitting the first partition satisfying the partition split condition, specifically, traversing the to-be-split list, and sequentially splitting the first partition in the to-be-split list until the upper limit value of the number of partitions for daily execution split is reached.

[0178] Preferably, the first partition where the request surge is detected is split first, and then the remaining first partitions in the to-be-split list are split.

[0179] Optionally, the process of detecting whether there is a first partition satisfying the partition split condition in each partition can also be performed in the splitter, which is not limited here.

[0180] Referring to Figure 5 The overall flowchart of the splitter provided in the present application is shown in FIG. 2.

[0181] Step S501, sequentially traverse the first partition in the to-be-split list.

[0182] Step S502, splitting the first partition traversed in the to-be-split list.

[0183] Step S503, judging whether the upper limit value of the number of partitions performing splitting per day is reached, if yes, executing step S504, if no, returning to step S501.

[0184] Here, the first partition in step S502 is different each time.

[0185] Step S504, judging whether the splitting balance strategy is enabled, if yes, executing step S505.

[0186] Step S505, adding the split sub-partition to the to-be-migrated list.

[0187] The combiner 222 is responsible for merging the second partitions meeting the partition merging condition, specifically, traversing the to-be-merged list, and merging the second partitions in the to-be-merged list one by one until the upper limit value of the number of partitions performing merging per day is reached.

[0188] Optionally, the process of detecting whether there is a second partition meeting the partition merging condition in each partition can also be performed by the combiner, which is not limited here.

[0189] It should be noted that, in order to avoid affecting larger partitions, only when the two partitions adjacent in physical address are small enough can they be merged, and the partitions added to the to-be-split list through the burst detection in the day will not be added to the to-be-merged list.

[0190] Referring to Figure 6 The overall flowchart of the combiner provided in the present application.

[0191] Step S601, traversing the second partitions in the to-be-merged list one by one.

[0192] Step S602, merging the second partition traversed in the to-be-merged list.

[0193] Step S603, judging whether the upper limit value of the number of partitions performing merging per day is reached, if no, returning to step S601.

[0194] Here, the second partition in step S602 is different each time.

[0195] The balancer 223 is responsible for partition migration according to the configured splitting balance strategy and inventory balance strategy to achieve partition load balancing, and ensures that the total value of the partitions migrated to the partition management node is minimized when the partition is migrated. Specifically, traversing the to-be-migrated list,

[0196] Referring to Figure 7A general flowchart of the equalizer provided for the present application.

[0197] Step S701, it is judged whether there is a list to be migrated, if yes, step S702 is executed, if no, step S706 is executed.

[0198] Step S702, the sub-partitions in the list to be migrated are traversed one by one.

[0199] Step S703, for the sub-partition traversed in the list to be migrated, total generation values of the sub-partition migrated to each partition management node in other partition management nodes are calculated, a partition management node making the total generation value minimum is determined from the other partition management nodes, as a first to-be-migrated node, and the sub-partition is migrated to the first to-be-migrated node.

[0200] Step S704, it is judged whether all the sub-partitions in the list to be migrated are migrated, if no, step S702 is returned, if yes, step S705 is executed.

[0201] Step S705, it is judged whether the inventory equalization strategy is started, if yes, step S706 is executed.

[0202] Step S706, it is judged whether information triggering a partition equalization operation of a target partition management node is received, if yes, step S707 is executed.

[0203] Step S707, at least one partition is randomly selected from the target partition management node.

[0204] Step S708, for each partition in the at least one partition, total generation values of the partition migrated to each partition management node in other partition management nodes are calculated, a partition management node making the total generation value minimum is determined from the other partition management nodes, as a second to-be-migrated node, and the partition is migrated to the second to-be-migrated node.

[0205] Step S709, load data of the target partition management node after the partition migration is calculated.

[0206] Step S710, it is judged whether the load data of the target partition management node after the partition migration is less than a load balancing threshold, if no, step S707 is returned.

[0207] It should be noted that the specific implementation manners of each module in the partition adaptive equalization device of the database table can refer to the related introduction in the partition adaptive equalization method of the database table, which will not be repeated here.

[0208] It should also be noted that each module in the aforementioned partitioned adaptive balancing device for the database table can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the aforementioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each of the aforementioned modules.

[0209] An embodiment of the present application further provides an electronic device, which may include at least one processor and a memory connected to the processor, wherein:

[0210] Memory is used to store computer programs;

[0211] The processor is used to execute a computer program so that the electronic device can implement any partition adaptive balancing method of a database table provided in the embodiments of the present application.

[0212] refer to Figure 8 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present application. The electronic device in the embodiments of the present application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 8 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0213] like Figure 8 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 802 or programs loaded from a storage device 508 into a random access memory (RAM) 803. When the electronic device is powered on, the RAM 803 also stores various programs and data required for the operation of the electronic device. The processing device 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0214] Typically, the following devices may be connected to the I / O interface 805: an input device 806 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 808 including, for example, a memory card, a hard disk, etc.; and a communication device 809. The communication device 809 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Figure 8Electronic devices with various pieces of equipment are shown, but it should be understood that not all of the equipment shown is required to be implemented or present. More or less equipment can alternatively be implemented or present.

[0215] The embodiment of the present application further provides a computer program product comprising computer readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the database table partition adaptive balancing methods provided by the embodiment of the present application.

[0216] The embodiment of the present application further provides a computer readable storage medium, which carries one or more computer programs, which, when executed by an electronic device, can cause the electronic device to implement any of the database table partition adaptive balancing methods provided by the embodiment of the present application.

[0217] In addition, it should be noted that the apparatus embodiments described above are merely illustrative, and the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment. In addition, the connection relationship between the modules in the apparatus embodiment provided by the present application indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines.

[0218] From the above description of the embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. Generally, functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure for implementing the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better embodiment. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., including a plurality of instructions to make a computer device (which can be a personal computer, training device, or network device, etc.) execute the methods described in various embodiments of the present application.

[0219] In the above embodiments, the implementation can be wholly or partially by software, hardware, firmware, or any combination thereof. When implemented by software, the implementation can be wholly or partially in the form of a computer program product.

[0220] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer, training device or data center to another website, computer, training device or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. integrated with one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)), etc.

Claims

1. A partition adaptive balancing method for a database table, characterized in that: include: Obtain the load data of each partition of the target table to be balanced in the target database; Determine, based on the load data of each partition, whether there is a partition in each partition that meets a preset partition splitting condition and a partition merging condition, and determine whether there is a node in the partition management nodes corresponding to each partition that meets a preset partition balancing condition, wherein the partition splitting condition and the partition merging condition are used to identify partitions with an uneven load distribution, and the partition balancing condition is used to identify partition management nodes with an uneven load distribution; If there is a first partition that meets the partition splitting condition, performing splitting and balancing processing on the first partition, the splitting and balancing processing includes splitting the first partition and / or migrating the split sub-partitions to other partition management nodes; If there is a second partition that meets the partition merging condition, merging and balancing the second partition; If there is a target partition management node that meets the partition balancing condition, then after the split balancing process and the merge balancing process are completed, at least one partition is randomly selected from the partitions corresponding to the target partition management node, and the selected partition is migrated to other partition management nodes.

2. The partition adaptive balancing method for a database table according to claim 1, characterized in that: The partition splitting conditions include one or more of the following conditions: The partition size is larger than the preset split threshold; The partition size is greater than a first ratio threshold, where the first ratio threshold is an average value of the partition sizes of the partitions multiplied by a preset split ratio; The incremental read and write requests for the partition meet the preset incremental conditions; And / or, the partition merging condition includes one or more of the following conditions: The sizes of two partitions with adjacent physical addresses are both smaller than the preset merge threshold; The sizes of the two partitions with adjacent physical addresses are both smaller than a second ratio threshold, where the second ratio threshold is an average value of the partition sizes of the partitions multiplied by a preset merging ratio.

3. The partition adaptive balancing method for a database table according to claim 1, characterized in that: The process of determining whether there is a node that meets a preset partition balance condition among the partition management nodes corresponding to each partition includes: Determine the load data of all the partition management nodes corresponding to each partition according to the load data of each partition; Determine a load standard deviation and a load mean according to the load data of each of the partition management nodes; Determine a total normalized value of each of the partition management nodes according to the load data of each of the partition management nodes, the load standard deviation, and the load mean; Determine whether there is a node among all the partition management nodes whose total normalized value is greater than a preset load balancing threshold, wherein the node whose total normalized value is greater than the load balancing threshold is the node that meets the preset partition balancing condition.

4. The partition adaptive balancing method for a database table according to claim 1, characterized in that: The splitting and balancing of the first partition includes: Splitting the first partition to obtain sub-partitions; When a preset split balancing strategy is enabled, determining the load data of each partition management node in the other partition management nodes and the localization rate of the sub-partitions migrated to each partition management node; Calculate the total cost of migrating the sub-partition to each partition management node according to the load data of each partition management node and the localization rate of migrating the sub-partition to each partition management node; Determine, from the other partition management nodes, a partition management node that minimizes the total cost value as the node to be migrated; Migrate the subpartition to the node to be migrated.

5. The partition adaptive balancing method for a database table according to claim 4, characterized in that: The load data includes a load value of at least one load indicator; The calculating, based on the load data of each partition management node and the localization rate of the sub-partition migrated to each partition management node, a total cost of migrating the sub-partition to each partition management node, includes: Determining, according to the load data of each partition management node, a load cost value corresponding to the at least one load indicator on each partition management node; Determining a locality cost on each partition management node according to a locality rate of the sub-partition migrated to each partition management node; The total cost of migrating the sub-partition to each partition management node is obtained according to the load cost value corresponding to the at least one load indicator on each partition management node and the locality cost value on each partition management node.

6. The partition adaptive balancing method for a database table according to claim 5, characterized in that: Obtaining a total cost of migrating the sub-partition to each partition management node according to the load cost value corresponding to the at least one load indicator on each partition management node and the locality cost value on each partition management node includes: Determine, according to the load cost value corresponding to the at least one load indicator on each partition management node and the localization rate cost value on each partition management node, the information entropy corresponding to the at least one load indicator and the information entropy corresponding to the localization rate indicator; Determining a cost weight corresponding to each of the at least one load indicators according to the information entropy corresponding to each of the at least one load indicators; Determining a cost weight corresponding to the localization rate indicator according to the information entropy corresponding to the localization rate indicator; Combine the cost weight corresponding to the at least one load indicator and the cost weight corresponding to the localization rate indicator, and perform weighted summation on the load cost value corresponding to the at least one load indicator on each partition management node and the localization rate cost value on each partition management node to obtain the total cost value of migrating the sub-partition to each partition management node.

7. A partition adaptive balancing device for a database table, characterized in that: include: A monitoring unit and an execution unit, wherein the monitoring unit includes a collector and an analyzer, and the execution unit includes a splitter, a merger, and an equalizer; The collector is used to obtain load data of each partition where the target table to be balanced is located in the target database; The analyzer is configured to determine, based on the load data of each partition, whether there is a partition in each partition that meets a preset partition splitting condition and a partition merging condition, and to determine whether there is a node in the partition management nodes corresponding to each partition that meets a preset partition balancing condition, wherein the partition splitting condition and the partition merging condition are used to identify partitions with an uneven load distribution, and the partition balancing condition is used to identify partition management nodes with an uneven load distribution; The splitter is configured to perform split balancing on the first partition if a first partition that meets the partition splitting condition exists, wherein the split balancing includes splitting the first partition and / or migrating the split sub-partitions to other partition management nodes; The merger is configured to merge and balance the second partition if there is a second partition that meets the partition merging condition; The balancer is used to randomly select at least one partition from the partition corresponding to the target partition management node if there is a target partition management node that meets the partition balancing condition, and migrate the selected partition to other partition management nodes after the split balancing processing and the merge balancing processing are completed.

8. The partition adaptive balancing device for a database table according to claim 7, characterized in that: Also includes: A configuration unit, comprising a parameter configurator and a policy configurator; The parameter configurator is used to configure parameters in the partition splitting condition, the partition merging condition, and the partition balancing condition; The policy configurator is used to configure a stock balancing strategy and / or a split balancing strategy. The split balancing strategy is used to perform split balancing on the first partition. The stock balancing strategy is used to randomly select at least one partition from the partition corresponding to the target partition management node after the split balancing and merge balancing are completed, and migrate the selected partition to other partition management nodes.

9. An electronic device, characterized in that: comprising at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is configured to execute the computer program so that the electronic device can implement the partition adaptive balancing method for a database table according to any one of claims 1 to 6.

10. A computer storage medium, characterized in that The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the partition adaptive balancing method of a database table as described in any one of claims 1 to 6.

Images