Implementation method of distributed database redistribution, database, server and medium

Abstract

The embodiment of the present specification discloses a distributed database redistribution implementation method, a database, a server and a storage medium. The distributed database redistribution implementation method comprises the following steps: when the number of data nodes changes, the hash distribution Hash value of the data node and the new mapping relationship of the data node are obtained; and the corresponding data node is re-assigned to the Hash value according to the new mapping relationship, which can improve the performance of the distributed database redistribution and reduce resource consumption.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of database technology, and in particular to a distributed database redistribution implementation method, a database, a server and a medium. BACKGROUND

[0002] A distributed database combines database and distributed technology, and organizes various database data nodes scattered in a geographical sense into a complete logical whole through a computer system and a network. The distributed database has the advantages of good scalability, horizontal expansion, vertical expansion, flexible data node addition and deletion, etc.

[0003] Most of the mainstream distributed databases on the market currently use a consistent hashing algorithm for distribution operation. Specifically, the entire hash value space is mapped into a virtual circular ring, the virtual circular ring is organized in a clockwise direction, each point on the virtual circular ring corresponds to a hash value, and all hash values on a continuous circular arc on the virtual circular ring correspond to a data node. Therefore, the distribution of data nodes is a cut of the virtual circular ring. When a data node is added or deleted, in order to maintain the continuity of the virtual circular ring, the circular arc segment corresponding to the changed data node needs to be shifted as a whole. The overall shift of the circular arc segment causes a large change in data distribution, resulting in a sharp increase in the amount of data to be migrated, which is much larger than the amount of data to be stored by the changed data node, resulting in poor redistribution performance and serious resource consumption. Therefore, how to improve the performance of distributed database redistribution and reduce resource consumption has become a problem to be solved. SUMMARY

[0004] An object of one or more embodiments of the present application is to provide a distributed database redistribution implementation method, a database, a server and a medium, which can improve the performance of distributed database redistribution and reduce resource consumption.

[0005] To solve the above technical problems, one or more embodiments of the present application are implemented as follows:

[0006] In a first aspect, a distributed database redistribution implementation method is provided, and the method comprises the following steps: when the number of data nodes changes, obtaining a Hash value of a hash distribution for the data nodes and a new mapping relationship of the data nodes; and reassigning corresponding data nodes for the Hash value according to the new mapping relationship.

[0007] In a second aspect, a distributed database redistribution implementation device based on a Hash table is provided, and the device comprises: an obtaining module, configured to obtain a Hash value and a new mapping relationship of data nodes when the number of the data nodes changes; and an assigning module, configured to reassign corresponding data nodes for the Hash value according to the new mapping relationship.

[0008] In a third aspect, a database is provided, which comprises an implementation device redistributed according to the distributed database redistribution method described above.

[0009] In a fourth aspect, a server is provided, which comprises a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing connection communication between the processor and the memory, and the program, when executed by the processor, realizes the steps of the implementation method of the distributed database redistribution method described above.

[0010] In a fifth aspect, a storage medium is provided for computer-readable storage, which stores one or more programs, and the one or more programs, when executed by one or more processors, realize the steps of the implementation method of the distributed database redistribution method described above.

[0011] As can be seen from the technical solutions provided by one or more embodiments of the present specification, the implementation method of the distributed database redistribution provided by the embodiments of the present specification modifies the mapping relationship between the Hash value and the data node when the number of data nodes changes and the stored data needs to be redistributed, where the data node is the changed data node, and then reassigns the corresponding data node to the Hash value according to the new mapping relationship. After reassigning the corresponding data node to the Hash value based on the new mapping relationship between the Hash value and the data node, the stored data corresponding to the Hash value is migrated to the corresponding data node. In this case, some Hash values corresponding to the data nodes do not change, and only the stored data corresponding to the data nodes that change needs to be moved, which reduces the migration amount of the stored data, improves the performance of the distributed database redistribution, and saves the disk space and reduces the resource consumption since only the stored data is migrated from the migrated-out data node to the migrated-in data node, and the consumption of the disk basically remains the original usage. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the one or more embodiments of the present specification or the prior art, the drawings needed in the description of the one or more embodiments or the prior art will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present specification, and those skilled in the art can also obtain other drawings according to these drawings without paying creative labor.

[0013] Figure 1 is a schematic diagram of the steps of the implementation method of the distributed database redistribution provided by the embodiments of the present specification.

[0014] Figure 2is a step schematic diagram of another implementation method of distributed database redistribution provided by an embodiment of the present specification.

[0015] Figure 3 is a step schematic diagram of yet another implementation method of distributed database redistribution provided by an embodiment of the present specification.

[0016] Figure 4 is an interaction schematic diagram between database modules of yet another implementation method of distributed database redistribution provided by an embodiment of the present specification.

[0017] Figure 5 is a step schematic diagram of yet another implementation method of distributed database redistribution provided by an embodiment of the present specification.

[0018] Figure 6 is a step schematic diagram of yet another implementation method of distributed database redistribution provided by an embodiment of the present specification.

[0019] Figure 7 is a step schematic diagram of yet another implementation method of distributed database redistribution provided by an embodiment of the present specification.

[0020] Figure 8 is a step schematic diagram of yet another implementation method of distributed database redistribution provided by an embodiment of the present specification.

[0021] Figure 9 is a step schematic diagram of yet another implementation method of distributed database redistribution provided by an embodiment of the present specification.

[0022] Figure 10 is a step schematic diagram of yet another implementation method of distributed database redistribution provided by an embodiment of the present specification.

[0023] Figure 11 is a step schematic diagram of yet another implementation method of distributed database redistribution provided by an embodiment of the present specification.

[0024] Figure 12 is a structure schematic diagram of an implementation device of distributed database redistribution provided by an embodiment of the present specification.

[0025] Figure 13 is a structure schematic diagram of another implementation device of distributed database redistribution provided by an embodiment of the present specification.

[0026] Figure 14 is a structure schematic diagram of a database provided by an embodiment of the present specification.

[0027] Figure 15 is a structure schematic diagram of another database provided by an embodiment of the present specification.

[0028] Figure 16 is a structural schematic diagram of a server provided by an embodiment of the present specification. DETAILED DESCRIPTION

[0029] In order for those skilled in the art to better understand the technical solutions in the present specification, the technical solutions in one or more embodiments of the present specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of the present specification. Obviously, the described one or more embodiments are only a part of the embodiments of the present specification, not all embodiments. Based on one or more embodiments of the present specification, all other embodiments obtained by those skilled in the art without creative labor should belong to the protection scope of the present document.

[0030] The implementation method of distributed database redistribution provided by the present specification can adjust the Hash value of the data node and the mapping relationship of the data node in real time for the Hash distribution when the number of data nodes needs to be redistributed, and then redistribute the Hash value to the changed data node based on the new mapping relationship after adjustment. Some Hash values may correspond to data nodes that do not change, and the storage data corresponding to these Hash values does not need to be migrated and only part of the data corresponding to the Hash value needs to be migrated. Therefore, the performance of the distributed database redistribution can be improved, and resource consumption can be reduced. The implementation method of distributed database redistribution provided by the present specification and each step thereof will be described in detail below.

[0031] The implementation method of distributed database redistribution provided by the present specification is suitable for a distributed database using Hash distribution, and the number of data nodes of the distributed database will be increased or decreased according to business needs.

[0032] Embodiment one

[0033] Referring to Figure 1 The figure shows a step schematic diagram of an implementation method of distributed database redistribution provided by an embodiment of the present specification. It should be understood that the entire Hash value space of the distributed database will not change, and the Hash value and the number of Hash values will not change. Usually, the data nodes of the distributed database change. The implementation method of distributed database redistribution provided by the present specification is a method for redistributing the data nodes that have changed and the storage data stored by the original data nodes after the data nodes have changed. The purpose is to minimize the amount of storage data migration, improve the performance of redistribution, and reduce resource consumption. The implementation method of distributed database redistribution provided by the present specification includes the following steps:

[0034] Step 100: When the number of data nodes changes, obtain the Hash value of the data nodes and the new mapping relationship of the data nodes of the Hash distribution;

[0035] When the distributed database needs to be expanded or reduced, the number of data nodes changes, and the mapping relationship of the Hash value and the data nodes needs to be modified to the mapping relationship of the Hash value and the changed data nodes. The mapping relationship of the Hash value and the data nodes is an important basis for the implementation method of redistribution provided by the embodiments of the present specification, rather than the data amount of the stored data stored in the data nodes as the basis for redistribution. The data nodes corresponding to each Hash value are determined, and the data nodes corresponding to the Hash value can be flexibly controlled. Only the stored data corresponding to a single Hash value needs to be migrated, and the stored data corresponding to the Hash value on the entire arc corresponding to the data node does not need to be migrated. Therefore, the overall offset situation does not occur, the migration amount of the stored data is reduced, and the performance of the redistribution of the distributed database is improved.

[0036] When the data is distributed by using the Hash table, a Hash value is calculated for each row of stored data according to the distribution key (the distribution key is one or more fields in the Hash table), and the data node corresponding to the Hash value is calculated by using a fixed distribution algorithm, that is, the data node to which the row of stored data belongs. The Hash value is used to confirm the data node to which the row of stored data belongs, and the Hash value does not belong to the original stored data and can be understood as the auxiliary information of the Hash table. The mapping relationship of the Hash value and the data node can be the existing mapping relationship, and the new mapping relationship is obtained in real time after the number of data nodes changes, or the mapping relationship of the Hash value and the data node is obtained in real time when the number of data nodes changes.

[0037] Step 110: Reallocate the corresponding data node for the Hash value according to the new mapping relationship;

[0038] When the number of data nodes changes, the Hash value is reallocated to all the changed data nodes, and the purpose of reallocating each Hash value to the new data node is to prepare for the subsequent migration of the stored data.

[0039] The implementation method of the redistribution of the distributed database provided by the embodiments of the present specification undoubtedly migrates the stored data corresponding to each Hash value to the data node corresponding to the Hash value according to the first mapping relationship: the mapping relationship of the Hash value and the data node and the second mapping relationship: the mapping relationship of the Hash value and the stored data.

[0040] By reallocating corresponding data nodes to the hash values ​​based on the new mapping relationship, the migration amount of stored data can be controlled to the amount of change in the stored data of the changing data nodes, based on the characteristics of distributed databases. This minimizes the amount of change in the stored data of the changing data nodes, reduces the amount of migrated stored data, and improves the redistribution performance of distributed databases.

[0041] The mapping relationship between hash values ​​and data nodes in a distributed database allows for the migration of stored data corresponding to a single hash value, reducing the amount of data migration. Each migration can move one or several hash values, increasing the speed of a single migration and improving the performance of distributed database redistribution. Furthermore, multiple migration batches can be paused and resumed at any time without affecting the normal operation of the distributed database. Redistribution can also utilize the idle time of the distributed database, helping to alleviate its pressure. Additionally, because data is migrated in multiple batches, disk space consumption is reduced, essentially maintaining the original disk usage without requiring additional disk space, thus saving disk space and reducing resource consumption.

[0042] See Figure 2 The diagram illustrates the overall process of sequentially executing single tasks in the task table to migrate and store data in batches, as part of the distributed database redistribution implementation method provided in this embodiment.

[0043] Step S10: The metadata server MDS receives a hash redistribution request;

[0044] Step S200: MDS generates a redistribution task table;

[0045] Step S210: MDS retrieves a single task from the task table;

[0046] Step S220: MDS executes a single task;

[0047] Step S222: If the single task is successful, update the table structure information;

[0048] Step S221: If a single task fails, record the error message;

[0049] Step S230: After a single task ends, check if the task table is empty;

[0050] Step S250: No tasks, task redistribution ends;

[0051] Step S240: After continuing the single task operation, jump to step S220 until step S250.

[0052] See Figure 3As shown, in some embodiments, before the Hash value and the new mapping relationship between the data nodes are obtained when the number of data nodes changes, the implementation method of the distributed database redistribution provided by the embodiments of the present specification further includes:

[0053] Step 120: establishing the mapping relationship between the Hash value and the data nodes based on the Hash table of the Hash distribution for the data nodes;

[0054] When the mapping relationship between the Hash value and the data nodes needs to be created, the mapping relationship between the Hash value and the data nodes is established based on the Hash table of the Hash distribution for the data nodes, and the form of the mapping relationship is not limited and can be in the form of a table.

[0055] When the distributed database is redistributed, the mapping relationship between the Hash value and the data nodes is initialized based on the Hash table first, and the mapping relationship between the Hash value and the data nodes can be initialized as a key-value relationship table similar to the key being the Hash value and the value being the data node number, which is stored in the table metadata information of the Hash table. In the future, the fixed algorithm will be discarded, and the data nodes are obtained by querying the mapping relationship. In this way, each Hash value corresponding to the data node is stored in the table metadata, which can point to any data node, so the distributed database redistribution is the redistribution of the Hash value.

[0056] Step 130: distributing the Hash value to the corresponding data node according to the mapping relationship.

[0057] The implementation method of the distributed database redistribution provided by the embodiments of the present specification is to increase the mapping relationship between the Hash value obtained by the consistent Hash algorithm and the data nodes, which is the mapping of the consistent Hash value and the data node number.

[0058] For step 120, the mapping relationship is created under the condition that there is no mapping relationship, and then each Hash value is distributed to the corresponding data node according to the created mapping relationship. If the number of data nodes changes in the future, only the data node information in the mapping relationship needs to be changed to obtain the new mapping relationship between the Hash value and the data nodes, and then the corresponding data nodes for the Hash value are redistributed according to the new mapping relationship. Therefore, steps 120 and 130 in the implementation method of the distributed database redistribution provided by the embodiments of the present specification are executed only once in the distributed database redistribution process, and in the future, only the data node information in the mapping relationship needs to be modified, and then the corresponding data nodes for the Hash value are redistributed according to the new mapping relationship.

[0059] Referring to Figure 4As shown is the interaction timing diagram between each subject of the distributed database in the implementation method of the distributed database redistribution provided by the embodiment of the present specification, as shown in the following figure: Figure 4 :

[0060] Step 101: the management data node OMM sends a redistribution request;

[0061] Step 101': the MDS returns a check success response;

[0062] Step 201: the MDS sends a disable request to the PROXY, and the data corresponding to the Hash value is disabled for reading and writing, only the storage data involved in the disabled data will affect the access service, which reduces the influence range of the access service compared with the prior art;

[0063] Step 201': the PROXY returns a disable response to the MDS;

[0064] Step 202: the MDS sends a migration data request to the PROXY;

[0065] Step 202': the PROXY returns a migration data response to the MDS;

[0066] Step 203: the MDS sends an update table structure request to the PROXY, that is, the table structure stored in the metadata, and the mapping relationship is stored therein;

[0067] Step 203': the PROXY returns an update table structure response to the MDS;

[0068] Step 204: the MDS sends an unban request to the PROXY, and unban the data corresponding to the Hash value;

[0069] Step 204': the PROXY returns an unban response to the MDS;

[0070] Referring to Figure 5 As shown is the flow chart of the implementation method of the distributed database redistribution provided by the embodiment of the present specification. Specifically, as shown below:

[0071] Step 101: the MDS receives a Hash redistribution request;

[0072] Step 102: the MDS queries whether there is a mapping relationship;

[0073] Step 103: there is a mapping relationship (not the first redistribution);

[0074] Step 103: calculate the Hash value to be migrated;

[0075] Step 240: return after executing the task table;

[0076] Step 103: no mapping relationship (first redistribution);

[0077] Step 120: establish the mapping relationship between Hash value and data node;

[0078] Step 130: calculate the Hash value of the data to be migrated;

[0079] Step 240: return after executing the task table.

[0080] As Figure 6 shown, in some embodiments, before Step 120: establishing the mapping relationship between Hash value and data node based on the Hash table, the implementation method of redistribution provided by the embodiments of the present specification further comprises: Step 140: making the number of Hash values corresponding to each data node the same.

[0081] Before the first redistribution, that is, before establishing the mapping relationship between Hash value and data node based on the Hash table, keeping the number of Hash values of each data node the same, so that the number of Hash values corresponding to each data node in the mapping relationship between Hash value and data node created based on the Hash table is the same, if the Hash values are evenly distributed among the data nodes during the first redistribution, the redistribution operation can be simplified, the speed of redistribution can be accelerated, and the performance of the distributed database redistribution can be improved. For example, originally, each data node corresponds to 25 Hash values, and there are 4 data nodes. Now, one data node needs to be added. Then, only the 5 Hash values corresponding to the original 4 data nodes need to be redistributed to the new fifth data node. In this way, the 20 Hash values corresponding to the original 4 data nodes remain unchanged. In this way, the amount of storage to be migrated is small, and no additional disk space is needed.

[0082] Referring to Figure 7 shown, in some embodiments, before Step 120: establishing the mapping relationship between Hash value and data node based on the Hash table, the implementation method of redistribution provided by the embodiments of the present specification further comprises:

[0083] Step 150: keep the storage data evenly distributed on all data nodes.

[0084] The implementation method of the distributed database redistribution provided by the embodiments of the present specification does not take the amount of storage data as the basis for redistribution during the redistribution operation, but takes the number of Hash values corresponding to each data node as the basis. A suitable Hash algorithm is needed to ensure the uniformity of the distribution of storage data among the data nodes, that is, the size of the storage data corresponding to each Hash value remains basically the same.

[0085] Therefore, in the case that the number of Hash values corresponding to each data node is the same, a suitable Hash algorithm is needed to keep the uniformity of the distribution of stored data on all data nodes. The mapping relationship between Hash values and data nodes is calculated by a fixed algorithm before redistribution establishes a new mapping relationship, and the Hash value distribution of each data node is continuous.

[0086] Referring to Figure 8 As shown in the figure, in some embodiments, the implementation method for distributed database redistribution provided by the embodiments of the present specification includes the following steps.

[0087] Step 103: Set the same number of Hash values corresponding to each data node in the mapping relationship.

[0088] Setting the same number of Hash values corresponding to each data node when creating a new mapping relationship ensures that the number of Hash values migrated out and the number of Hash values migrated in are consistent during redistribution, and the amount of migrated storage data is minimized, based on the mapping relationship between Hash values and data nodes created by the Hash table in the first redistribution.

[0089] When determining the Hash values that need to be migrated, first calculate the number difference of Hash values of each data node before and after redistribution. If it is a positive value, it is the number of Hash values that need to be migrated out for this data node, and if it is a negative value, it is the number of Hash values that need to be migrated in for this data node. 0 indicates that the Hash values corresponding to this data node do not need to be migrated. Take out the Hash values that need to be migrated (the difference is positive), and take the minimum value according to the minimum value principle. Each time, the minimum Hash value is taken, and then it is assigned to the data node that needs to be migrated in. The data node is also sequentially migrated in the appropriate Hash value in the order from small to large. The number of migrated Hash values and the number of migrated Hash values need to be the same. In this way, the number of Hash values that need to be migrated is the same as the number of Hash values of the changed data nodes, reducing the amount of data migration.

[0090] After completing the redistribution, the migration of real data is completed, which will be described in the following embodiments.

[0091] Referring to Figure 9 As shown in the figure, in some embodiments, the implementation method for distributed database redistribution provided by the embodiments of the present specification includes the following steps.

[0092] Step 104: When the number of data nodes changes, determine the migration-out data nodes that need to migrate out hash values and the migration-in data nodes that need to migrate in hash values, the number of hash values that need to migrate out is the same as the number of hash values that need to migrate in;

[0093] If the original hash values corresponding to each data node are different, for example, the first data node corresponds to 20 hash values, the second data node corresponds to 25 hash values, the third data node corresponds to 30 hash values, and the fourth data node corresponds to 25 hash values, now one data node is added, and the same number of hash values is set for each data node, then each data node corresponds to 20 hash values, then the first data node does not need to migrate in hash values, the second data node needs to migrate out 5 hash values, the third data node needs to migrate out 10 hash values, the fourth data node needs to migrate out 5 hash values, and the fifth data node needs to migrate in 20 hash values, so the first migration is to migrate the 5 hash values that the second data node needs to migrate out to the fifth data node (the ranking of the second data node is less than the ranking of the third data node), after the 10 hash values that the third data node needs to migrate out are migrated to the fifth data node, the 5 hash values that the fourth data node needs to migrate out are migrated to the fifth data node. The number of migrated hash values and the number of migrated hash values are the same, which reduces the amount of data migration.

[0094] For example, each data node originally corresponds to 25 hash values, and there are 4 data nodes, now one data node needs to be added, then the original 4 data nodes need to migrate out the corresponding 5 hash values to the new fifth data node, so the original 4 data nodes correspond to 20 hash values which remain unchanged, so the number of hash values that need to be migrated is the same as the number of hash values of the changed data node (the fifth data node), the amount of storage migration is smaller, and no additional disk space is needed. Step 105: Reallocate the hash values that need to be migrated out of the migration-out data nodes to the migration-in data nodes.

[0095] Reallocating the hash values that need to be migrated out to the migration-in data nodes completes the reallocation of the hash values and data nodes of the distributed database redistribution, and then the storage data corresponding to the hash values that need to be migrated out is migrated to the corresponding data nodes.

[0096] If the hash values ​​of each data node are the same before the first redistribution, the number of hash values ​​migrated out and the number of hash values ​​migrated into the new mapping relationship can be kept the same. In this way, the number of hash values ​​that need to be migrated is the same as the number of hash values ​​corresponding to the changed data nodes, which reduces the amount of data migration.

[0097] See Figure 10 As shown, in some embodiments, after step 110: reallocating the corresponding data node to the Hash value according to the new mapping relationship, the distributed database redistribution implementation method provided in this specification embodiment further includes:

[0098] Step 200: Generate the corresponding task table, which includes at least the hash value, the data node to be migrated out, and the data node to be migrated in;

[0099] The redistribution task table contains three pieces of information: hash value, outgoing data node, and incoming data node. After the redistribution is completed, the actual storage data is migrated.

[0100] Step 220: According to the task table, migrate the storage data corresponding to the hash value from the migrated-out data node to the migrated-in data node;

[0101] The migration of the storage data corresponding to the hash value is completed by migrating the data to the receiving data node and then deleting the original storage data corresponding to the hash value from the sending data node. This process is repeated until all the storage data corresponding to the hash values ​​in the task table are migrated, thus completing the distributed database redistribution.

[0102] The stored data corresponding to each hash value can be determined by the hash value in the last column of the hash table.

[0103] Step 222: Delete the stored data corresponding to the hash value on the migrated data node.

[0104] Data migration can be performed in multiple steps, with each migration covering one or more stored data corresponding to a hash value. Each migration is independent of the others, and a single failure does not affect the overall redistribution. Furthermore, multiple redistribution tasks can be initiated until the stored data on each data node is evenly distributed.

[0105] See Figure 11 As shown, in some embodiments, before step 220: migrating the stored data corresponding to the Hash value from the migrated-out data node to the migrated-in data node according to the task table, the distributed database redistribution implementation method provided in the embodiments of this specification further includes:

[0106] Step 111: Implicitly insert the hash value into the hash table;

[0107] In order to redistribute the migrated data according to the Hash value, a column is implicitly added at the end of the Hash table for storing the Hash value, and the Hash value is also implicitly inserted when the data is inserted.

[0108] Step 241: Obtain the storage data corresponding to the Hash value based on the Hash value in the Hash table.

[0109] When migrating the storage data, the storage data corresponding to the Hash value is obtained based on the Hash value implicitly inserted in the Hash table, and the storage data is migrated. Step 111: The Hash value is implicitly inserted into the Hash table, which needs to perform two steps, one is the Hash table building process as follows:

[0110] 1. The PROXY receives the table building statement

[0111] 2. Determine the type of table building statement

[0112] 2.1. Hash table

[0113] 2.1.1. Implicitly add a column at the end of the table building statement, which is used to store the Hash value

[0114] 2.1.2. Issue a SQL for adding a column to inform the single data node database of the position of the Hash value

[0115] 2.2. Not Hash table

[0116] 2.2.1. Keep the original process unchanged

[0117] Two is the process of inserting Hash value in Hash table:

[0118] 1. The PROXY receives the table building statement

[0119] 2. Determine the type of table building statement

[0120] 2.1. Hash table

[0121] 2.1.1. Calculate the Hash value according to the distribution key

[0122] 2.1.2. Modify the insert statement to add a column at the end to insert the Hash value, which is used to query the storage data corresponding to the Hash value and determine the data during migration.

[0123] 2.1.3. Issue a SQL, which is a SQL with Hash value to inform the single data node database

[0124] 2.2. Not Hash table

[0125] 2.2.1. Keep the original process unchanged.

[0126] As can be seen from the above analysis, the distributed database redistribution method provided in this specification modifies the mapping relationship between hash values ​​and data nodes when the number of data nodes changes and data redistribution is required. Here, the data nodes refer to the changed data nodes. Then, based on the new mapping relationship, the corresponding data nodes are reassigned to the hash values. After reassigning the corresponding data nodes based on the hash value and data node mapping relationship, the stored data corresponding to the hash value is migrated to the corresponding data node. In this case, some data nodes corresponding to hash values ​​may not have changed; only the stored data whose corresponding data nodes have changed needs to be moved, reducing the amount of stored data migration and improving the performance of distributed database redistribution. Since only the stored data is migrated from the outgoing data node to the incoming data node, disk consumption remains essentially the same, saving disk space and reducing resource consumption.

[0127] Example 2

[0128] like Figure 12 The diagram shown is a structural schematic of a distributed database redistribution implementation apparatus 100 provided in an embodiment of this specification. The distributed database redistribution implementation apparatus includes:

[0129] The acquisition module 101 is used to acquire the new mapping relationship between the hash distribution and the hash value of the data node when the number of data nodes changes.

[0130] When a distributed database needs to be scaled up or down, the number of data nodes changes, requiring modification of the mapping relationship between hash values ​​and data nodes. This modification establishes a new mapping relationship between hash values ​​and the changed data nodes. The mapping relationship between hash values ​​and data nodes is used as a crucial basis for the redistribution implementation method provided in this specification's embodiments, rather than relying on the amount of data stored by each data node. By determining the data node corresponding to each hash value, flexible control over the corresponding data node is possible. Only the stored data corresponding to a single hash value needs to be migrated; there's no need to offset the entire arc of stored data corresponding to hash values ​​on the data node. Therefore, overall offsetting is avoided, reducing the amount of stored data migration and improving the performance of distributed database redistribution.

[0131] When data distribution is performed by using a Hash table, data in each row is stored according to a distribution key (the distribution key is one or more fields in the Hash table), a Hash value is calculated according to the distribution key, and a data node corresponding to the Hash value is calculated by using a fixed distribution algorithm, that is, the data node to which the data in the row belongs. The Hash value is used to determine the data node to which the data in the row belongs, and the Hash value is not part of the original stored data and can be understood as auxiliary information of the Hash table. The mapping relationship between the Hash value and the data node can be a previously existing mapping relationship, or the mapping relationship between the Hash value and the data node is obtained in real time when the number of data nodes changes.

[0132] The distribution module 102 is configured to reassign the data node corresponding to the Hash value according to the new mapping relationship.

[0133] When the number of data nodes changes, the Hash value is reassigned to all the changed data nodes, and the purpose of reassigning each Hash value to a new data node is to prepare for the subsequent migration of stored data.

[0134] The implementation method for the redistribution of the distributed database provided in the embodiments of the present specification undoubtedly migrates the stored data corresponding to each Hash value to the data node corresponding to the Hash value according to the first mapping relationship: the mapping relationship between the Hash value and the data node, and the second mapping relationship: the mapping relationship between the Hash value and the stored data.

[0135] According to the new mapping relationship, the Hash value is reassigned to the corresponding data node, and based on the characteristics of the distributed database, the amount of migrated stored data can be controlled to be the amount of change of the stored data of the changed data node, and the amount of change of the stored data of the changed data node is minimized as much as possible, the amount of migrated stored data is reduced, and the performance of the redistribution of the distributed database is improved.

[0136] Based on the mapping relationship between the Hash value and the data node of the distributed database, the stored data corresponding to a single Hash value can be migrated, the amount of migrated stored data is reduced, and each migration can migrate the stored data corresponding to one Hash value or several Hash values, which can improve the speed of single migration, improve the performance of the redistribution of the distributed database, and in addition, multiple batches of migration can be paused and continued at any time, without affecting the normal work of the distributed database, and the idle time of the distributed database can be used for redistribution, which is beneficial to reduce the pressure of the distributed database. In addition, since the data is subdivided into multiple batches for migration, the consumption of disk space is reduced, the consumption of the disk remains basically unchanged, no additional disk space is needed, disk space is saved, and resource consumption is reduced.

[0137] Referring to Figure 13 As shown in some embodiments, the implementation device for distributed database redistribution provided by the embodiments of the present application further comprises:

[0138] The initialization module 103 is configured to establish a mapping relationship between the Hash value and the data node based on the Hash operation on the Hash table of the data node.

[0139] The mapping relationship needs to be created when there is no mapping relationship between the Hash value and the data node, and the mapping relationship is established based on the Hash table. The form of the mapping relationship is not limited and can be in the form of a table.

[0140] During the redistribution of the distributed database, the mapping relationship between the Hash value and the data node is first initialized based on the Hash table, and the mapping relationship between the Hash value and the data node can be initialized as a key-value relationship table similar to the key-value relationship table, the key is the Hash value, and the value is the data node number. The table metadata information is stored in the Hash table. In the future, the fixed algorithm will be discarded, and the data node is obtained from the mapping relationship. In this way, each Hash value corresponding to the data node is stored in the table metadata, and can point to any data node. Therefore, the redistribution of the distributed database is the redistribution of the Hash value.

[0141] The distribution module 102 is configured to distribute the Hash value to the corresponding data node according to the mapping relationship.

[0142] The implementation method for distributed database redistribution provided by the embodiments of the present application is to increase the mapping relationship between the Hash value obtained by using the consistent Hash algorithm of the Hash distribution and the data node, which is the mapping of the corresponding relationship between the consistent Hash value and the data node number.

[0143] The step 120 is to create the mapping relationship under the condition that there is no mapping relationship, and then distribute each Hash value to the corresponding data node according to the created mapping relationship. If the number of data nodes changes in the future, only the data node information in the mapping relationship needs to be changed to obtain the new mapping relationship between the Hash value and the data node, and then the corresponding data node for the Hash value is redistributed according to the new mapping relationship. Therefore, the steps 120 and 130 in the implementation method for distributed database redistribution provided by the embodiments of the present application are executed only once in the process of distributed database redistribution, and in the future, only the data node information in the mapping relationship needs to be modified, and then the corresponding data node for the Hash value is redistributed according to the new mapping relationship.

[0144] It can be seen from the above analysis that the implementation method for distributed database redistribution provided by the embodiments of the present specification, when the number of data nodes changes and the stored data needs to be redistributed, modifies the mapping relationship between the Hash value of the data node and the data node, and then reassigns the corresponding data node for the Hash value according to the new mapping relationship. After reassigning the corresponding data node for the Hash value based on the new mapping relationship between the Hash value and the data node, the stored data corresponding to the Hash value is migrated to the corresponding data node. In this case, some data nodes corresponding to the Hash value do not change, and only the stored data corresponding to the data nodes that change needs to be moved, which reduces the migration amount of the stored data and improves the performance of the distributed database redistribution. Since only the stored data is migrated from the migrated-out data node to the migrated-in data node, the consumption of the disk basically remains the same as the original usage, which saves the disk space and reduces resource consumption.

[0145] Embodiment Three

[0146] Referring to Figure 14 Fig. 1 is a structural schematic diagram of a database 10 provided by the embodiments of the present specification. The distributed database redistribution implementation device 100 in the database 10 includes an acquisition module 101 and an assignment module 102, wherein:

[0147] The acquisition module 101 is configured to acquire a new mapping relationship between the Hash value of the data node and the data node when the number of data nodes changes.

[0148] When the distributed database needs to be expanded or contracted, the number of data nodes changes, and the mapping relationship between the Hash value and the data node needs to be modified to a new mapping relationship between the Hash value and the changed data node. The mapping relationship between the Hash value and the data node is used as an important basis for the redistribution implementation method provided by the embodiments of the present specification, rather than using the data amount of the stored data stored in the data node as the basis for redistribution. The data node corresponding to each Hash value is determined, and the data node corresponding to the Hash value can be flexibly controlled. Only the stored data corresponding to a single Hash value needs to be migrated, and the stored data corresponding to the Hash values on the entire arc corresponding to the data node does not need to be migrated. Therefore, the overall offset does not occur, the migration amount of the stored data is reduced, and the performance of the distributed database redistribution is improved.

[0149] When data distribution is performed by using a Hash table, data in each row is stored according to a distribution key (the distribution key is one or more fields in the Hash table), a Hash value is calculated according to the distribution key, and a data node corresponding to the Hash value is calculated by using a fixed distribution algorithm, that is, the data node to which the data in the row belongs. The Hash value is used to determine the data node to which the data in the row belongs, and the Hash value is not part of the original stored data and can be understood as auxiliary information of the Hash table. The mapping relationship between the Hash value and the data node can be a previously existing mapping relationship, or the mapping relationship between the Hash value and the data node is obtained in real time when the number of data nodes changes.

[0150] The distribution module 102 is configured to re-distribute the data nodes corresponding to the Hash values according to the new mapping relationship.

[0151] When the number of data nodes changes, the Hash values are re-distributed to all the changed data nodes, and the purpose of re-distributing each Hash value to a new data node is to prepare for the subsequent migration of stored data.

[0152] The implementation device for distributed database redistribution provided by the embodiments of the present specification undoubtedly migrates the stored data corresponding to each Hash value to the data node corresponding to the Hash value according to the first mapping relationship: the mapping relationship between the Hash value and the data node, and the second mapping relationship: the mapping relationship between the Hash value and the stored data.

[0153] According to the new mapping relationship, the data nodes corresponding to the Hash values are re-distributed, and based on the characteristics of the distributed database, the migration amount of the stored data can be controlled to be the change amount of the stored data of the changed data nodes, the change amount of the stored data of the changed data nodes is minimized as much as possible, the migration amount of the stored data is reduced, and the distributed database redistribution performance is improved.

[0154] Based on the mapping relationship between the Hash value and the data node of the distributed database, the stored data corresponding to a single Hash value can be migrated, the migration amount of the stored data is reduced, and each migration can migrate the stored data corresponding to one Hash value or several Hash values, which can improve the speed of single migration, improve the performance of distributed database redistribution, and in addition, multiple batches of migration can be paused and continued at any time, without affecting the normal work of the distributed database. In addition, the idle time of the distributed database can be used for redistribution, which is conducive to reducing the pressure of the distributed database. In addition, since the data is subdivided into multiple batches for migration, the consumption of disk space is reduced, the consumption of the disk remains basically unchanged, no additional disk space is needed, disk space is saved, and resource consumption is reduced.

[0155] Referring to Figure 15 As shown in the figure, it is a structural schematic diagram of the database 10 provided by the embodiment of the present application. The implementation device 100 of the redistribution of the distributed database further comprises an initialization module 103, wherein:

[0156] The initialization module 103 is configured to establish a mapping relationship between the Hash value and the data node based on the Hash table; and

[0157] The mapping relationship between the Hash value and the data node needs to be created when there is no mapping relationship between the Hash value and the data node, and the mapping relationship between the Hash value and the data node is established based on the Hash table. The form of the mapping relationship is not limited and can be in the form of a table.

[0158] During the redistribution of the distributed database, firstly, the mapping relationship between the Hash value and the data node is initialized based on the Hash table. The mapping relationship between the Hash value and the data node can be initialized as a key-value relationship table similar to the key-value relationship table. The key is the Hash value and the value is the data node number. The table metadata information stored in the Hash table is stored in the table metadata information. In the following operation of the Hash table, the fixed algorithm will be discarded, and the data node is obtained from the mapping relationship. In this way, each Hash value corresponding to the data node is stored in the table metadata, and can point to any data node. Therefore, the implementation method of the redistribution of the distributed database is the redistribution of the Hash value.

[0159] The distribution module 102 is configured to distribute the Hash value to the corresponding data node according to the mapping relationship;

[0160] The implementation method of the redistribution of the distributed database provided by the embodiment of the present application is to increase the mapping relationship between the Hash value and the data node by using the consistent Hash algorithm of the Hash distribution, which is the mapping of the consistent Hash value and the data node number.

[0161] The step 120 is to create the mapping relationship under the condition that there is no mapping relationship, and then distribute each Hash value to the corresponding data node according to the created mapping relationship. If the number of data nodes changes in the subsequent process, only the data node information in the mapping relationship needs to be changed to obtain the new mapping relationship between the Hash value and the data node, and then the corresponding data node of the Hash value is redistributed according to the new mapping relationship. Therefore, the step 120 and the step 130 in the implementation method of the redistribution of the distributed database provided by the embodiment of the present application are executed only once in the process of the redistribution of the distributed database. In the subsequent process, only the data node information in the mapping relationship needs to be modified, and then the corresponding data node of the Hash value is redistributed according to the new mapping relationship.

[0162] It can be seen from the above analysis that the implementation method of the distributed database redistribution provided by the embodiments of the present specification, when the number of data nodes changes and the stored data needs to be redistributed, modifies the mapping relationship between the Hash value and the data nodes, where the data nodes are the changed data nodes, and then reassigns the corresponding data nodes for the Hash value according to the new mapping relationship. After reassigning the corresponding data nodes for the Hash value based on the mapping relationship between the Hash value and the data nodes, the stored data corresponding to the Hash value is migrated to the corresponding data nodes. In this case, some of the data nodes corresponding to the Hash value do not change, and only the stored data corresponding to the data nodes that change needs to be moved, which reduces the migration amount of the stored data and improves the performance of the distributed database redistribution. Since only the stored data is migrated from the migrated-out data nodes to the migrated-in data nodes, the consumption of the disk basically remains the original usage amount, the disk space is saved, and the resource consumption is reduced.

[0163] Embodiment Four

[0164] Referring to 16, the server 1600 provided by the embodiments of the present specification includes a memory 1620, a processor 1610, a program stored on the memory 1620 and executable on the processor 1610, and a data bus 1640 for realizing the connection and communication between the processor 1610 and the memory 1620. The program is executed by the processor 1610 to realize the steps of the implementation method of the redistribution as shown in Figures 1 to 11 The specific implementation steps are as follows:

[0165] Step 100: When the number of data nodes changes, obtain the new mapping relationship between the Hash value and the data nodes for the Hash distribution.

[0166] When the distributed database needs to be expanded or reduced, the number of data nodes changes, and the mapping relationship between the Hash value and the data nodes needs to be modified to the mapping relationship between the Hash value and the changed data nodes. The mapping relationship between the Hash value and the data nodes is used as an important basis for the implementation method of the redistribution provided by the embodiments of the present specification, rather than the data amount of the stored data stored in the data nodes as the basis for redistribution. The data nodes corresponding to each Hash value are determined, so that the data nodes corresponding to the Hash value can be flexibly controlled. Only the stored data corresponding to a single Hash value needs to be migrated, and the stored data corresponding to the Hash values on the entire arc corresponding to the data nodes does not need to be migrated. Therefore, the overall offset does not occur, the migration amount of the stored data is reduced, and the performance of the distributed database redistribution is improved.

[0167] When data distribution is performed by using a Hash table, data in each row is stored according to a distribution key (the distribution key is one or more fields in the Hash table), a Hash value is calculated according to the distribution key, and a data node corresponding to the Hash value is calculated by using a fixed distribution algorithm, that is, the data node to which the data in the row belongs. The Hash value is used to determine the data node to which the data in the row belongs, and the Hash value is not part of the original stored data and can be understood as auxiliary information of the Hash table. The mapping relationship between the Hash value and the data node can be a previously existing mapping relationship, or the mapping relationship between the Hash value and the data node is obtained in real time when the number of data nodes changes.

[0168] Step 110: Reallocate the data node corresponding to the Hash value according to the new mapping relationship.

[0169] When the number of data nodes changes, the Hash value is reallocated to all the changed data nodes, and the purpose of reallocating each Hash value to a new data node is to prepare for the subsequent migration of stored data.

[0170] The implementation method of the distributed database redistribution provided by the embodiments of the present specification undoubtedly migrates the stored data corresponding to each Hash value to the data node corresponding to the Hash value according to the first mapping relationship: the mapping relationship between the Hash value and the data node, and the second mapping relationship: the mapping relationship between the Hash value and the stored data.

[0171] According to the new mapping relationship, the Hash value is reallocated to the corresponding data node, and based on the characteristics of the distributed database, the migration amount of the stored data can be controlled to be the change amount of the stored data of the changed data nodes, and the change amount of the stored data of the changed data nodes is minimized as much as possible, the migration amount of the stored data is reduced, and the performance of the distributed database redistribution is improved.

[0172] Based on the mapping relationship between the Hash value and the data node of the distributed database, the stored data corresponding to a single Hash value can be migrated, the migration amount of the stored data is reduced, and each migration can migrate the stored data corresponding to one or several Hash values, which can improve the speed of single migration, improve the performance of the distributed database redistribution, and in addition, multiple batches of migration can be paused and continued at any time, without affecting the normal work of the distributed database. In addition, the idle time of the distributed database can be used for redistribution, which is conducive to reducing the pressure of the distributed database. In addition, since the data is subdivided into multiple batches for migration, the consumption of disk space is reduced, the consumption of the disk is basically maintained at the original usage amount, no additional disk space is required, disk space is saved, and resource consumption is reduced.

[0173] From the above analysis, it can be seen that the implementation method of the distributed database redistribution provided by the embodiments of the present specification, when the number of data nodes changes and the stored data needs to be redistributed, modifies the new mapping relationship between the Hash value and the data node, where the data node is the changed data node, and then reassigns the corresponding data node for the Hash value according to the new mapping relationship. After reassigning the corresponding data node for the Hash value based on the mapping relationship between the Hash value and the data node, the stored data corresponding to the Hash value is migrated to the corresponding data node. In this case, some Hash values corresponding to the data nodes do not change, and only the stored data corresponding to the data nodes that change needs to be moved, which reduces the migration amount of the stored data and improves the performance of the distributed database redistribution. Since only the stored data is migrated from the migrated-out data node to the migrated-in data node, the consumption of the disk basically remains the original usage, saving the disk space and reducing resource consumption.

[0174] Embodiment five

[0175] The storage medium provided by the embodiments of the present specification is used for computer readable storage, and the storage medium stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps of the implementation method of the redistribution as shown in the above embodiments, and the specific steps are as follows: Figures 1 to 11

[0176] Step 100: When the number of data nodes changes, obtain the new mapping relationship between the Hash value and the data node for the Hash distribution to the data node.

[0177] When the distributed database needs to be expanded or reduced, the number of data nodes changes, and the mapping relationship between the Hash value and the data node needs to be modified to the mapping relationship between the Hash value and the changed data node. The mapping relationship between the Hash value and the data node is used as an important basis for the implementation method of the redistribution provided by the embodiments of the present specification, rather than the data amount of the stored data stored in the data node as the basis for redistribution. The data node corresponding to each Hash value is determined, so that the data node corresponding to the Hash value can be flexibly controlled. Only the stored data corresponding to a single Hash value needs to be migrated, and the stored data corresponding to the Hash values on the entire arc corresponding to the data node does not need to be migrated. Therefore, the overall offset does not occur, the migration amount of the stored data is reduced, and the performance of the distributed database redistribution is improved.

[0178] ​When data distribution is performed by using a Hash table, data in each row is stored according to a distribution key (the distribution key is one or more fields in the Hash table), a Hash value is calculated according to the distribution key, and a data node corresponding to the Hash value is calculated by using a fixed distribution algorithm, that is, the data node to which the data in the row belongs. The Hash value is used to determine the data node to which the data in the row belongs, and the Hash value is not part of the original stored data and can be understood as auxiliary information of the Hash table. The mapping relationship between the Hash value and the data node can be a previously existing mapping relationship, or the mapping relationship between the Hash value and the data node is obtained in real time when the number of data nodes changes.

[0179] Step 110: Reallocate the data node corresponding to the Hash value according to the new mapping relationship.

[0180] When the number of data nodes changes, the Hash value is reallocated to all the changed data nodes, and the purpose of reallocating each Hash value to a new data node is to prepare for the subsequent migration of stored data.

[0181] The implementation method of the distributed database redistribution provided by the embodiments of the present specification undoubtedly migrates the stored data corresponding to each Hash value to the data node corresponding to the Hash value according to the first mapping relationship: the mapping relationship between the Hash value and the data node, and the second mapping relationship: the mapping relationship between the Hash value and the stored data.

[0182] According to the new mapping relationship, the Hash value is reallocated to the corresponding data node, and based on the characteristics of the distributed database, the migration amount of the stored data can be controlled to be the change amount of the stored data of the changed data nodes, and the change amount of the stored data of the changed data nodes is minimized as much as possible, the migration amount of the stored data is reduced, and the performance of the distributed database redistribution is improved.

[0183] Based on the mapping relationship between the Hash value and the data node of the distributed database, the stored data corresponding to a single Hash value can be migrated, the migration amount of the stored data is reduced, and each migration can migrate the stored data corresponding to one Hash value or several Hash values, which can improve the speed of single migration, improve the performance of the distributed database redistribution, and in addition, multiple batches of migration can be paused and continued at any time, without affecting the normal work of the distributed database. In addition, the idle time of the distributed database can be used for redistribution, which is conducive to reducing the pressure of the distributed database. In addition, since the data is subdivided into multiple batches for migration, the consumption of disk space is reduced, the consumption of the disk is basically maintained at the original usage amount, no additional disk space is required, disk space is saved, and resource consumption is reduced.

[0184] It can be seen from the above analysis that the implementation method of the distributed database redistribution provided by the embodiments of the present specification, when the number of data nodes changes and the stored data needs to be redistributed, the mapping relationship between the Hash value and the data node is modified, the data node here is the changed data node, and then the corresponding data node is re-allocated for the Hash value according to the new mapping relationship. After re-allocating the corresponding data node for the Hash value based on the mapping relationship between the Hash value and the data node, the stored data corresponding to the Hash value is migrated to the corresponding data node. In this case, some Hash values corresponding to the data node do not change, and only the stored data corresponding to the data node that changes needs to be moved, which reduces the migration amount of the stored data and improves the performance of the distributed database redistribution. Since only the stored data is migrated from the migrated-out data node to the migrated-in data node, the consumption of the disk basically remains the original usage amount, the disk space is saved, and the resource consumption is reduced.

[0185] In summary, the above only describes the preferred embodiments of the present specification, and is not used to limit the protection scope of the present specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the protection scope of the present specification.

[0186] The system, device, module or unit illustrated by one or more embodiments described above can be specifically implemented by a computer chip or entity, or by a product with certain functions. A typical implementation device is a computer. Specifically, the computer may, for example, be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0187] Computer-readable storage media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassette, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0188] It should also be noted that the terms "comprising," "including," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements recited, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the phrase "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0189] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the difference from other embodiments. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0190] The above describes specific embodiments of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve the desired result. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired result. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

Claims

1. A method for implementing distributed database redistribution, the method comprising the following steps: When the number of data nodes changes, obtain the new mapping relationship between the hash distribution and the hash value of the data node and the data node; The method redistributes the corresponding data nodes to the hash value based on the new mapping relationship; the method utilizes the idle time of the distributed database for redistribution; Specifically, when the number of data nodes changes, the data nodes that need to migrate out of hash values ​​and the data nodes that need to migrate in hash values ​​are determined. The number of hash values ​​that need to be migrated out is the same as the number of hash values ​​that need to be migrated in, so that the number of data nodes that need to be migrated is the same as the number of hash values ​​of the data nodes that are changing. The hash values ​​that need to be migrated out of the data nodes are then redistributed to the data nodes that need to be migrated in. When determining the hash values ​​that need to be migrated, calculate the difference in the number of hash values ​​for each data node before and after redistribution. If the difference is positive, it represents the number of hash values ​​that need to be migrated out for this data node; if the difference is negative, it represents the number of hash values ​​that need to be migrated in for this data node. 0 indicates that the hash value corresponding to this data node does not need to be migrated. Extract the hash values ​​that need to be migrated out, selecting the minimum value each time, and then assign the minimum hash value to the data node that needs to be migrated in. The method further includes, after reallocating the corresponding data node to the Hash value according to the modified mapping relationship: Generate a corresponding task table, which includes at least a hash value, outgoing data nodes, and incoming data nodes; The hash value is implicitly inserted into the hash table; The storage data corresponding to the hash value is obtained based on the hash value in the hash table; According to the task table, the stored data corresponding to the hash value is migrated from the migrated-out data node to the migrated-in data node; wherein, the data migration is carried out in multiple times, and each migration is independent of each other, and multiple batches of migration can be paused and resumed at any time; Delete the stored data corresponding to the hash value on the migrated data node.

2. The implementation method as described in claim 1, before obtaining the new mapping relationship between the Hash value and the data node when the number of data nodes changes, the method further includes: A mapping relationship between the hash value and the data node is established based on the hash table of the hash distribution; The hash value is assigned to the corresponding data node according to the mapping relationship.

3. The implementation method as described in claim 1 or 2, before establishing the mapping relationship between the hash value and the data node based on the hash table of the hash distribution, the method further includes: This ensures that the number of hash values ​​corresponding to each data node is the same.

4. The implementation method as described in claim 3, before establishing the mapping relationship between hash values ​​and data nodes based on the hash table, the method further includes: Maintain a uniform distribution of stored data across all data nodes.

5. The implementation method as described in claim 3, wherein when the number of data nodes changes, obtaining the new mapping relationship between the Hash value and the data nodes specifically includes: In the new mapping relationship, each data node is assigned the same number of hash values.

6. An apparatus for implementing distributed database redistribution, the apparatus comprising: The acquisition module is used to acquire the hash distribution for the hash value of the data node and the new mapping relationship of the data node when the number of data nodes changes; The allocation module is used to reallocate the corresponding data node to the Hash value according to the new mapping relationship; and to redistribute the data using the idle time of the distributed database. Specifically, the acquisition module is used to determine, when the number of data nodes changes, the data nodes that need to migrate out of hash values ​​and the data nodes that need to migrate in hash values. The number of hash values ​​to be migrated out is the same as the number of hash values ​​to be migrated in, so that the number of data nodes to be migrated is the same as the number of hash values ​​of the data nodes that have changed. The hash values ​​to be migrated out of the data nodes are redistributed to the data nodes to be migrated in. When determining the hash values ​​to be migrated, the difference in the number of hash values ​​of each data node before and after redistribution is calculated. If the difference is positive, it is the number of hash values ​​to be migrated out for this data node. If the difference is negative, it is the number of hash values ​​to be migrated in for this data node. 0 indicates that the hash value corresponding to this data node does not need to be migrated. The hash values ​​to be migrated out are taken according to the minimum value principle. The minimum hash value is taken each time and then assigned to the data nodes to be migrated in. The device is further configured to: after reallocating the corresponding data node to the Hash value according to the modified mapping relationship, generate a corresponding task table, the task table including at least the Hash value, the data node to be migrated out, and the data node to be migrated in; implicitly insert the Hash value into the hash table; obtain the storage data corresponding to the Hash value based on the Hash value in the hash table; migrate the storage data corresponding to the Hash value from the data node to the data node to be migrated in according to the task table; delete the storage data corresponding to the Hash value on the data node to be migrated out; wherein, the data migration is carried out in multiple migrations, and each migration is independent of each other, and multiple batch migrations can be paused and resumed at any time.

7. The implementing apparatus of claim 6, further comprising: The initialization module is used to establish a mapping relationship between hash values ​​and data nodes based on the hash table; as well as, The allocation module is used to allocate the Hash value to the corresponding data node according to the mapping relationship.

8. A database comprising the implementation apparatus according to claim 6 or 7.

9. A server comprising a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory, wherein the program, when executed by the processor, implements the steps of the distributed database redistribution method as described in any one of claims 1 to 5.

10. A storage medium for computer-readable storage, the storage medium storing one or more programs, which, when executed by one or more processors, implement the steps of the distributed database redistribution implementation method as described in any one of claims 1 to 5.

Images