Method and device for expanding and shrinking capacity of key value database and electronic equipment

Abstract

The invention provides a key value database capacity expansion and contraction method and device and electronic equipment, and aims to solve the problem that a key value database cluster has a data loss risk in the capacity expansion and contraction process. The method is applied to a key value database cluster, the key value database cluster comprises a plurality of application node groups, and slot data stored in a node hard disk is transmitted among a plurality of nodes in each application node group through a distributed consistency consensus algorithm. Obtaining to-be-migrated slot data from a node hard disk in the first application node group, and sending the to-be-migrated slot data to a second application node group in the plurality of application node groups so as to perform data capacity reduction on the first application node group; and receiving the to-be-migrated data sent by the first application node group through the second application node group, and storing the to-be-migrated data into a node hard disk in the second application node group so as to carry out data expansion on the second application node group.

Description

Technical Field

[0001] The present application relates to the technical field of key-value databases, server clusters, and data expansion and contraction, and in particular, to a key-value database expansion and contraction method, device, and electronic device. Background Art

[0002] The current key-value database cluster may face the risk of data loss when scaling up or down. For example, when scaling up or down, the Redis cluster needs to persist the data in the memory to the hard disk first. This persistence operation, such as RDB (Redis DataBase) snapshot or Append Only File (AOF) log recording, may cause downtime, so non-persistent data may be lost. Therefore, the current database cluster has the risk of data loss during the scaling process. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a method, device and electronic device for scaling a key-value database, so as to improve the risk of data loss during the scaling of a key-value database.

[0004] An embodiment of the present application provides a key-value database scaling method, which is applied to a key-value database cluster, wherein the key-value database cluster includes: multiple application node groups, each application node group includes multiple nodes, and the slot data stored in the node hard disk is transmitted between the multiple nodes through a distributed consistency consensus algorithm; the above-mentioned key-value database scaling method includes: multiple application node groups, each application node group includes multiple nodes, and the slot data stored in the node hard disk is transmitted between the multiple nodes through a distributed consistency consensus algorithm; the key-value database scaling method includes: through a first application node group among the multiple application node groups, obtaining the slot data to be migrated from the node hard disk in the first application node group, and sending the slot data to be migrated to the second application node group among the multiple application node groups, so as to scale down the data of the first application node group; through the second application node group, receiving the data to be migrated sent by the first application node group, and storing the data to be migrated in the node hard disk in the second application node group, so as to scale up the data of the second application node group. In the implementation process of the above scheme, the first application node group in the key-value database cluster obtains the slot data to be migrated from the data that has been persisted to the hard disk, rather than directly obtaining the slot data to be migrated from the memory, thereby ensuring that even if the slot data to be migrated fails during the transmission process, it can be restored by relying on the data on the hard disk, reducing the risk of data loss due to the failure to save temporary data in the memory in time. Furthermore, the second application node group that receives the slot data persists the received slot data on the hard disk, further ensuring the security of the data. Even if an unexpected situation occurs during the receiving process, it can be restored through the data on the hard disk, thereby further reducing the risk of data loss in the process of scaling the database cluster.

[0005] Optionally, in an embodiment of the present application, the key-value database cluster further includes: a metadata center group; before sending the slot data to be migrated to the second application node group in the multiple application node groups, it also includes: obtaining metadata information of the slot data to be migrated through the first application node group, and sending the metadata information to the metadata center group; receiving the metadata information sent by the first application node group through the metadata center group, and screening out the second application node group for storing the slot data to be migrated from the multiple application node groups according to the metadata information, and sending data migration commands to the first application node group and the second application data respectively, so that the first application node group sends the slot data to be migrated to the second application node group. In the implementation process of the above scheme, data migration is realized in an automated manner, reducing the need for manual intervention. The metadata center group is responsible for the coordination and management of the entire migration process, so that the operation and maintenance personnel do not need to intervene manually, reducing the operation and maintenance costs. Automated migration is achieved by cooperating with the metadata center group, the first application node group, and the second application node group, which can also reduce human errors and improve the reliability and success rate of migration.

[0006] Optionally, in an embodiment of the present application, before sending data migration commands to the first application node group and the second application data respectively, it also includes: blocking the write operation of the first application node group through the metadata center group. In the implementation process of the above scheme, the process of data migration can be simplified by blocking the write operation of the first application node group, because during the blocking period, the system will not process new write requests, which makes data migration simpler and controllable. This simplification helps to improve the reliability of migration and reduce errors that may occur during the migration process. Furthermore, during the data migration process, if the write operation is not blocked, data conflicts may occur due to concurrent write operations. The scheme of the embodiment of the present application can avoid the occurrence of such data conflicts by blocking the write operation, effectively reducing the possibility of data conflicts and improving the reliability of data migration.

[0007] Optionally, in an embodiment of the present application, the data migration command includes: compression parameters and segmentation parameters; sending the slot data to be migrated to the second application node group among the multiple application node groups, including: compressing the slot data to be migrated according to the compression parameters to obtain a compressed file; segmenting the compressed file according to the segmentation parameters to obtain multiple block files and serial numbers of the multiple block files; sending multiple block files to the second application node group, so that the second application node group combines the multiple block files into a compressed file, and decompresses the compressed file into the slot data to be migrated. In the implementation process of the above scheme, by performing a series of steps such as compressing, segmenting, transmitting, reorganizing, and decompressing the slot data to be migrated, not only can the time and bandwidth required for data transmission be significantly reduced, but also parallel transmission can be achieved by segmenting the compressed file into multiple block files. In addition, if a block fails to be transmitted, only the block needs to be retransmitted, and the entire file does not need to be retransmitted, thereby making full use of the network bandwidth and further improving the transmission efficiency of the slot data to be migrated.

[0008] Optionally, in an embodiment of the present application, after receiving the data to be migrated sent by the first application node group through the second application node group, it also includes: sending a migration completion notification message to the metadata center group through the second application node group, so that the metadata center group resumes the write operation of the second application node group after receiving the migration completion notification message. In the implementation process of the above scheme, by resuming the write operation after the migration is completed, the possibility of data conflicts can be reduced. Since no new write operations occur during the migration, the write operation can be resumed immediately after the migration is completed to ensure data consistency. This method reduces the possibility of data conflicts and improves the reliability of data migration. By resuming the write operation after the migration is completed, the fluctuation of the system during the data migration process can be reduced. After the migration is completed, the system can immediately resume the write operation to ensure the continuity of the service. This stability helps to ensure the smooth progress of the data migration process and reduce the time of service interruption.

[0009] Optionally, in an embodiment of the present application, the multiple nodes include a master node; the key-value database expansion and contraction method also includes: parsing the index keyword from the received Table command through the master node in the metadata center group, and finding the slot data identifier corresponding to the index keyword, and sending the Table command to the target application node group corresponding to the slot data identifier; parsing the sub-command type and index keyword from the Table command through the master node in the target application node group, and executing the Table command according to the sub-command type and index keyword. In the implementation process of the above scheme, by dynamically routing the Table command to the target application node group, the system has better scalability and flexibility. This dynamic routing mechanism can adapt to the ever-changing workload and network conditions, and improves the adaptability of the system. Furthermore, by parsing the sub-command type in the Table command through the master node in the target application node group, and executing the Table command according to the sub-command type and index keyword, complex query operations can be supported. This mechanism enables the system to handle more complex query requests, improving the functionality and practicality of the system.

[0010] Optionally, in an embodiment of the present application, the multiple nodes also include slave nodes; executing Table commands according to the sub-command type and index keywords includes: filtering out the target slave node from the multiple slave nodes in the target application node group according to the load data of each slave node through the master node in the target application node group, and sending the Table command and index keywords to the target slave node; determining the index name corresponding to the index keyword from the index tree stored in the local slot data through the target slave node. In the implementation process of the above scheme, by filtering out the target slave nodes with lower load to process the commands, it can be ensured that the commands are executed on nodes with higher processing efficiency. This efficient task allocation helps to reduce the time of command processing and improve the response speed of the system. Furthermore, by determining the index name corresponding to the index keyword from the index tree stored in the local slot data through the target slave node, the accuracy of command processing can be ensured. This precise positioning helps to reduce errors in the data processing process and improve the consistency and accuracy of the data.

[0011] Optionally, in an embodiment of the present application, the subcommand types include: Search subcommand and List subcommand; determining the index name corresponding to the index keyword from the index tree of the local slot data storage through the target slave node, including: if the subcommand type is a Search subcommand, searching the index name corresponding to the index keyword from the index tree of the local slot data storage through the target slave node; if the subcommand type is a List subcommand, enumerating all index names under the level where the index keyword is located from the index tree of the local slot data storage through the target slave node. In the implementation process of the above scheme, by supporting different types of Search subcommands and List subcommands, the system can handle diverse query requirements and enhance flexibility. Different query requirements can be implemented through different subcommands, which improves the versatility of the system. Furthermore, for the Search subcommand, the system only needs to find a specific index name, while for the List subcommand, the system needs to enumerate all index names within a certain range. This differentiated processing method can optimize resource usage according to actual needs and avoid unnecessary computing and storage overhead.

[0012] The embodiment of the present application also provides a key-value database expansion and contraction device, which is applied to a key-value database cluster, wherein the key-value database cluster includes: multiple application node groups, each application node group includes multiple nodes, and the slot data stored in the node hard disk is transmitted between the multiple nodes through a distributed consistency consensus algorithm; the key-value database expansion and contraction device includes: multiple application node groups, each application node group includes multiple nodes, and the slot data stored in the node hard disk is transmitted between the multiple nodes through a distributed consistency consensus algorithm; the key-value database expansion and contraction device includes: a slot data contraction module, which is used to obtain the slot data to be migrated from the node hard disk in the first application node group through the first application node group in the multiple application node groups, and send the slot data to be migrated to the second application node group in the multiple application node groups, so as to shrink the data of the first application node group; the slot data expansion module, which is used to receive the data to be migrated sent by the first application node group through the second application node group, and store the data to be migrated in the node hard disk in the second application node group, so as to expand the data of the second application node group.

[0013] Optionally, in an embodiment of the present application, the key-value database cluster also includes: a metadata center group; the key-value database expansion and contraction device also includes: a metadata information acquisition module, which is used to obtain metadata information of the slot data to be migrated through the first application node group, and send the metadata information to the metadata center group; a metadata receiving and utilization module, which is used to receive the metadata information sent by the first application node group through the metadata center group, and filter out the second application node group for storing the slot data to be migrated from multiple application node groups according to the metadata information, and send data migration commands to the first application node group and the second application data respectively, so that the first application node group sends the slot data to be migrated to the second application node group.

[0014] Optionally, in an embodiment of the present application, the key-value database expansion and contraction device further includes: a node write operation blocking module, which is used to block the write operation of the first application node group through the metadata center group.

[0015] Optionally, in an embodiment of the present application, the data migration command includes: compression parameters and segmentation parameters; the slot data shrinking module includes: a migration data compression sub-module, used to compress the slot data to be migrated according to the compression parameters to obtain a compressed file; a compressed file segmentation sub-module, used to segment the compressed file according to the segmentation parameters to obtain multiple block files and serial numbers of the multiple block files; a block file sending sub-module, used to send multiple block files to the second application node group, so that the second application node group combines the multiple block files into a compressed file, and decompresses the compressed file into the slot data to be migrated.

[0016] Optionally, in an embodiment of the present application, the key-value database expansion and contraction device also includes: a node write operation recovery module, which is used to send a migration completion notification message to the metadata center group through the second application node group, so that the metadata center group can restore the write operation of the second application node group after receiving the migration completion notification message.

[0017] Optionally, in an embodiment of the present application, the multiple nodes include a master node; the key-value database expansion and contraction device also includes: a Table command sending module, which is used to parse the index keyword from the received Table command through the master node in the metadata center group, and find the slot data identifier corresponding to the index keyword, and send the Table command to the target application node group corresponding to the slot data identifier; a Table command execution module, which is used to parse the sub-command type and index keyword from the Table command through the master node in the target application node group, and execute the Table command according to the sub-command type and index keyword.

[0018] Optionally, in an embodiment of the present application, the multiple nodes also include slave nodes; the Table command execution module includes: a slave node screening submodule, used to screen out target slave nodes from multiple slave nodes in the target application node group according to the load data of each slave node through the master node in the target application node group, and send Table commands and index keywords to the target slave nodes; an index name determination submodule, used to determine the index name corresponding to the index keyword from the index tree of the local slot data storage through the target slave node.

[0019] Optionally, in an embodiment of the present application, sub-command types include: Search sub-command and List sub-command; the index name determination sub-module includes: an index name search unit, which is used to search for the index name corresponding to the index keyword from the index tree of the local slot data storage through the target slave node if the sub-command type is a Search sub-command; an index name listing unit, which is used to list all index names under the level where the index keyword is located from the index tree of the local slot data storage through the target slave node if the sub-command type is a List sub-command.

[0020] An embodiment of the present application further provides an electronic device, including: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions execute the method described above when executed by the processor.

[0021] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described above is executed.

[0022] The embodiment of the present application also provides a computer program product, including: a computer program or a computer instruction, and the computer program or the computer instruction executes the method described above when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A schematic diagram of a flow chart of a key-value database expansion and contraction method provided in an embodiment of the present application is shown;

[0025] Figure 2 A schematic diagram of dividing and managing data by slots provided in an embodiment of the present application is shown;

[0026] Figure 3 A schematic diagram of an index tree for storing local slot data provided by an embodiment of the present application is shown;

[0027] Figure 4 A schematic diagram of the structure of a key-value database expansion and contraction device provided in an embodiment of the present application is shown;

[0028] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] To make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. It should be understood that the drawings in the embodiment of the present application only serve the purpose of explanation and description, and are not used to limit the protection scope of the embodiment of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in the embodiment of the present application shows the operation implemented according to some embodiments of the embodiment of the present application. It should be understood that the operation of the flowchart can be implemented out of order, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the embodiment of the present application, or remove one or more operations from the flowchart.

[0030] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all embodiments. The components of the embodiments of the present application described and shown in the drawings generally here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the embodiments of the present application claimed, but merely represents the selected embodiments of the embodiments of the present application.

[0031] It is understandable that the "first" and "second" in the embodiments of the present application are used to distinguish similar objects. Those skilled in the art will understand that the words "first" and "second" do not limit the quantity and execution order, and the words "first" and "second" do not necessarily limit the difference. In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that there may be three relationships, such as A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the related objects before and after are in an "or" relationship. The term "multiple" refers to more than two (including two), and similarly, "multiple groups" refers to more than two groups (including two groups).

[0032] Before introducing the key-value database expansion and contraction method provided in the embodiment of the present application, some concepts involved in the embodiment of the present application are first introduced:

[0033] A key-value database, also known as a key-value storage database, KV database or key-value database, is a database that stores and retrieves data in the form of key-value pairs. In this database, a data item consists of a unique key and an associated value. The key can be used to quickly access the corresponding value, which makes the KV database very efficient in data retrieval.

[0034] It should be noted that the key-value database expansion and contraction method provided in the embodiment of the present application can be executed by an electronic device, where the electronic device refers to a device terminal or server with the function of executing a computer program, such as a smart phone, a personal computer, a tablet computer, a personal digital assistant, or a mobile Internet device. A server refers to a device that provides computing services through a network, such as an x86 server and a non-x86 server, and a non-x86 server includes a mainframe, a minicomputer, and a UNIX server.

[0035] In the related art, a memory-based key-value database is usually used for high-performance data processing. For example, a memory-based key-value database such as Memcached and Redis can quickly respond to user requests and provide millisecond-level read and write performance. However, due to the limited memory capacity of the server, when the data capacity increases sharply and exceeds the memory capacity, it is impossible to continue to use the memory database to store all data. Therefore, it is necessary to persist the data in the memory to the hard disk first, or the memory database needs to persist the data to the hard disk regularly to prevent data loss, and this persistence operation itself may have the risk of data loss. Specifically, the above-mentioned persistence operation such as RDB snapshot or append-only file (AOF) logging may cause downtime, that is, during the snapshot process or logging process, the server may not persist the data in the memory to the hard disk due to a sudden power outage. Therefore, the current database cluster may have the risk of losing non-persistent data in the memory during the expansion and contraction process.

[0036] In order to overcome the above defects, the embodiment of the present application provides a method for scaling up and down a key-value database. The main idea of ​​the method is to use the node hard disk in the key-value database cluster to store data persistently, and manage and store data in the form of slots. When scaling up and down, data migration is performed through this slot mechanism to ensure data consistency and high availability. Specifically, even if the data in the slot to be migrated fails during transmission, or an unexpected situation occurs during the receiving process, it can be restored by relying on the data on the hard disk, reducing the risk of data loss caused by the failure to save temporary data in the memory in time.

[0037] See also Figure 1 The flowchart of the key-value database expansion and contraction method provided by the embodiment of the present application is shown; the method can be applied to a key-value database cluster, which includes: multiple application node groups, each application node group includes multiple nodes, and the slot data stored in the node hard disk is transmitted between the multiple nodes through a distributed consistency consensus algorithm. The distributed consistency consensus algorithm here includes but is not limited to: Raft algorithm, Paxos algorithm, Zab algorithm, Practical Byzantine Fault Tolerance (PBFT) algorithm or Total Order Broadcast (TOB) algorithm, etc. The implementation method of the above-mentioned key-value database expansion and contraction method may include:

[0038] Step S110: obtaining the slot data to be migrated from the node hard disks in the first application node group through the first application node group among the multiple application node groups, and sending the slot data to be migrated to the second application node group among the multiple application node groups to shrink the data of the first application node group.

[0039] It should be noted that the reason why the above-mentioned key-value database cluster uses node hard disks to store slot data is to reduce the burden of server nodes frequently using Fork operations and Copy-On-Write (COW) mechanisms in latency-sensitive business scenarios, while also reducing storage costs and reducing the frequency of memory usage. The above-mentioned Fork operation is an operation called in the Linux operating system to create a new process. The new process (child process) is a completely copied copy of the parent process, including memory mapping, file descriptors, etc. When the database node performs a Fork operation, a large number of memory pages need to be copied. Especially during latency-sensitive business peaks, frequent Fork operations (for example, when persisting data) may cause long-term blocking, thereby affecting the response time and overall performance of the system. The above-mentioned Copy-On-Write (COW) is a memory management technology. When the parent process and the child process share the same memory page, the operating system will only actually copy a page to the modifier when one of the processes attempts to modify the page. In a high-concurrency scenario, if multiple processes try to modify the same memory page at the same time, the operating system needs to frequently perform page copy operations, which consumes a lot of CPU resources. When the number of memory pages is large, the overhead of the COW mechanism becomes more significant, especially when the database needs to persist a large amount of data.

[0040] See also Figure 2 The schematic diagram of dividing and managing data by slots provided by the embodiment of the present application is shown; the figure shows that the above-mentioned key-value database cluster adopts a distributed architecture, and this distributed architecture is reflected in that the key-value database cluster includes N application node groups, including: application node group 1, application node group 2, ..., application node group N, wherein each application node group can manage multiple slot data, for example: application node group 1 manages the 0th to Xth slot data, while application node group 2 manages the Xth to Yth slot data, and application node group N manages the Yth to Nth slot data. In order to overcome the defects of reducing server nodes in the above latency-sensitive business scenarios, each node in each application node group in the above-mentioned key-value database cluster manages data in the form of slots, specifically, the data is divided and managed by slots, and each slot contains a certain amount of key-value KV data, so that data can be managed more efficiently, and data migration can be performed through the slot division mechanism when expanding or shrinking. In addition, data can be managed in a hierarchical storage manner in the node hard disk. For example, most of the data can be stored on the hard disk to reduce storage costs and memory usage, thereby reducing the burden brought by fork operations and COW mechanisms, while frequently accessed data can be stored in memory. Through such a data tiering mechanism, unnecessary memory copy operations can be reduced to ensure high-performance read and write performance.

[0041] Step S120: receiving the data to be migrated sent by the first application node group through the second application node group, and storing the data to be migrated in the node hard disk in the second application node group to expand the data capacity of the second application node group.

[0042] It can be understood that when storing the data to be migrated in the node hard disk in the second application node group, an asynchronous persistence mechanism can be used to asynchronously perform persistence operations on the data to be migrated, so that the data to be migrated is stored in the node hard disk in the second application node group without affecting normal read and write operations, thereby reducing the impact of persistence operations on system performance.

[0043] In the implementation process of the above scheme, the first application node group in the key-value database cluster obtains the slot data to be migrated from the data that has been persisted to the hard disk, rather than directly obtaining the slot data to be migrated from the memory, thereby ensuring that even if the slot data to be migrated fails during the transmission process, it can be restored by relying on the data on the hard disk, reducing the risk of data loss due to the failure to save temporary data in the memory in time. Furthermore, the second application node group that receives the slot data persists the received slot data on the hard disk, further ensuring the security of the data. Even if an unexpected situation occurs during the receiving process, it can be restored through the data on the hard disk, thereby further reducing the risk of data loss in the process of scaling the database cluster.

[0044] As an optional implementation of the above-mentioned key-value database expansion and contraction method, the above-mentioned key-value database cluster may also include: a metadata center group; the key-value database cluster may also use a byte to store logical partitions, and the maximum number of supported logical partitions is 256, and each logical partition may include a metadata center group and an application node group for data storage, the metadata center group is responsible for managing the metadata of the nodes in the entire cluster, including slot allocation, node status, logical database configuration, etc., while the application node group is responsible for actual data storage, and each logical database may have its own application node group. Before sending the slot data to be migrated to the second application node group among the multiple application node groups, the data may be migrated after receiving a data migration command, and the data migration command may include a first data migration command sent to the first application node group and a second data migration command sent to the second application node group. This implementation may include:

[0045] Step S101: Obtain metadata information of the slot data to be migrated through the first application node group, and send the metadata information to the metadata center group.

[0046] It can be understood that if the master node in the first application node group senses that the number of disconnected nodes in the group is greater than a preset threshold (due to node downtime or network failure), that is, there may be a risk of being unable to continue to elect the master node, then the potential risk can be sensed in advance and actions can be taken to prevent the occurrence of failures, and step S101 and its subsequent steps can be executed to migrate slot data. This early prevention mechanism can reduce the risk of data loss or service interruption due to node failure.

[0047] An example of an implementation of the above step S101 is: the first application node group obtains metadata information of the slot data to be migrated, and sends the metadata information to the metadata center group. Then, the metadata center group can filter out the second application node group that is most suitable for storing the slot data to be migrated from multiple application node groups based on the received metadata information. This means that the metadata center group can make the best choice based on factors such as the node load, storage capacity, and geographical location, which can optimize resource allocation, avoid resource waste, and improve the overall resource utilization of the system.

[0048] Step S102: receiving metadata information sent by the first application node group through the metadata center group, and selecting a second application node group for storing the slot data to be migrated from multiple application node groups according to the metadata information.

[0049] An implementation method of the above step S102 is, for example: by uniformly managing the data migration process through the metadata center group, the controllability and consistency of data migration can be ensured. The metadata center group receives the metadata information sent by the first application node group, and selects a suitable second application node group for data storage based on this information. Specifically, the second application node group for storing the slot data to be migrated is screened out from multiple application node groups based on the metadata information. This centralized management method reduces the uncertainty in the data migration process and ensures that the migration process meets expectations. By receiving and processing metadata information through the metadata center group, the secure transmission of metadata can be ensured. The metadata information contains important information about the slot data to be migrated. If it is leaked or tampered with, it may cause data migration failure or data loss. By centrally managing metadata information through the metadata center group, more stringent security measures can be taken to protect the security of data.

[0050] Step S103: sending a first data migration command to the first application node group through the metadata center group, so that the first application node group sends the slot data to be migrated to the second application node group.

[0051] An example of an implementation of the above step S103 is: the metadata center group sends a first data migration command and a second data migration command to the first application node group and the second application node group respectively to guide the data migration process. This method avoids sending instructions to all nodes at the same time and reduces the possibility of network congestion. By issuing commands in stages, the rhythm of data migration can be better controlled and the migration efficiency can be improved.

[0052] Step S104: Send a second data migration command to the second application data through the metadata center group, so that the second application node group receives the slot data to be migrated sent by the first application node group.

[0053] The implementation method of the above step S104 is, for example: through the coordination of the metadata center group, the data migration process is ensured to proceed in an orderly manner. The first application node group starts to send the slot data to be migrated after receiving the first data migration command, and the second application node group starts to receive data after receiving the second data migration command sent by the metadata center group to the second application data. This orderly data migration process reduces the risk of data loss and enhances the stability of the system.

[0054] In addition, since the data migration process is uniformly coordinated and managed by the metadata center group, users hardly feel any interruption or delay during the migration process. Of course, the metadata center group can also dynamically adjust the data migration strategy according to the actual situation. For example, if a node is found to be overloaded during the migration process, other nodes can be reselected for data storage. This flexibility enables the system to better adapt to changing workloads and network conditions, improving the system's adaptability and stability.

[0055] In the implementation process of the above solution, data migration is achieved through automation, reducing the need for manual intervention. The metadata center group is responsible for the coordination and management of the entire migration process, so that operation and maintenance personnel do not need to intervene manually, reducing operation and maintenance costs. The metadata center group, the first application node group, and the second application node group cooperate with each other to achieve automated migration, which can also reduce human errors and improve the reliability and success rate of migration.

[0056] As an optional implementation of the above-mentioned key-value data expansion and contraction method, before sending the data migration command to the first application node group and the second application data respectively, the write operation of the first application node to be contracted may be blocked first, and the implementation may include:

[0057] Step S105: Block the write operation of the first application node group through the metadata center group.

[0058] The implementation method of the above step S104 is, for example: during the data migration process, by blocking the write operation of the first application node group through the metadata center group, it can be ensured that no new write operation occurs during the data migration, thereby reducing the risk of data loss. This can ensure the consistency of data before and after the migration, and avoid data inconsistency or loss due to new write operations. Optionally, before blocking the write operation, the existing data synchronization operation can be performed first, without blocking the read and write operations, and the update operation uses a double write mechanism, that is, when the client updates the data, it writes to the original node and the target node at the same time to ensure the consistency of the data during the migration process. After blocking the write operation, the incremental data synchronization operation is then performed. Since no new write operation occurs during the blocking period, the incremental data volume is small, and the synchronization process is faster. This incremental data synchronization can reduce the overall time of data migration and improve migration efficiency. Since the existing synchronization is performed first, the incremental synchronization can be performed in a short time, which greatly shortens the locking time of the slot, and almost achieves the effect of lossless expansion and contraction.

[0059] Optionally, although the write operation of the first application node group is blocked by the metadata center group during the data migration process, the write operation of the second application node group is not blocked. In order to further reduce the risk of data loss, a dual write mechanism or even a multi-write mechanism can be used during the write operation of the first application node group, that is, two or more nodes in the first application node group write data simultaneously, which can ensure data consistency and reduce the risk of data loss. After blocking the write operation, the data integrity check can also be performed. Since no new write operation occurs, it can ensure that the data before the migration is consistent. This integrity check helps to ensure the integrity of the data after the migration and reduces the problem of data inconsistency caused by concurrent write operations.

[0060] In the implementation process of the above scheme, the process of data migration can be simplified by blocking the write operation of the first application node group, because during the blocking period, the system will not process new write requests, which makes data migration simpler and controllable. This simplification helps to improve the reliability of migration and reduce errors that may occur during the migration process. Furthermore, during the data migration process, if the write operation is not blocked, data conflicts may occur due to concurrent write operations. The scheme of the embodiment of the present application can avoid the occurrence of such data conflicts by blocking the write operation, effectively reducing the possibility of data conflicts and improving the reliability of data migration.

[0061] As an optional implementation of the above step S110, the above data migration command may include: a compression parameter and a segmentation parameter; the above implementation of sending the slot data to be migrated to the second application node group among the multiple application node groups may include:

[0062] Step S111: the first application node group compresses the data in the to-be-migrated slot according to the compression parameter to obtain a compressed file.

[0063] An implementation example of the above step S111 is: compressing the migration slot data according to the compression parameters by the first application node group, which can significantly reduce the time and bandwidth required for data transmission. This compression technology can reduce the original data volume by several times or even more, thereby speeding up the transmission speed.

[0064] Step S112: the first application node group divides the compressed file according to the division parameter to obtain a plurality of block files and sequence numbers of the plurality of block files.

[0065] For example, the implementation method of the above step S112 is as follows: since the first application node group divides the compressed file according to the segmentation parameter, the data can be divided into multiple block files, and the receiving party can load the block files into the memory as needed for reorganization and decompression. This can improve the situation in which the entire large file (such as a DB file) is loaded into the memory at one time in the traditional memory-based key-value database such as Redis or Memcached, and reduce the memory usage, which is particularly important when processing large-scale data migration.

[0066] Step S113: the first application node group sends a plurality of block files and sequence numbers of the plurality of block files to the second application node group.

[0067] The implementation method of the above step S113 is, for example: when the first application node group fails to send multiple block files and the serial numbers of multiple block files to the second application node group, the compression parameters and the segmentation parameters can be dynamically adjusted, so that the recompression and segmentation can be flexibly adjusted according to the current network conditions and system load conditions. The hot slot data can be migrated to the slave node with lower load to ensure the stability of the entire service process. When the network condition is good, a higher compression ratio and a larger block size can be used; when the network condition is poor, a lower compression ratio and a smaller block size can be used. After being segmented into multiple block files, the first application node group sends multiple block files and the serial numbers of multiple block files to the second application node group in parallel to improve the robustness of the transmission. This solution is applicable to a variety of network environments. Whether it is a high-bandwidth or low-bandwidth network, the transmission strategy can be optimized by adjusting the compression parameters and the segmentation parameters to ensure the efficiency and reliability of data migration.

[0068] Step S114: the second application node group receives the multiple block files and the sequence numbers of the multiple block files sent by the first application node group.

[0069] The implementation method of the above step S114 is, for example: when reorganizing the block files, an integrity check can be performed to ensure that the reorganized files are consistent with the original files. If the check fails, the corresponding block files can be resent to ensure the integrity and security of the data. In other words, since the data is divided into multiple block files, even if a failure occurs during the transmission process, the first application node group can resend the lost or erroneous block files, so that the second application node group can re-receive the lost or erroneous block files among the multiple block files sent by the first application node group, as well as the sequence number of the block file.

[0070] Step S115: the second application node group combines the multiple block files into a compressed file according to the sequence numbers of the multiple block files, and decompresses the compressed file into the above-mentioned slot data to be migrated.

[0071] The implementation method of the above step S115 is, for example, to assign a unique sequence number to each block file, so as to ensure that the receiving party can reassemble the block files in the correct order. Even if packet loss or errors occur during network transmission, the lost or erroneous parts can be resent by sequence number, thereby improving the reliability of transmission.

[0072] In the implementation process of the above scheme, by compressing, splitting, transmitting, reorganizing, decompressing and other steps of the slot data to be migrated, not only can the time and bandwidth required for data transmission be significantly reduced, but also the compressed file can be split into multiple block files for parallel transmission. In addition, if a block fails to be transmitted, only the block needs to be retransmitted, and the entire file does not need to be retransmitted, thereby making full use of the network bandwidth and further improving the transmission efficiency of the slot data to be migrated.

[0073] As an optional implementation of the above-mentioned key-value database expansion method, after receiving the data to be migrated sent by the first application node group through the second application node group, the write operation of the expanded second application node group can also be restored, including:

[0074] Step S130: Send a migration completion notification message to the metadata center group through the second application node group, so that the metadata center group resumes the write operation of the second application node group after receiving the migration completion notification message.

[0075] The implementation method of the above step S130 is, for example: sending a migration completion notification message to the metadata center group through the second application node group to ensure that the write operation of the second application node group can be restored only after the data migration is completed, which can reduce the risk of data loss. This step ensures the integrity and consistency of the data migration. After the metadata center group receives the migration completion notification, it can confirm that the data migration has been successfully completed, thereby restoring the write operation of the second application node group to ensure the continuity of the service. This method reduces the problems caused by data inconsistency or loss during the migration process.

[0076] In the implementation process of the above solution, by resuming the write operation after the migration is completed, the possibility of data conflict can be reduced. Since no new write operation occurs during the migration, the write operation can be resumed immediately after the migration is completed to ensure data consistency. This method reduces the possibility of data conflict and improves the reliability of data migration. By resuming the write operation after the migration is completed, the fluctuation of the system during the data migration process can be reduced. After the migration is completed, the system can immediately resume the write operation to ensure the continuity of the service. This stability helps to ensure the smooth progress of the data migration process and reduce the time of service interruption.

[0077] As an optional implementation of the above-mentioned key-value database expansion and contraction method, the above-mentioned multiple nodes may include a master node (Leader) and multiple slave nodes (Followers); the above-mentioned key-value database expansion and contraction method may also include:

[0078] Step S140: The master node in the metadata center group parses the index keyword from the received Table command.

[0079] An implementation example of the above-mentioned step S140 is as follows: the master node in the metadata center group parses the index keyword from the received Table command, finds the slot data identifier corresponding to the index keyword, and sends the Table command to the target application node group corresponding to the slot data identifier, which can ensure that the command is accurately forwarded to the correct node group. This mechanism reduces errors in the command execution process and improves the accuracy and efficiency of command execution.

[0080] Step S150: searching for the slot data identifier corresponding to the index keyword through the master node in the metadata center group, and sending a Table command to the target application node group corresponding to the slot data identifier.

[0081] The implementation method of the above step S150 is, for example: by searching the slot data identifier corresponding to the index keyword through the master node in the metadata center group, and sending the Table command to the target application node group, resource scheduling can be optimized to ensure that the command is executed at the most appropriate location. This optimization helps to improve the overall performance of the system and reduce unnecessary waste of resources. By searching the slot data identifier through the master node in the metadata center group, and dynamically routing the Table command to the target application node group, the system has better scalability and flexibility. This dynamic routing mechanism can adapt to the ever-changing workload and network conditions, and improves the adaptability of the system. By parsing the Table command, searching for the corresponding slot data identifier, and forwarding the command to the target application node group through the master node in the metadata center group, the master node in the target application node group finally parses and executes the command, which can ensure data consistency. In other words, all commands are uniformly parsed and forwarded by the metadata center group, reducing the risk of data inconsistency. This consistency guarantee helps to improve the reliability of the system and data integrity.

[0082] Step S160: The main node in the target application node group parses the sub-command type and the index keyword from the Table command, and executes the Table command according to the sub-command type and the index keyword.

[0083] The master node in the metadata center group above parses the command, obtains the sub-command type and index keyword, and forwards the parsed sub-command type and index keyword to the target application node group, which can simplify the command processing process. This simplification helps reduce system complexity and reduces the possibility of errors in the command processing process. By parsing and forwarding commands through the master node in the metadata center group, tasks can be completed through other nodes when the master node fails. This redundancy mechanism improves the high availability of the system and ensures that the system can still operate normally even in the event of partial node failure.

[0084] In the implementation process of the above solution, by dynamically routing Table commands to the target application node group, the system has better scalability and flexibility. This dynamic routing mechanism can adapt to the ever-changing workload and network conditions, improving the adaptability of the system. Furthermore, by parsing the sub-command type in the Table command through the master node in the target application node group, and executing the Table command according to the sub-command type and index keyword, complex query operations can be supported. This mechanism enables the system to handle more complex query requests, improving the functionality and practicality of the system.

[0085] As an optional implementation of the above step S160, the above implementation of executing the Table command according to the sub-command type and the index keyword may include:

[0086] Step S161: The master node in the target application node group selects a target slave node from a plurality of slave nodes in the target application node group according to the load data of each slave node.

[0087] The implementation method of the above step S161 is, for example: the master node in the target application node group selects the target slave node from multiple slave nodes according to the load data of each slave node, so as to realize the function of dynamic load balancing. This means that high-load nodes will not be overused, while low-load nodes can take on more tasks. This dynamic load balancing helps to improve the overall performance and response speed of the system. By selecting the target slave nodes with lower loads to process commands, it can be ensured that the commands are executed on nodes with higher processing efficiency. This efficient task allocation helps to reduce the time of command processing and improve the response speed of the system.

[0088] Step S162: Sending Table commands and index keywords to the target slave nodes through the master node in the target application node group.

[0089] The implementation method of the above step S162 is, for example: by screening the target slave nodes with lower load to process the Table command according to the index keyword, it is possible to avoid a node from failing due to overload, thereby enhancing the stability of the system. This mechanism reduces the risk of single point failure and improves the reliability and high availability of the system. By dynamically screening the target slave nodes, resource utilization can be optimized, and the nodes with lower load can be fully utilized, while the nodes with higher load can get a proper rest. This resource optimization helps to improve the overall performance of the system and reduce resource waste.

[0090] Step S163: Determine the index name corresponding to the index keyword from the index tree stored in the local slot data through the target slave node.

[0091] See also Figure 3The schematic diagram of the index tree of the local slot data storage provided by the embodiment of the present application is shown; the storage structure of the above-mentioned local slot data may include: a unique identifier (Unique IDentification, UID), a database (DataBase, DB) name, a first index field (Field1), a second index field (Field2), a third index field (Field3), a record identifier (Record IDentification, RID) of the slot data, a valid lifetime (Time To Live, TTL) of the slot data, and the data content (Data) of the slot data. Among them, the above-mentioned index tree can be composed of three fields, namely, the first index field (Field1), the second index field (Field2), and the third index field (Field3). Of course, it can also be composed of more index fields or fewer index fields. Specifically, the index tree of the local slot data storage can store index contents such as Field1, Field2a, Field2b, Field2c, Field3d, and Field3e.

[0092] In the above scheme, by selecting the target slave node with lower load to process the command, it can be ensured that the command is executed on the node with higher processing efficiency. This efficient task allocation helps to reduce the time of command processing and improve the response speed of the system. Furthermore, by determining the index name corresponding to the index keyword from the index tree stored in the local slot data by the target slave node, the accuracy of command processing can be ensured. This precise positioning helps to reduce errors in the data processing process and improve the consistency and accuracy of the data.

[0093] As an optional implementation of the above step S163, the above subcommand types include: Search subcommand and List subcommand; the above implementation of determining the index name corresponding to the index keyword from the index tree stored in the local slot data through the target slave node may include:

[0094] Step S163a: If the subcommand type is a Search subcommand, the index name corresponding to the index keyword is searched from the index tree of the local slot data storage through the target slave node.

[0095] For example, the implementation method of the above step S163a is as follows: assuming that the index tree of the local slot data storage stores index contents such as Field1, Field2a, Field2b, Field2c, Field3d and Field3e, and when the subcommand type is a Search subcommand, and the index keyword is Field2a, then the index name corresponding to the returned index keyword is Field2a under the Field1 level. Since it is a Search subcommand, Field2b and Field2c under the Field1 level do not need to be returned, and only Field2a under the Field1 level needs to be returned. The accuracy of the query results is ensured by returning the corresponding index name through precise matching of the index keywords. This precise matching reduces the possibility of false positives and false negatives, and improves the reliability of the query results.

[0096] Optionally, the above key-value database may support string subcommands, hash subcommands, list subcommands, zset subcommands, set subcommands, and the like in addition to the subcommand types of Search subcommands and List subcommands.

[0097] Step S163b: If the subcommand type is a List subcommand, all index names under the level where the index keyword is located are listed from the index tree of the local slot data storage through the target slave node.

[0098] For example, the implementation method of the above step S163b is as follows: assuming that the index tree of the local slot data storage stores index contents such as Field1, Field2a, Field2b, Field2c, Field3d and Field3e, when the subcommand type is the List subcommand, and the index keyword is Field2a, then the index names corresponding to the returned index keywords are Field3d and Field3e under the Field2a level. By listing all the index names under the level where the index keyword is located from the index tree of the local slot data storage through the target subordinate node, unnecessary search operations can be avoided, thereby improving query efficiency. The List subcommand can list all the index names under the level where the index keyword is located, supporting multi-level queries. This mechanism enables the system to handle more complex query requests, improving the functionality and practicality of the system.

[0099] In the above scheme, by supporting different types of Search subcommands and List subcommands, the system can handle diverse query requirements and enhance flexibility. Different query requirements can be implemented through different subcommands, which improves the versatility of the system. Furthermore, for the Search subcommand, the system only needs to find a specific index name, while for the List subcommand, the system needs to enumerate all index names within a certain range. This differentiated processing method can optimize resource usage according to actual needs and avoid unnecessary computing and storage overhead.

[0100] See also Figure 4 The structural diagram of the key-value database expansion and contraction device provided by the embodiment of the present application is shown; the embodiment of the present application provides a key-value database expansion and contraction device 200, including: multiple application node groups, each application node group includes multiple nodes, and the slot data stored in the node hard disk is transmitted between the multiple nodes through a distributed consistency consensus algorithm; the key-value database expansion and contraction device includes:

[0101] The slot data shrinking module 210 is used to obtain the slot data to be migrated from the node hard disk in the first application node group through the first application node group in the multiple application node groups, and send the slot data to be migrated to the second application node group in the multiple application node groups to shrink the data of the first application node group.

[0102] The slot data expansion module 220 is used to receive the data to be migrated sent by the first application node group through the second application node group, and store the data to be migrated in the node hard disk in the second application node group to expand the data capacity of the second application node group.

[0103] As an optional implementation of the above device, the key-value database cluster further includes: a metadata center group; the key-value database expansion and contraction device further includes:

[0104] The metadata information acquisition module is used to acquire the metadata information of the slot data to be migrated through the first application node group, and send the metadata information to the metadata center group.

[0105] The metadata receiving and utilizing module is used to receive metadata information sent by the first application node group through the metadata center group, and select a second application node group for storing the slot data to be migrated from multiple application node groups according to the metadata information, and send data migration commands to the first application node group and the second application data respectively, so that the first application node group sends the slot data to be migrated to the second application node group.

[0106] As an optional implementation of the above device, the key-value database expansion and contraction device further includes:

[0107] The node write operation blocking module is used to block the write operation of the first application node group through the metadata center group.

[0108] As an optional implementation of the above device, the data migration command includes: compression parameters and segmentation parameters; the slot data shrinking module includes:

[0109] The migration data compression submodule is used to compress the migration slot data according to the compression parameters to obtain a compressed file.

[0110] The compressed file segmentation submodule is used to segment the compressed file according to the segmentation parameters to obtain multiple segment files and sequence numbers of the multiple segment files.

[0111] The block file sending submodule is used to send multiple block files to the second application node group, so that the second application node group combines the multiple block files into a compressed file and decompresses the compressed file into slot data to be migrated.

[0112] As an optional implementation of the above device, the key-value database expansion and contraction device further includes:

[0113] The node write operation recovery module is used to send a migration completion notification message to the metadata center group through the second application node group, so that the metadata center group recovers the write operation of the second application node group after receiving the migration completion notification message.

[0114] As an optional implementation of the above device, the multiple nodes include a master node; the key-value database expansion and contraction device also includes:

[0115] The Table command sending module is used to parse the index keyword from the received Table command through the master node in the metadata center group, find the slot data identifier corresponding to the index keyword, and send the Table command to the target application node group corresponding to the slot data identifier.

[0116] The Table command execution module is used to parse the sub-command type and index keyword from the Table command through the master node in the target application node group, and execute the Table command according to the sub-command type and index keyword.

[0117] As an optional implementation of the above device, the multiple nodes further include a slave node; the Table command execution module includes:

[0118] The slave node screening submodule is used to screen out target slave nodes from multiple slave nodes in the target application node group according to the load data of each slave node through the master node in the target application node group, and send Table commands and index keywords to the target slave nodes.

[0119] The index name determination submodule is used to determine the index name corresponding to the index keyword from the index tree stored in the local slot data through the target slave node.

[0120] As an optional implementation of the above device, the subcommand types include: Search subcommand and List subcommand; the index name determination submodule includes:

[0121] The index name search unit is used to search for the index name corresponding to the index keyword from the index tree of the local slot data storage through the target slave node if the sub-command type is the Search sub-command.

[0122] The index name enumeration unit is used to enumerate all index names under the level where the index keyword is located from the index tree of the local slot data storage through the target slave node if the sub-command type is the List sub-command.

[0123] It should be understood that the device corresponds to the above-mentioned key-value database expansion and contraction method embodiment, and can execute the various steps involved in the above-mentioned method embodiment. The specific functions of the device can be referred to the description above, and the detailed description is appropriately omitted here. The device includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the operating system (OS) of the device.

[0124] See also Figure 5 The electronic device 300 provided in the embodiment of the present application includes: a processor 310 and a memory 320, wherein the memory 320 stores machine-readable instructions executable by the processor 310, and when the machine-readable instructions are executed by the processor 310, the above method is executed.

[0125] The embodiment of the present application also provides a computer-readable storage medium 330, on which a computer program is stored, and the computer program is executed by the processor 310 to execute the above method. The computer-readable storage medium 330 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk or optical disk.

[0126] The embodiment of the present application also provides a computer program product, including: a computer program or a computer instruction, and the computer program or the computer instruction executes the method described above when executed by a processor.

[0127] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0128] In several embodiments provided by the embodiments of the present application, it should be understood that the disclosed device and method can also be implemented in other ways. The device embodiments described above are only schematic, for example, the flowcharts and block diagrams in the accompanying drawings show the possible implementation architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a part of a module, a program segment or a code, and a part of a module, a program segment or a code includes one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also be different from the order of occurrence marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which is mainly determined by the functions involved.

[0129] In addition, each functional module of each embodiment in the embodiment of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part. In addition, in the description of this specification, the description of reference terms "one embodiment", "some embodiments", "example", "specific example", "some examples", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present application. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples without contradiction.

[0130] The above description is only an optional implementation manner of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the embodiments of the present application, which should be covered within the protection scope of the embodiments of the present application.

Claims

1. A method for scaling a key-value database, characterized in that: Applied to a key-value database cluster, the key-value database cluster includes: multiple application node groups, each of the application node groups includes multiple nodes, and the slot data stored in the node hard disk is transmitted between the multiple nodes through a distributed consistency consensus algorithm; the key-value database expansion and contraction method includes: Obtaining the slot data to be migrated from the node hard disks in the first application node group through a first application node group among the multiple application node groups, and sending the slot data to be migrated to a second application node group among the multiple application node groups, so as to perform data capacity reduction on the first application node group; The data to be migrated sent by the first application node group is received through the second application node group, and the data to be migrated is stored in the node hard disk in the second application node group, so as to expand the data capacity of the second application node group.

2. The method according to claim 1, characterized in that The key-value database cluster further includes: a metadata center group; before sending the slot data to be migrated to the second application node group among the multiple application node groups, further includes: Acquire metadata information of the slot data to be migrated through the first application node group, and send the metadata information to the metadata center group; The metadata information sent by the first application node group is received through the metadata center group, and a second application node group for storing the slot data to be migrated is screened out from the multiple application node groups according to the metadata information, and data migration commands are sent to the first application node group and the second application data respectively, so that the first application node group sends the slot data to be migrated to the second application node group.

3. The method according to claim 2, characterized in that Before sending the data migration command to the first application node group and the second application node group respectively, the method further includes: The write operation of the first application node group is blocked by the metadata center group.

4. The method according to claim 2, characterized in that: The data migration command includes: a compression parameter and a segmentation parameter; the sending the to-be-migrated slot data to the second application node group among the multiple application node groups includes: Compressing the slot data to be migrated according to the compression parameter to obtain a compressed file; Splitting the compressed file according to the segmentation parameter to obtain a plurality of block files and sequence numbers of the plurality of block files; The multiple block files are sent to the second application node group, so that the second application node group combines the multiple block files into the compressed file, and decompresses the compressed file into the slot data to be migrated.

5. The method according to claim 2, characterized in that: After the second application node group receives the data to be migrated sent by the first application node group, the method further includes: A migration completion notification message is sent to the metadata center group through the second application node group, so that the metadata center group resumes the write operation of the second application node group after receiving the migration completion notification message.

6. The method according to any one of claims 1 to 5, characterized in that: The multiple nodes include a master node; the key-value database expansion and contraction method further includes: The master node in the metadata center group parses the index keyword from the received Table command, finds the slot data identifier corresponding to the index keyword, and sends the Table command to the target application node group corresponding to the slot data identifier; The master node in the target application node group parses the sub-command type and the index keyword from the Table command, and executes the Table command according to the sub-command type and the index keyword.

7. The method according to claim 6, characterized in that The plurality of nodes also include a subordinate node; and executing the Table command according to the sub-command type and the index keyword includes: The master node in the target application node group selects a target slave node from a plurality of slave nodes in the target application node group according to the load data of each slave node, and sends the Table command and the index keyword to the target slave node; The target slave node determines the index name corresponding to the index keyword from the index tree stored in the local slot data.

8. The method according to claim 6, characterized in that The subcommand types include: Search subcommand and List subcommand; the determining the index name corresponding to the index keyword from the index tree stored in the local slot data through the target slave node includes: If the subcommand type is the Search subcommand, searching the index name corresponding to the index keyword from the index tree stored in the local slot data through the target slave node; If the subcommand type is the List subcommand, all index names under the level where the index keyword is located are listed from the index tree of the local slot data storage through the target slave node.

9. A key-value database expansion and contraction device, characterized in that: Applied to a key-value database cluster, the key-value database cluster includes: a plurality of application node groups, each of the application node groups includes a plurality of nodes, and the slot data stored in the node hard disk is transmitted between the plurality of nodes through a distributed consistency consensus algorithm; the key-value database expansion and contraction device includes: a slot data shrinking module, configured to obtain slot data to be migrated from node hard disks in a first application node group among the multiple application node groups through the first application node group among the multiple application node groups, and send the slot data to be migrated to a second application node group among the multiple application node groups, so as to shrink data of the first application node group; The slot data expansion module is used to receive the data to be migrated sent by the first application node group through the second application node group, and store the data to be migrated in the node hard disk in the second application node group to expand the data capacity of the second application node group.

10. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions are executed by the processor to perform any method according to claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is executed.

12. A computer program product, characterized in that include: A computer program or a computer instruction, wherein when the computer program or the computer instruction is executed by a processor, the method according to any one of claims 1 to 8 is executed.

Images