A method and device for reorganizing and forwarding data blocks in a process of adding or deleting nodes of a database shared storage cluster

Abstract

This invention relates to the field of database management technology, and provides a method and device for reorganizing and forwarding data blocks during the node addition and deletion process of a database shared storage cluster. The method includes: establishing a virtual node mapping mechanism to map data blocks in the cluster to a preset number of virtual nodes through hash calculation, and maintaining a dynamic mapping table from virtual nodes to physical nodes; determining a set of virtual nodes to be migrated, calculating the virtual nodes whose mapping relationships need to be changed according to a balancing rule, and updating the dynamic mapping table; constructing a state machine to control the migration process, setting a state machine for each virtual node to be migrated, and controlling the transmission process of data blocks from the source node to the target node through state transitions; and performing concurrent forwarding and request processing, processing data block transmission and data access requests in parallel during data block migration, and determining the forwarding path of the request based on the current state of the virtual node. This invention can achieve balanced reorganization of data blocks and efficient forwarding of requests.

Description

Technical Field

[0001] This invention relates to the field of database management technology, and in particular to a method and device for reassembling and forwarding data blocks in the process of adding or deleting nodes in a database shared storage cluster. Background Technology

[0002] When adding or deleting nodes in a shared storage cluster database, a key technical challenge lies in handling the redistribution of data blocks across all nodes. Due to the nature of shared storage, data blocks are physically stored in the shared storage device, but each compute node caches a portion of the data blocks locally to improve access performance. Simultaneously, the cluster typically employs a data distribution strategy to assign management responsibilities for different data blocks to different nodes. When the number of nodes changes, the distribution of data blocks across the nodes must be readjusted to ensure a balanced number of data blocks maintained by all nodes, avoiding a "data skew" phenomenon where some nodes are overloaded while others are idle. The balance of data block distribution directly impacts the overall performance and resource utilization of the cluster, and is a crucial metric for the success of scaling operations.

[0003] More complexly, in real-world business scenarios, online scaling of clusters often occurs simultaneously with continuous processing of business requests. Database systems cannot suspend all operations for node changes, requiring the scaling process to be "unaffected" by online business activity. However, existing technologies have several shortcomings in addressing this requirement:

[0004] First, traditional data redistribution solutions typically employ a crude approach of global locking or service suspension, blocking all business requests during node changes and restoring service only after data migration is complete. While this method is simple to implement, for 24 / 7 business systems, even minute- or second-level service interruptions can lead to severe business losses, failing to meet the requirements of high availability scenarios.

[0005] Second, while some existing solutions attempt to maintain service during data migration, they lack granular forwarding mechanisms. When business requests access data blocks that are being migrated, issues such as incorrect request routing, response timeouts, or data inconsistencies may occur. Even worse, the lack of precise control over the migration status could lead to the same data block being modified simultaneously by multiple nodes, resulting in serious data corruption.

[0006] Third, existing data balancing algorithms often focus only on the balance of the final result, neglecting the overhead of the migration process itself. Some solutions require migrating a large number of data blocks to achieve absolute balance, resulting in prolonged network bandwidth occupation and affecting the response time of normal business requests. Other solutions, while reducing the amount of migration, still achieve poor balancing, with significant load differences between nodes remaining.

[0007] Fourth, in terms of the control mechanism for the migration process, existing technologies lack fine-grained state management and concurrent processing capabilities. Data block migration is usually carried out in a coarse-grained manner, either causing network congestion by migrating a large amount of data at once, or resulting in a lengthy process by migrating data blocks one by one. At the same time, request processing and data migration are often executed serially by the same group of threads, which cannot fully utilize the parallel capabilities of multi-core processors, further exacerbating the system performance degradation during migration.

[0008] Chinese patent CN104219163A discloses a load balancing method based on dynamic replication and virtual node methods for dynamic node forward migration. This solution is aimed at cloud storage systems based on DHT (Distributed Hash Table), and its core is to solve the disk overload problem caused by uneven data distribution and uneven resource access popularity of storage nodes. However, when applied to the node addition and deletion process of shared storage cluster databases, it suffers from many defects, including lack of online business continuity assurance, coarse migration control granularity, insufficient concurrent processing capabilities, lack of network bandwidth control, fixed migration targets, and lack of data consistency assurance.

[0009] Therefore, how to achieve a balanced distribution of data blocks among nodes while minimizing the impact on online business performance, and ensuring the correctness and consistency of data access when adding or deleting nodes in a shared storage cluster database, has become an urgent technical problem to be solved. Summary of the Invention

[0010] In view of this, in order to overcome the shortcomings of the prior art, the present invention aims to provide a method and device for reassembling and forwarding data blocks in the process of adding or deleting nodes in a database shared storage cluster.

[0011] According to a first aspect of the present invention, a method for reassembling and forwarding data blocks during the process of adding or deleting nodes in a database shared storage cluster is provided, the method comprising the following steps:

[0012] Step S1: Establish a virtual node mapping mechanism to map all data blocks in the cluster to a preset number of virtual nodes through hash calculation, and maintain a dynamic mapping table from virtual nodes to physical nodes for each physical node;

[0013] Step S2: Determine the set of virtual nodes to be migrated. When the cluster triggers a node addition or deletion event, calculate the virtual nodes whose mapping relationship needs to be changed according to the load balancing rules and update the dynamic mapping table.

[0014] Step S3: Construct a state machine to control the migration process. Set up a state machine for each virtual node to be migrated, and control the transmission process of data blocks from the source node to the target node through state transitions.

[0015] Step S4: Perform concurrent forwarding and request processing. During data block migration, data block transmission and data access requests are processed in parallel by the first thread and the second thread. The forwarding path of the request is determined based on the current state of the virtual node.

[0016] Optionally, in the method for reassembling and forwarding data blocks during the node addition / deletion process of the database shared storage cluster of the present invention, in step S1, a virtual node mapping mechanism is established in the following manner:

[0017] All data blocks in the cluster are divided into virtual nodes according to a preset total number. Each data block uses its unique identifier within the database as the input value of the hash function. The virtual node number to which it belongs is determined through hash calculation. Multiple data blocks belonging to the same database object are mapped to the same virtual node.

[0018] A virtual node mapping table is maintained for each physical node, recording the physical node identifier to which each virtual node currently belongs. When a business thread needs to access any data block, it determines the physical node where the data block is located by querying the mapping table.

[0019] When the number of cluster nodes changes, the ownership of data blocks is redistributed by updating the mapping table on each physical node.

[0020] Optionally, in the method for reassembling and forwarding data blocks during the node addition / deletion process of the database shared storage cluster of the present invention, step S2 involves determining the set of virtual nodes to be migrated in the following manner:

[0021] In the node addition scenario, the target number of virtual nodes that each physical node should maintain is calculated based on the total number of nodes after addition, ensuring that the difference in the number of virtual nodes between physical nodes does not exceed 1. This target number is the total number of virtual nodes divided by the total number of nodes and then rounded down.

[0022] Divide the total number of virtual nodes by the original number of nodes plus 1 and round down to get the total number of virtual nodes to be migrated. Iterate sequentially from the beginning of the virtual node sequence. For each virtual node, determine whether the number of virtual nodes held by its current physical node after removing the virtual node is still not less than the target number. If so, mark the virtual node as to be migrated and assign it to the newly added node until the required total number of migrations is reached.

[0023] In the scenario of node exit, all virtual nodes under the responsibility of the exiting node are treated as a set to be migrated and evenly distributed to the remaining physical nodes according to the principle of balance, ensuring that the difference in the number of virtual nodes of each physical node after distribution does not exceed 1.

[0024] Optionally, in the method for reassembling and forwarding data blocks during the node addition / deletion process of the database shared storage cluster of the present invention, step S3 involves constructing a state machine to control the migration process as follows:

[0025] Four states are set for each virtual node, including normal state, pre-migration state, in-migration state, and migration completed state;

[0026] When no node change is triggered, all virtual nodes are in a normal state and process data block access requests normally.

[0027] Once a virtual node is identified as a target to be migrated, its status is changed from normal to pre-migration, indicating that the node has been selected but actual data transmission has not yet begun. When the data blocks corresponding to the virtual node begin to be transmitted to the target node, its status is changed to migration in progress. Once all data blocks corresponding to the virtual node have been transmitted and confirmed by the target node, its status is changed to migration complete.

[0028] After the reorganization process of all participating nodes is completed, all virtual nodes that have completed migration will be restored to their normal state.

[0029] Optionally, in the data block reorganization and forwarding method of the database shared storage cluster of the present invention, in step S3, the state machine further includes an accelerated state transition path implemented in the following manner: when any node receives a data block access request during the reorganization process, and finds through hash calculation that the original mapping of the virtual node to which the requested data block belongs is not on this node, but this node has already entered the reorganization state, without going through the pre-migration state, the local state of the virtual node is directly marked as in migration, and the request is forwarded to the new mapping physical node.

[0030] Optionally, in the data block reassembly and forwarding method of the database shared storage cluster of the present invention during the node addition and deletion process, in step S4, the first thread and the second thread cooperate in the following manner:

[0031] The first thread, as the global control thread, is used to receive and process broadcast messages of node changes, recalculate and generate new virtual node mapping relationships based on the changed node numbers and preset algorithms, pass the calculated virtual node information to be migrated to the second thread, maintain and update the status of each virtual node in real time, and process data block access requests from business threads.

[0032] The second thread, acting as the data migration thread, after receiving the information of the virtual nodes to be migrated, divides the virtual nodes to be migrated into multiple batches based on the current network bandwidth and cluster load. Each batch contains several virtual nodes, and the data blocks are transmitted sequentially according to the batch order. After the transmission of each batch is completed, the first thread is notified to update the status of the corresponding virtual nodes.

[0033] Optionally, in the method for reassembling and forwarding data blocks during the node addition / deletion process of the database shared storage cluster of the present invention, the second thread sends virtual node data blocks in batches in the following manner:

[0034] Before sending a batch, the second thread sends a request to the first thread to send the virtual nodes of the current batch. This request contains a list of virtual node numbers to be sent.

[0035] After receiving the send request, the first thread updates the state of each virtual node involved in the request from the pre-migration state to the in-migration state, and replies to the second thread with a send permission instruction;

[0036] After receiving the permission to send instruction, the second thread iterates through each virtual node in the batch, reads all the data blocks that the virtual node is responsible for from the local cache or storage, and sends them to the target node through the network connection.

[0037] After receiving all the data blocks from a virtual node, the target node broadcasts a message to the entire cluster that the virtual node has finished receiving the data; upon receiving the broadcast message, the first thread updates the status of the virtual node to the migration complete status.

[0038] Repeat the above interaction process until all batches of virtual nodes have been migrated.

[0039] Optionally, in the data block reassembly and forwarding method of the database shared storage cluster node addition and deletion process of the present invention, in step S4, the requested forwarding path is determined according to the current state of the virtual node in the following manner:

[0040] When the first thread receives a data block access request from the business thread, it uses the data block identifier information carried in the request to re-determine the virtual node number to which the data block belongs through hash calculation; it queries the locally maintained virtual node mapping table to obtain the current mapped physical node of the virtual node, and checks the status of the virtual node on the current node;

[0041] If the current node is not in a reorganization state, the request will be sent to the corresponding physical node for processing according to the original mapping relationship; if the current node is in a reorganization state and the virtual node is in a normal state or before migration, the request will be sent to the original mapping physical node; if the current node is in a reorganization state and the virtual node is in a migration state or has completed migration, the request will be forwarded to the new mapping physical node.

[0042] Optionally, in the data block reassembly and forwarding method of the database shared storage cluster of the present invention during the node addition and deletion process, the processing flow after requesting forwarding to the target physical node in step S4 is as follows:

[0043] The first thread of the target physical node receives the forwarded data block access request and hands it over to the second thread of the same node for processing.

[0044] The second thread checks the local status of the virtual node to which the requested data block belongs. If the virtual node is in the process of migration, the second thread reads the requested data block from the local cache or storage, constructs a response message, and returns it to the original requesting node.

[0045] If the virtual node's status is "migration complete," the second thread will forward the request back to the first thread of this node, which will then process the request using the latest metadata information and return a response.

[0046] According to a second aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect of the present invention.

[0047] The method and device for reassembling and forwarding data blocks in the process of adding or deleting nodes in a database shared storage cluster, as described in this invention, have the following beneficial technical effects:

[0048] 1. Achieve dynamic balance in data block distribution and improve cluster resource utilization.

[0049] This ensures that the number of virtual nodes maintained by each physical node does not differ by more than 1, achieving absolute balance in the distribution of data blocks on a macro level. This effectively avoids load skew caused by differences in node performance or data access hotspots, allowing the computing, memory, and network resources of each node in the cluster to be fully utilized, and maximizing the overall system throughput.

[0050] Second, achieve seamless migration of online business operations to ensure business continuity and service quality.

[0051] This allows data block migration and business request processing to proceed in parallel. During the migration process, business requests are intelligently routed to the correct processing node based on the current state of the virtual node, ensuring that every data block request initiated by the business thread receives a timely response throughout the entire node addition and deletion process. This prevents request blocking, timeouts, or failures due to data migration, truly achieving "zero-awareness" of scaling operations for online services.

[0052] Third, significantly reduce network overhead during the migration process to avoid impacting normal business operations.

[0053] This effectively avoids the problem of network bandwidth being instantly saturated due to the migration of large amounts of data at once, allowing data migration traffic to smoothly occupy network resources and coexist harmoniously with normal business request traffic. For new requests initiated during the reorganization process, virtual nodes can be directly set to a migration state and forwarded, reducing an additional message interaction and further reducing network overhead.

[0054] IV. Ensure data consistency and the correctness of request processing.

[0055] In the forwarding scenario, a complete request processing logic is designed to cover various state combinations that may occur during the reorganization process. For requests forwarded to the target node, it is ensured that no matter how many times the request is forwarded, the target node maintains the latest version of the data, thus preventing data consistency issues such as dirty reads and lost updates caused by data migration.

[0056] V. Improve migration efficiency and shorten cluster state recovery time.

[0057] The architecture employs a dual-threaded approach of GCS and GNS, which collaborates through a clear interface to form an efficient pipeline processing mode. This significantly improves the efficiency of data migration, shortens the overall completion time of node addition and deletion operations, and enables the cluster to recover to a stable operating state more quickly. Attached Figure Description

[0058] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a flowchart illustrating a method for reassembling and forwarding data blocks in the process of adding or deleting nodes in a database shared storage cluster according to an exemplary embodiment 1 of the present invention.

[0060] Figure 2 This is an example diagram illustrating the node states and transitions in the data block reassembly and forwarding method during the node addition / deletion process of a database shared storage cluster according to an exemplary embodiment 2 of the present invention.

[0061] Figure 3 This is an example diagram illustrating the "virtual" node mapping change in the data block reassembly and forwarding method during the node addition / deletion process of a database shared storage cluster according to Exemplary Embodiment 2 of the present invention.

[0062] Figure 4 This is an example diagram of the data block reassembly and forwarding method in the node addition and deletion process of the database shared storage cluster according to Exemplary Embodiment 3 of the present invention, showing the reassembly process when a node is added.

[0063] Figure 5 This is an example diagram of the data block reassembly and forwarding method in the node addition / deletion process of a database shared storage cluster according to an exemplary embodiment 3 of the present invention, showing the reassembly process when a node exits.

[0064] Figure 6 This is a schematic diagram of the structure of the device provided by the present invention. Detailed Implementation

[0065] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0066] It should be noted that, in the absence of conflict, the following embodiments and features can be combined with each other; and, based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0067] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0068] Example 1

[0069] Exemplary embodiment 1 of the present invention provides a method for reassembling and forwarding data blocks in the process of adding or deleting nodes in a database shared storage cluster. Figure 1 This is a flowchart illustrating a method for reassembling and forwarding data blocks in the process of adding or deleting nodes in a database shared storage cluster according to an exemplary embodiment 1 of the present invention. Figure 1 As shown, the method of this embodiment is implemented in the following manner:

[0070] Step S1: Establish a virtual node mapping mechanism to map all data blocks in the cluster to a preset number of virtual nodes through hash calculation, and maintain a dynamic mapping table from virtual nodes to physical nodes for each physical node.

[0071] As an optional example, the virtual node mapping mechanism is established in this embodiment as follows:

[0072] All data blocks in the cluster are divided into virtual nodes according to a preset total number. Each data block uses its unique identifier within the database as the input value of the hash function. The virtual node number to which it belongs is determined through hash calculation. Multiple data blocks belonging to the same database object are mapped to the same virtual node.

[0073] A virtual node mapping table is maintained for each physical node, recording the physical node identifier to which each virtual node currently belongs. When a business thread needs to access any data block, it determines the physical node where the data block is located by querying the mapping table.

[0074] When the number of cluster nodes changes, the ownership of data blocks is redistributed by updating the mapping table on each physical node.

[0075] Step S2: Determine the set of virtual nodes to be migrated. When the cluster triggers a node addition or deletion event, calculate the virtual nodes whose mapping relationship needs to be changed according to the load balancing rules and update the dynamic mapping table.

[0076] As an optional example, this embodiment determines the set of virtual nodes to be migrated in the following manner:

[0077] In the node addition scenario, the target number of virtual nodes that each physical node should maintain is calculated based on the total number of nodes after addition, ensuring that the difference in the number of virtual nodes between physical nodes does not exceed 1. This target number is the total number of virtual nodes divided by the total number of nodes and then rounded down.

[0078] Divide the total number of virtual nodes by the original number of nodes plus 1 and round down to get the total number of virtual nodes to be migrated. Iterate sequentially from the beginning of the virtual node sequence. For each virtual node, determine whether the number of virtual nodes held by its current physical node after removing the virtual node is still not less than the target number. If so, mark the virtual node as to be migrated and assign it to the newly added node until the required total number of migrations is reached.

[0079] In the scenario of node exit, all virtual nodes under the responsibility of the exiting node are treated as a set to be migrated and evenly distributed to the remaining physical nodes according to the principle of balance, ensuring that the difference in the number of virtual nodes of each physical node after distribution does not exceed 1.

[0080] Step S3: Construct a state machine to control the migration process. Set up a state machine for each virtual node to be migrated, and control the transmission process of data blocks from the source node to the target node through state transitions.

[0081] As an optional example, this embodiment constructs the state machine control transition process in the following manner:

[0082] Four states are set for each virtual node, including normal state, pre-migration state, in-migration state, and migration completed state;

[0083] When no node change is triggered, all virtual nodes are in a normal state and process data block access requests normally.

[0084] Once a virtual node is identified as a target to be migrated, its status is changed from normal to pre-migration, indicating that the node has been selected but actual data transmission has not yet begun. When the data blocks corresponding to the virtual node begin to be transmitted to the target node, its status is changed to migration in progress. Once all data blocks corresponding to the virtual node have been transmitted and confirmed by the target node, its status is changed to migration complete.

[0085] After the reorganization process of all participating nodes is completed, all virtual nodes that have completed migration will be restored to their normal state.

[0086] It should be noted that the state machine constructed in this embodiment also includes an accelerated state transition path implemented in the following way: when any node receives a data block access request during the reorganization process, and finds through hash calculation that the original mapping of the virtual node to which the requested data block belongs is not on this node, but this node has already entered the reorganization state, without going through the pre-migration state, the local state of the virtual node is directly marked as in migration, and the request is forwarded to the new mapping physical node.

[0087] Step S4: Perform concurrent forwarding and request processing. During data block migration, data block transmission and data access requests are processed in parallel by the first thread and the second thread. The forwarding path of the request is determined based on the current state of the virtual node.

[0088] In this embodiment, the first thread and the second thread cooperate in the following manner:

[0089] The first thread, as the global control thread, is used to receive and process broadcast messages of node changes, recalculate and generate new virtual node mapping relationships based on the changed node numbers and preset algorithms, pass the calculated virtual node information to be migrated to the second thread, maintain and update the status of each virtual node in real time, and process data block access requests from business threads.

[0090] The second thread, acting as the data migration thread, after receiving the information of the virtual nodes to be migrated, divides the virtual nodes to be migrated into multiple batches based on the current network bandwidth and cluster load. Each batch contains several virtual nodes, and the data blocks are transmitted sequentially according to the batch order. After the transmission of each batch is completed, the first thread is notified to update the status of the corresponding virtual nodes.

[0091] In practical applications, the second thread sends virtual node data blocks in batches as follows:

[0092] Before sending a batch, the second thread sends a request to the first thread to send the virtual nodes of the current batch. This request contains a list of virtual node numbers to be sent.

[0093] After receiving the send request, the first thread updates the state of each virtual node involved in the request from the pre-migration state to the in-migration state, and replies to the second thread with a send permission instruction;

[0094] After receiving the permission to send instruction, the second thread iterates through each virtual node in the batch, reads all the data blocks that the virtual node is responsible for from the local cache or storage, and sends them to the target node through the network connection.

[0095] After receiving all the data blocks from a virtual node, the target node broadcasts a message to the entire cluster that the virtual node has finished receiving the data; upon receiving the broadcast message, the first thread updates the status of the virtual node to the migration complete status.

[0096] Repeat the above interaction process until all batches of virtual nodes have been migrated.

[0097] As an optional example, this embodiment determines the request forwarding path based on the current state of the virtual node in the following manner:

[0098] When the first thread receives a data block access request from the business thread, it uses the data block identifier information carried in the request to re-determine the virtual node number to which the data block belongs through hash calculation; it queries the locally maintained virtual node mapping table to obtain the current mapped physical node of the virtual node, and checks the status of the virtual node on the current node;

[0099] If the current node is not in a reorganization state, the request will be sent to the corresponding physical node for processing according to the original mapping relationship; if the current node is in a reorganization state and the virtual node is in a normal state or before migration, the request will be sent to the original mapping physical node; if the current node is in a reorganization state and the virtual node is in a migration state or has completed migration, the request will be forwarded to the new mapping physical node.

[0100] In this embodiment, the processing flow after the request is forwarded to the target physical node is as follows:

[0101] The first thread of the target physical node receives the forwarded data block access request and hands it over to the second thread of the same node for processing.

[0102] The second thread checks the local status of the virtual node to which the requested data block belongs. If the virtual node is in the process of migration, the second thread reads the requested data block from the local cache or storage, constructs a response message, and returns it to the original requesting node.

[0103] If the virtual node's status is "migration complete," the second thread will forward the request back to the first thread of this node, which will then process the request using the latest metadata information and return a response.

[0104] Example 2

[0105] Exemplary embodiment 2 of the present invention provides a method for reassembling and forwarding data blocks in the process of adding or deleting nodes in a database shared storage cluster. The method of this embodiment is implemented in the following manner:

[0106] In this embodiment, fine-grained (e.g., 1024) "virtual" nodes and a mapping from "virtual" nodes to physical nodes are used to achieve balanced control of all data blocks across all physical nodes within the cluster. Each physical node maintains this mapping to know the physical node information to which a specific data block belongs.

[0107] For each data block, its own identifier (usually object ID, object type, and page number within the database) is used as the hash key to calculate its corresponding "virtual" node. Furthermore, data blocks belonging to the same physical object are placed within the same virtual node, effectively accelerating cache utilization.

[0108] In this embodiment, the mapping from "virtual" nodes to physical nodes satisfies the following rules:

[0109] Rule 1: To balance data block resources, the difference in the number of "virtual" nodes between physical nodes shall not exceed 1;

[0110] Rule 2: To reduce the amount of messages in the process of adding or deleting nodes, the number of change messages in the mapping from virtual nodes to physical nodes should be minimized.

[0111] In the process of adding or deleting nodes, the parts concerning "virtual" nodes can all be viewed as a reorganization process, that is, the change in mapping relationships leading to a shift in ownership of the data blocks to which they belong. To support the reorganization capability of "virtual" nodes, this embodiment designs the "virtual" node reorganization as follows:

[0112] First, this embodiment divides the "virtual" node into four states: normal, before migration, during migration, and after migration. The normal state indicates that the current "virtual" node has not been modified in mapping and is usable. The before migration state indicates that the "virtual" node is selected and needs to be migrated, but has not yet started. The during migration state indicates that the "virtual" node has started to migrate, but the corresponding data block has not yet been migrated. The after migration state indicates that the "virtual" node has been migrated and will return to the normal state after the subsequent processes of adding or deleting nodes are completed.

[0113] Figure 2This diagram illustrates the node states and transitions in the data block reassembly and forwarding method during the node addition / deletion process of a database shared storage cluster according to Exemplary Embodiment 2 of the present invention. In this embodiment, the state change line from Normal → Before Migration → During Migration → Migration Completed → Normal represents the main process of a "virtual" node being selected for migration. The Normal → During Migration → Migration Completed → Normal state change is an acceleration measure for "virtual" nodes being migrated when their original mapping does not belong to this physical node. This prepares for subsequent forwarding; that is, if a "virtual" node that does not originally belong to this physical node is found to need migration during the request processing, it is directly treated as being migrated and forwarded to the new physical node, thus reducing one message forwarding step.

[0114] In practical applications, the mapping of "virtual" nodes is changed as follows:

[0115] Step a, determine the number of "virtual" nodes to be moved.

[0116] If the scenario involves adding nodes, assuming the current number of instances is X, then according to rule 1, the maximum number of virtual nodes for the newly added instance is [total number of virtual nodes / (X+1)], and the minimum number of virtual nodes is the total number of virtual nodes / (X+1). According to rule 1, the number of virtual nodes to be migrated is the total number of virtual nodes / (X+1).

[0117] If it is an exit node scenario, the moved "virtual" node is the "virtual" node corresponding to this physical node, which needs to be allocated to the remaining instances. The maximum number of virtual nodes per instance is [total number of "virtual" nodes / (X-1)].

[0118] Step b: Determine the "virtual" node to be migrated.

[0119] If the scenario involves adding nodes, and the number of "virtual" nodes that the existing instance needs to allocate to the new instance is known, then the following algorithm is used to determine which "virtual" nodes need to be migrated:

[0120] Starting from the head of the "virtual" node group, if the number of corresponding physical nodes that are "virtual" nodes is greater than or equal to the total number of "virtual" nodes / (X+1) + 1, then this "virtual" node can be migrated; otherwise, continue to check the next "virtual" node. When the number of migrated nodes reaches the total number of "virtual" nodes / (X+1), the work is completed.

[0121] If the exit occurs in a node exit scenario, the "virtual" nodes of the exiting instance are evenly distributed among the remaining physical nodes.

[0122] Step c: Modify the mapping relationship.

[0123] Modify the physical node corresponding to the migrated "virtual" node determined in step b to the new physical node after migration.

[0124] The method of this embodiment will be further described below in a specific scenario. In this scenario, the mapping changes of 10 "virtual" nodes when nodes are added or removed will be explained in detail.

[0125] Figure 3 This is an example diagram illustrating the "virtual" node mapping change in the data block reassembly and forwarding method during the node addition / deletion process of a database shared storage cluster according to Exemplary Embodiment 2 of the present invention.

[0126] like Figure 3 As shown, in this embodiment, at time 1, there is only one physical node X, so all 10 "virtual" nodes are mapped to node X. At time 2, a new physical node Y is added, requiring the migration of at least 10 / (1+1) = 5 "virtual" nodes. Judging from the order of migration, it is found that the number of physical nodes corresponding to "virtual" nodes 1 to 5, after removing them, is no greater than the maximum number of "virtual" nodes after reorganization (5+1=6). Therefore, "virtual" nodes 1 to 5 are migrated to Y. At time 3, a new physical node Z is added. Following the same calculation, the migration requires 10 / (2+1) = 3 "virtual" nodes, and the number of "virtual" nodes of the physical node must not exceed 3+1=4. Therefore, the mapping becomes... Figure 3 In the case where "virtual" nodes 1, 2, and 6 are assigned to Z; at time 4, physical node Z exits, and its corresponding "virtual" nodes 1, 2, and 6 need to have their mapping relationships re-established. Similarly, the re-established mappings must satisfy the condition that physical nodes X and Y are balanced. Ultimately, "virtual" node 1 is assigned to X, and "virtual" nodes 2 and 6 are assigned to Y. This embodiment's mapping method is efficient and simple, reducing the number of "virtual" nodes to be migrated while maintaining balance.

[0127] Example 3

[0128] Exemplary embodiment 3 of the present invention provides a method for reassembling and forwarding data blocks during the node addition and deletion process of a database shared storage cluster. In order to support reassembly operations during the node addition and deletion process, the following two threads are designed in this embodiment:

[0129] The GCS thread is used during the reorganization process to respond to / forward node page requests, receive reorganization signals, initiate instance reorganization, broadcast virtual node migration completion signals, and change virtual node states.

[0130] The gns thread is used during the reorganization process to calculate the batch of migrated virtual nodes, process forwarded page requests, send virtual node data, and complete the instance reorganization work.

[0131] Figure 4This is an example diagram illustrating the data block reassembly and forwarding method in the node addition / deletion process of a database shared storage cluster according to Exemplary Embodiment 3 of the present invention, specifically the reassembly process when a node is added. Figure 4 As shown, in this embodiment, node addition is performed according to the following process:

[0132] 1. During the startup phase, the postmaster thread of the newly added node first changes to the preparation for reorganization state and sends a start reorganization command to the gcs thread of this node. After receiving this command, the new node's gcs sends a broadcast message to the existing nodes in the current cluster and activates the gns thread to start the receiving task.

[0133] 2. After receiving the reorganization request from the new node, the existing node also changes its current state to the reorganization state and generates a new hash mapping relationship based on the node number of the new node.

[0134] 3. The new hash mapping relationship is sent to the newly joined node through the existing node. The newly joined node determines which "virtual" nodes need to wait to receive the data based on the node number.

[0135] 4. Existing nodes determine the "virtual" node information that needs to be migrated based on the new hash mapping relationship, and initiate the actual reorganization of GNS.

[0136] 5. Before initiating the reorganization process, gns divides the "virtual" nodes to be migrated into multiple batches to avoid network congestion and balance network bandwidth usage at different times.

[0137] 6. GNS sends the data block information corresponding to the "virtual" node sequentially in batches. The sending interaction in one round is as follows:

[0138] 1) GNS sends a send request for the current "virtual" node to GCS;

[0139] 2) GCS changes the status of the "virtual" node to "migration in progress";

[0140] 3) GCS replies to GNS, allowing this "virtual" node to migrate;

[0141] 4) GNS sends all data block information corresponding to this "virtual" node to the newly joined node;

[0142] 5) The newly joined node receives all the data from this "virtual" node and broadcasts a message indicating that reception is complete;

[0143] 6) When GCS receives this broadcast message, it updates the status of the "virtual" node to indicate that the migration is complete.

[0144] 7. After all the "virtual" nodes to be migrated have been migrated, all nodes will undergo an inspection phase to determine whether the impact of this migration has ended. The basis for this determination is whether any old business uses the original hash mapping information. Based on this, the reorganization end flag of the current node will be set.

[0145] Figure 5 This is an example diagram illustrating the data block reassembly and forwarding method in the node addition / deletion process of a database shared storage cluster according to Exemplary Embodiment 3 of the present invention, specifically the reassembly process when a node exits. Figure 5 As shown, in this embodiment, node exit follows the following process:

[0146] 1. When the main thread of the exiting node exits, it sends an exit message to the gcs thread of this node. After receiving the message, the gcs thread of the exiting node broadcasts it to other existing nodes, calculates the "virtual" node information to be migrated, generates a new hash mapping, and starts the GNS sending task.

[0147] 2. When an existing node receives a broadcast from a departing node, it first changes its own reorganization state and generates a new hash map based on the departing node's number. Since all nodes maintain consistent information and use the same calculation method, the final hash map of the cluster will also be consistent. After generation, it sends an "Enable Reception" message to its local GNSS server and then waits for the reception process to begin.

[0148] 3. For bandwidth control purposes, the pending "virtual" nodes of the exiting node are also divided into batches.

[0149] 4. GNS sends the data block information corresponding to the "virtual" node in batches sequentially. The sending interaction for the same round is as follows:

[0150] 1) GNS sends a send request for the current "virtual" node to GCS;

[0151] 2) GCS changes the status of the "virtual" node to "migration in progress";

[0152] 3) GCS replies to GNS, allowing this "virtual" node to migrate;

[0153] 4) GNS sends all data block information corresponding to this "virtual" node to the newly joined node;

[0154] 5) The newly joined node receives all the data from this "virtual" node and broadcasts a message indicating that reception is complete;

[0155] 6) When GCS receives this broadcast message, it updates the status of the "virtual" node to indicate that the migration is complete.

[0156] 5. Once all the "virtual" nodes to be migrated have been migrated, all nodes undergo a check phase to determine whether the impact of this migration has ended. The determination is based on whether any old services are using the original hash mapping information, and the reorganization end flag for the current node is set accordingly. Exit nodes are directly removed from the cluster after the migration is complete.

[0157] It should be noted that, in this embodiment, Figure 5 The migration impact range shown refers to the time period outside of which nodes need to determine whether to perform a forwarding operation when processing page requests. The time period outside of this range has no impact on the overall performance of the cluster.

[0158] Example 4

[0159] Exemplary Example 4 of the present invention provides a method for reorganizing and forwarding data blocks in the process of adding or deleting nodes in a database shared storage cluster. In this embodiment, when an instance is set to a migration state, the GCS must recalculate whether the original master instance number and the local instance number marked on the corresponding page are consistent before processing the request. If they are consistent, the request is processed locally; otherwise, it is forwarded.

[0160] During the migration, the page requests handled by this instance may have the following characteristics:

[0161] 1) The virtual node corresponding to the requested page has not been migrated, and the request is sent to the original mapped physical node;

[0162] 2) The virtual node corresponding to the requested page is in a state of pending migration; send a request to the original mapped physical node.

[0163] 3) The virtual node corresponding to the requested page is being migrated, and the request is being forwarded to the new mapped physical node;

[0164] 4) The virtual node corresponding to the requested page has been migrated, and the request is forwarded to the new mapped physical node;

[0165] 5) When the instance is not in a migration state, use the mapped physical node in the request to process the request.

[0166] To avoid impacting cluster responses, requests forwarded to GNS are handled by the GNS of the newly mapped physical node. If GNS detects that the corresponding virtual node has already migrated, it sends a response to the corresponding requesting business thread via the GCS in this instance; otherwise, it handles and responds itself. Forwarded requests include the request's metadata. Due to forwarding, the new mapped physical node may receive multiple sets of metadata. If the metadata has already been received, it is ignored, ensuring that the new mapped physical node maintains the latest version.

[0167] Example 5

[0168] Exemplary Example 4 of the present invention provides a method for reassembling and forwarding data blocks in the process of adding or deleting nodes in a database shared storage cluster. The method of this embodiment will be described in detail in a specific scenario.

[0169] The instances involved in the reorganization can be categorized by role into A (the requester), B (the original virtual node master), and C (the new virtual node master). The following lists the processing actions for each of the three roles in various scenarios:

[0170] (1) A received the reorganization message, but B has not yet received it.

[0171] When A initiates a page request, it finds that the current node is undergoing a reorganization operation. Using the tag, it recalculates that the master of the page is B (since B has not yet started reorganization, the permissions of this virtual node have certainly not been transferred). Therefore, it sends the request to B according to the original process.

[0172] B receives the request and, since reorganization has not yet started, processes it directly, sending a reply to the request backend (business thread) thread above A to update the local metadata about this page.

[0173] (2) A did not receive the reorganization message, while B has received it and entered the reorganization process.

[0174] A initiates a page request, but is not in a reorganization state, so it sends the request to B according to the original process.

[0175] B receives the request and has started the reorganization locally, recalculating the master of the virtual node, as shown in (3), (4) and (5).

[0176] (3) B received the reorganization message, but the corresponding virtual node has not yet been migrated.

[0177] B receives the request and finds that the instance is undergoing reorganization. Based on the marker, it obtains the latest master (physical node) information, which has been modified, but the virtual node status is still pending. Therefore, it is directly handled by GCS, which sends a reply to the request backend thread on A to update the local metadata about this page.

[0178] (4) B received the reorganization message and requested that the metadata migration of the corresponding virtual node was not completed.

[0179] A initiates a page request and finds that the current node is undergoing a reorganization operation, and the virtual node corresponding to the current page has not yet completed reorganization (broadcast information has not yet been received), but still sends a request to B.

[0180] B receives the request, discovers that the instance is undergoing reorganization, and obtains the latest master information based on the tag, which has been changed to C, and the corresponding virtual node is sending data. Therefore, B forwards this message along with its local page metadata to C for processing.

[0181] C receives a forward from B. Since the virtual node has not yet completed its migration, C processes it itself, updates and inserts the page's metadata, returns a reply to the request backend thread above A, and skips this page when it receives reassembled metadata later.

[0182] (5) B received the reorganization message and requested the corresponding virtual node to complete the migration, but the broadcast was not received.

[0183] A initiates a page request and finds that the current node is undergoing a reorganization operation, and the virtual node corresponding to the current page has not yet completed reorganization (broadcast information has not yet been received), but still sends a request to B.

[0184] B receives the request and finds that the instance is undergoing reorganization (broadcast not receiving update status). Based on the tag, it obtains the latest master, which has been changed to C, and the corresponding virtual node is sending. Therefore, it forwards this message along with the local page metadata to C for processing.

[0185] C receives a forward from B, indicating that the virtual node migration is complete. Therefore, the receiving thread forwards the request to GCS, allowing GCS to process it using the latest metadata information and return a response to the request backend thread on A.

[0186] (6) A received the reorganization message and the broadcast of the migration completion has been received.

[0187] A initiates a page request, discovers that the current node is undergoing a reorganization operation, calculates the virtual node status corresponding to the tag to be already migrated, and therefore directly sends the page request to the new C node.

[0188] Node C receives the request and responds to the request from the backend thread on node A.

[0189] The data block reassembly and forwarding method in the node addition and deletion process of the database shared storage cluster in this embodiment can be applied to shared storage databases and has the following beneficial technical effects:

[0190] 1. Achieve dynamic balance in data block distribution and improve cluster resource utilization.

[0191] By introducing a fine-grained virtual node mechanism, data blocks are first mapped to a preset number of virtual nodes, and then the ownership of data blocks is flexibly adjusted through dynamic mapping from virtual nodes to physical nodes. When nodes join or leave, a load balancing algorithm based on a number threshold is used to ensure that the difference in the number of virtual nodes maintained by each physical node does not exceed 1. This achieves absolute balance in the distribution of data blocks on a macro level, effectively avoiding load skew caused by differences in node performance or data access hotspots. This ensures that the computing, memory, and network resources of each node in the cluster are fully utilized, maximizing the overall system throughput.

[0192] Second, achieve seamless migration of online business operations to ensure business continuity and service quality.

[0193] Through a sophisticated virtual node state machine design, the migration process is divided into four states: normal, pre-migration, during migration, and completed migration. Combined with a dual-thread concurrent processing architecture, this allows data block migration and business request processing to proceed in parallel. During migration, business requests are intelligently routed to the correct processing node based on the current state of the virtual node: for virtual nodes that have not yet migrated, requests are processed along the original path; for virtual nodes that are migrating or have completed migration, requests are seamlessly forwarded to the new node for processing. This ensures that every data block request initiated by the business thread receives a timely response throughout the entire node addition and deletion process, preventing request blocking, timeouts, or failures due to data migration, truly achieving "zero-awareness" of scaling operations for online services.

[0194] Third, significantly reduce network overhead during the migration process to avoid impacting normal business operations.

[0195] A batch control mechanism is introduced in the data block transmission stage. The GNS thread divides the virtual nodes to be migrated into multiple batches and sends the corresponding data blocks of each virtual node in batch order. This effectively avoids the problem of network bandwidth being instantly saturated due to the migration of a large amount of data at once, allowing data migration traffic to smoothly occupy network resources and coexist with normal business request traffic. At the same time, through the accelerated state transition path design in the state machine, for new requests initiated during the reorganization process, the virtual node can be directly set to the migration state and forwarded, reducing an additional message interaction and further reducing network overhead.

[0196] IV. Ensure data consistency and the correctness of request processing.

[0197] In the forwarding scenario, a complete request processing logic is designed to cover various state combinations that may occur during the reorganization process, including situations where the request initiating node and the target node are in different reorganization stages, or virtual nodes are in different migration states. For requests forwarded to the target node, the GNS thread decides whether to process the request itself or hand it over to the GCS thread based on the actual state of the virtual node. Through a metadata deduplication mechanism, it is ensured that no matter how many times the request is forwarded, the target node maintains the latest version of the data, thus preventing data consistency issues such as dirty reads and lost updates caused by data migration.

[0198] V. Improve migration efficiency and shorten cluster state recovery time.

[0199] Employing a dual-threaded architecture of GCS and GNS, the system separates request forwarding control from data block transmission, enabling these two critical processes to execute in parallel. They collaborate through a clear interface, forming a highly efficient pipelined processing model. Compared to traditional serial processing methods, this significantly improves data migration efficiency, shortens the overall completion time for node addition and deletion operations, and allows the cluster to recover to a stable operating state more quickly.

[0200] like Figure 6 As shown, the present invention also provides a device including a processor 310, a communication interface 320, a memory 330 for storing processor-executable computer programs, and a communication bus 340. The processor 310, communication interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 executes the executable computer program to implement the data block reassembly and forwarding method in the above-described process of adding and deleting nodes in a database shared storage cluster.

[0201] The computer program in memory 330, when implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0202] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected based on actual needs to achieve the purpose of this embodiment. Those skilled in the art can understand and implement this without any creative effort.

[0203] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0204] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for reorganizing and forwarding data blocks in a process of adding or deleting a node of a database shared storage cluster, characterized in that, The method includes the following steps: Step S1: Establish a virtual node mapping mechanism to map all data blocks in the cluster to a preset number of virtual nodes through hash calculation, and maintain a dynamic mapping table from virtual nodes to physical nodes for each physical node; Step S2: Determine the set of virtual nodes to be migrated. When the cluster triggers a node addition or deletion event, calculate the virtual nodes whose mapping relationship needs to be changed according to the load balancing rules and update the dynamic mapping table. Step S3: Construct a state machine to control the migration process. Set up a state machine for each virtual node to be migrated, and control the transmission process of data blocks from the source node to the target node through state transitions. Step S4: Perform concurrent forwarding and request processing. During data block migration, data block transmission and data access requests are processed in parallel by the first thread and the second thread. The forwarding path of the request is determined based on the current state of the virtual node.

2. The method for reassembling and forwarding data blocks in the process of adding or deleting nodes in a database shared storage cluster according to claim 1, characterized in that, In step S1, the virtual node mapping mechanism is established as follows: All data blocks in the cluster are divided into virtual nodes according to a preset total number. Each data block uses its unique identifier within the database as the input value of the hash function. The virtual node number to which it belongs is determined through hash calculation. Multiple data blocks belonging to the same database object are mapped to the same virtual node. A virtual node mapping table is maintained for each physical node, recording the physical node identifier to which each virtual node currently belongs. When a business thread needs to access any data block, it determines the physical node where the data block is located by querying the mapping table. When the number of cluster nodes changes, the ownership of data blocks is redistributed by updating the mapping table on each physical node.

3. The method for reassembling and forwarding data blocks in the process of adding or deleting nodes in a database shared storage cluster according to claim 1, characterized in that, In step S2, the set of virtual nodes to be migrated is determined as follows: In the node addition scenario, the target number of virtual nodes that each physical node should maintain is calculated based on the total number of nodes after addition, ensuring that the difference in the number of virtual nodes between physical nodes does not exceed 1. This target number is the total number of virtual nodes divided by the total number of nodes and then rounded down. Divide the total number of virtual nodes by the original number of nodes plus 1 and round down to get the total number of virtual nodes to be migrated. Iterate sequentially from the beginning of the virtual node sequence. For each virtual node, determine whether the number of virtual nodes held by its current physical node after removing the virtual node is still not less than the target number. If so, mark the virtual node as to be migrated and assign it to the newly added node until the required total number of migrations is reached. In the scenario of node exit, all virtual nodes under the responsibility of the exiting node are treated as a set to be migrated and evenly distributed to the remaining physical nodes according to the principle of balance, ensuring that the difference in the number of virtual nodes of each physical node after distribution does not exceed 1.

4. The method for reassembling and forwarding data blocks in the process of adding or deleting nodes in a database shared storage cluster according to claim 1, characterized in that, In step S3, the state machine control transition process is constructed as follows: Four states are set for each virtual node, including normal state, pre-migration state, in-migration state, and migration completed state; When no node change is triggered, all virtual nodes are in a normal state and process data block access requests normally. Once a virtual node is identified as a target to be migrated, its status is changed from normal to pre-migration, indicating that the node has been selected but actual data transmission has not yet begun. When the data blocks corresponding to the virtual node begin to be transmitted to the target node, its status is changed to migration in progress. Once all data blocks corresponding to the virtual node have been transmitted and confirmed by the target node, its status is changed to migration complete. After the reorganization process of all participating nodes is completed, all virtual nodes that have completed migration will be restored to their normal state.

5. The method for reassembling and forwarding data blocks in the process of adding or deleting nodes in a database shared storage cluster according to claim 4, characterized in that, In step S3, the state machine also includes an accelerated state transition path implemented as follows: when any node receives a data block access request during the reorganization process, and finds through hash calculation that the original mapping of the virtual node to which the requested data block belongs is not on this node, but this node has already entered the reorganization state, without going through the pre-migration state, the local state of the virtual node is directly marked as in migration, and the request is forwarded to the new mapping physical node.

6. The method for reassembling and forwarding data blocks in the process of adding or deleting nodes in a database shared storage cluster according to claim 1, characterized in that, In step S4, the first thread and the second thread cooperate in the following manner: The first thread, as the global control thread, is used to receive and process broadcast messages of node changes, recalculate and generate new virtual node mapping relationships based on the changed node numbers and preset algorithms, pass the calculated virtual node information to be migrated to the second thread, maintain and update the status of each virtual node in real time, and process data block access requests from business threads. The second thread, acting as the data migration thread, after receiving the information of the virtual nodes to be migrated, divides the virtual nodes to be migrated into multiple batches based on the current network bandwidth and cluster load. Each batch contains several virtual nodes, and the data blocks are transmitted sequentially according to the batch order. After the transmission of each batch is completed, the first thread is notified to update the status of the corresponding virtual nodes.

7. The method for reassembling and forwarding data blocks in the process of adding or deleting nodes in a database shared storage cluster according to claim 6, characterized in that, The second thread sends virtual node data blocks in batches as follows: Before sending a batch, the second thread sends a request to the first thread to send the virtual nodes of the current batch. This request contains a list of virtual node numbers to be sent. After receiving the send request, the first thread updates the state of each virtual node involved in the request from the pre-migration state to the in-migration state, and replies to the second thread with a send permission instruction; After receiving the permission to send instruction, the second thread iterates through each virtual node in the batch, reads all the data blocks that the virtual node is responsible for from the local cache or storage, and sends them to the target node through the network connection. After receiving all the data blocks from a virtual node, the target node broadcasts a message to the entire cluster that the virtual node has finished receiving the data; upon receiving the broadcast message, the first thread updates the status of the virtual node to the migration complete status. Repeat the above interaction process until all batches of virtual nodes have been migrated.

8. The method for reassembling and forwarding data blocks in the process of adding or deleting nodes in a database shared storage cluster according to claim 1, characterized in that, In step S4, the forwarding path of the request is determined based on the current state of the virtual node as follows: When the first thread receives a data block access request from the business thread, it uses the data block identifier information carried in the request to re-determine the virtual node number to which the data block belongs through hash calculation; it queries the locally maintained virtual node mapping table to obtain the current mapped physical node of the virtual node, and checks the status of the virtual node on the current node; If the current node is not in a reorganization state, the request will be sent to the corresponding physical node for processing according to the original mapping relationship; If the current node is in a reorganization state and the virtual node is in a normal state or before migration, the request will be sent to the original mapped physical node; if the current node is in a reorganization state and the virtual node is in a migration state or has completed migration, the request will be forwarded to the new mapped physical node.

9. The method for reassembling and forwarding data blocks in the process of adding or deleting nodes in a database shared storage cluster according to claim 8, characterized in that, In step S4, the processing flow after the request is forwarded to the target physical node is as follows: The first thread of the target physical node receives the forwarded data block access request and hands it over to the second thread of the same node for processing. The second thread checks the local status of the virtual node to which the requested data block belongs. If the virtual node is in the process of migration, the second thread reads the requested data block from the local cache or storage, constructs a response message, and returns it to the original requesting node. If the virtual node's status is "migration complete," the second thread will forward the request back to the first thread of this node, which will then process the request using the latest metadata information and return a response.

10. A computer device, characterized in that, The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method according to any one of claims 1-9.

Images