Method and apparatus for replica self-adaptation based on graph database

Abstract

The application discloses a kind of based on graph database's copy adaptive adjustment method and device.The method is by in cluster creation stage, to the corresponding expected copy value of each Zone of predefinition is set;Real-time detection is reported on the copy state of storage layer, and the PartView of current data copy distribution and the StoreView of physical resource use condition are obtained;The number of copies in PartView is compared with the expected copy value;When the number of copies is different from the expected copy value, generate copy change instance, and according to StoreView, select target storage node through filter and speed limiter, perform the expansion or shrinkage operation of copy;When the number of copies is same with the expected copy value, copy adjustment process ends.The application can achieve the technical effect of improving the availability of database.

Description

Technical Field

[0001] This invention relates to the field of graph database technology applications, and in particular to a method and apparatus for adaptive adjustment of replicas based on a graph database. Background Technology

[0002] Traditional graph databases have some limitations in data storage and management, such as:

[0003] Regarding the static number of replicas, if the number of replicas is fixed during cluster deployment, it is impossible to lower the replica threshold in time to ensure writable service when the primary area is undergoing maintenance failure.

[0004] Regarding manual intervention, operations such as master-slave failover and replica completion require manual maintenance, resulting in a high RTO.

[0005] Regarding disk failures, when a storage node experiences a disk failure that results in data loss, it does not automatically replenish replicas and remains in a state of degraded replica count for an extended period, increasing the risk of data loss.

[0006] There is currently no effective solution to the problem of low operating efficiency of graph databases due to the fixed number of replicas required by traditional graph databases and the need for manual intervention. Summary of the Invention

[0007] To address the aforementioned technical problems, embodiments of the present invention aim to provide a method and apparatus for adaptive adjustment of replicas based on graph databases, thereby at least solving the problem of low operating efficiency of graph databases due to the need for manual intervention caused by the fixed number of replicas in traditional graph databases.

[0008] The technical solution of this invention is implemented as follows:

[0009] This invention provides a method for adaptive replica adjustment based on a graph database, comprising: during the cluster creation phase, setting a corresponding expected replica value for each predefined Zone; real-time monitoring of the replica status reported by the storage layer to obtain the PartView of the current data replica distribution and the StoreView of the physical resource usage status; comparing the number of replicas in the PartView with the expected replica value; when the number of replicas differs from the expected replica value, generating a replica change instance, and selecting a target storage node based on the StoreView through filters and rate limiters to perform replica scaling up or scaling down operations; when the number of replicas is the same as the expected replica value, the replica adjustment process ends.

[0010] Optionally, the method further includes: when the primary zone fails, sending a metadata replica scaling-down request to the raft group of the metadata service metad; performing a member change operation on the raft group of the metadata service metad according to the metadata replica scaling-down request, making the metadata service metad of the standby zone the leader node; after the metadata service metad of the standby zone becomes the leader node, sending a data replica scaling-down request to the leader node; performing a data replica scaling-down operation on the leader node according to the data replica scaling-down request, so that the leaders of all data shards are concentrated in the standby zone, and the standby zone provides write services; when the primary zone recovers, sending a metadata replica scaling-up request to the metadata service metad, the metadata replica scaling-up request being used to restore the number of replicas of the metadata service metad; after the number of replicas of the metadata service metad is restored, sending a data replica scaling-up request to the metadata service metad of the standby zone of the leader node; performing a data shard replica scaling-up operation according to the data replica scaling-up request and the expected replica value; and performing a balance operation when the number of replicas of all data shards reaches the expected replica value. The leader operation makes the leader of the data shard appear in the primary area, and the primary area resumes providing write services.

[0011] Optionally, selecting a target storage node based on a filter in StoreView includes: selecting a target storage node to host the new replica based on at least one factor, including: isolation, node capacity and status, number of snapshots, and replica filling efficiency.

[0012] Optionally, a rate limiter can be used to maintain each storage node. During the replication expansion process, the allocation rate of storage nodes selected for multiple data shards is controlled so that the selection of storage nodes for multiple data shards is staggered and resources are allocated evenly.

[0013] Optionally, performing replica scaling operations includes: sending a metadata replica adjustment command to the metadata service metad via the ngctl tool; sending a data replica adjustment command to the metadata service metad via the ngql tool; and having the leader node perform the replica adjustment process.

[0014] Optionally, the method further includes: when a disk failure of a storage node is detected, resulting in data loss, automatically reducing the size of the replicas of the affected shards; and when the number of replicas of a shard is found to be lower than the expected replica value in subsequent detection, automatically triggering the replica expansion process, selecting a new storage node, and issuing a Raft group to the new storage node to complete the automatic data recovery.

[0015] Optionally, the method further includes: in the data import scenario, constructing a single-replica cluster; wherein, replica expansion is initiated by specifying the desired number of replicas for a single zone or multiple zones, and the metadata service metad completes the distribution adjustment of replicas based on filters and rate limiters.

[0016] Optionally, the method further includes: when the metadata service metad receives a replica change request, persisting the change information to local storage; periodically reading the global configuration through the background scheduling thread of the metadata service metad, comparing the shard information with the expected replica value, and triggering the replica adjustment process when there is a discrepancy.

[0017] Optionally, the method runs in a graph database system consisting of a metadata service metad, a storage service stored, and a query service graphd. The query service graphd periodically retrieves sharded distribution views from the metadata service metad and adjusts the access strategy based on the availability status of the Zone.

[0018] This invention provides an apparatus for adaptive replica adjustment based on a graph database, comprising: a configuration module for setting a corresponding expected replica value for each predefined Zone during the cluster creation phase; a detection module for real-time detection of the replica status reported by the storage layer to obtain a PartView of the current data replica distribution and a StoreView of the physical resource usage status; a comparison module for comparing the number of replicas in the PartView with the expected replica value; a first adjustment module for generating a replica change instance when the number of replicas differs from the expected replica value, and selecting a target storage node based on the StoreView through a filter and a rate limiter to perform replica scaling up or scaling down operations; and a second adjustment module for ending the replica adjustment process when the number of replicas is the same as the expected replica value.

[0019] This invention provides a method and apparatus for adaptive replica adjustment based on a graph database. During cluster creation, a corresponding expected replica value is set for each predefined Zone; the replica status reported by the storage layer is monitored in real time to obtain the PartView of the current data replica distribution and the StoreView of the physical resource usage; the number of replicas in the PartView is compared with the expected replica value; when the number of replicas differs from the expected replica value, a replica change instance is generated, and a target storage node is selected based on the StoreView using filters and rate limiters to perform replica scaling up or down operations; when the number of replicas is the same as the expected replica value, the replica adjustment process ends, thereby achieving the technical effect of improving database availability. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0021] Figure 1 A flowchart illustrating a method for adaptive adjustment of replicas based on a graph database, provided in an embodiment of the present invention;

[0022] Figure 2 A flowchart illustrating another method for adaptive adjustment of replicas based on a graph database, provided in an embodiment of the present invention;

[0023] Figure 3 This is a schematic diagram of an apparatus for adaptive adjustment of replicas based on a graph database, provided as an embodiment of the present invention. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish different objects, rather than to limit a specific order.

[0026] It should also be noted that the various embodiments of the present invention described below can be executed individually or in combination with each other, and the embodiments of the present invention do not impose specific limitations in this regard.

[0027] Technical terms used in the embodiments of this application:

[0028] Graph Database: A type of database that stores data in a graph structure, enabling efficient processing of complex relational data.

[0029] Primary zone: The geographical area or availability zone (AZ) that provides read and write services under normal conditions.

[0030] Backup zone: Geographically isolated from the primary zone, the availability zone is used for disaster recovery and usually serves as a read-only or warm backup zone.

[0031] High availability: The ability of a system to continue providing services even if some components fail.

[0032] Replica: Multiple copies of data, used to improve data reliability and availability.

[0033] Expected replica count: The target number of replicas set by the user for the data.

[0034] Partition failure: The primary zone becomes unreachable due to network outages, power failures, or node crashes, causing the partition to lose its service capabilities.

[0035] Disk failure: A single node's disk is damaged or data is lost, but the node's processes / OS are still alive.

[0036] metad: Metadata service.

[0037] graphd: A stateless query service.

[0038] store / storaged: Storage service.

[0039] leader: The primary node in a multi-replica cluster.

[0040] ngctl: A built-in tool in Nebula Graph used to interact with metad and adjust resources.

[0041] ngql: A built-in tool in Nebula graph used to interact with graphd and execute GQL statements.

[0042] Raft group: One set of data has multiple replicas, and the replicas use the Raft synchronization protocol to ensure strong data consistency.

[0043] Reset / Member Change: The former forces a change to the number of replicas in a Raft group if a majority is unavailable; the latter ensures that the member change process proceeds normally if a majority is available.

[0044] Balance leader: This involves performing a balance adjustment of the leader distribution to ensure that the number of leaders on each node is roughly even.

[0045] This invention provides a method for adaptive adjustment of replicas based on a graph database. Figure 1 This is a flowchart illustrating a method for adaptive replication adjustment based on a graph database, provided by an embodiment of the present invention; as shown below. Figure 1 As shown in the embodiments of this application, the method for adaptive adjustment of replicas based on a graph database includes:

[0046] Step S100: During the cluster creation phase, set the corresponding expected replica value for each predefined Zone;

[0047] Step S102: Real-time detection of the replica status reported by the storage layer to obtain the PartView of the current data replica distribution and the StoreView of the physical resource usage status;

[0048] Step S104: Compare the number of replicas in PartView with the expected number of replicas;

[0049] Step S106: When the number of replicas differs from the expected number of replicas, a replica change instance is generated, and the target storage node is selected based on the StoreView through filters and rate limiters to perform replica scaling up or scaling down operations.

[0050] Step S108: When the number of replicas is the same as the expected number of replicas, the replica adjustment process ends.

[0051] Optionally, the method for adaptive adjustment of replicas based on a graph database provided in this application embodiment further includes: when the primary zone fails, sending a metadata replica scaling-down request to the raft group of the metadata service metad; performing a member change operation on the raft group of the metadata service metad according to the metadata replica scaling-down request, so that the metadata service metad of the backup zone becomes the leader node; after the metadata service metad of the backup zone becomes the leader node, sending a data replica scaling-down request to the leader node; and performing a data replica scaling-down operation on the leader node according to the data replica scaling-down request, so that the leaders of all data shards are concentrated in the backup zone, and the backup zone provides write services.

[0052] Further, optionally, after the primary zone recovers, a metadata replica expansion request is sent to the metadata service metad to restore the number of replicas in the metadata service metad. After the number of replicas in the metadata service metad is restored, a data replica expansion request is sent to the metadata service metad of the leader node's backup zone. The data shard replica expansion operation is performed according to the data replica expansion request and the expected replica value. When the number of replicas of all data shards reaches the expected replica value, a balance leader operation is performed to make the leader of the data shard appear in the primary zone, and the primary zone resumes providing write services.

[0053] Optionally, selecting a target storage node based on a filter in StoreView includes: selecting a target storage node to host the new replica based on at least one factor, including: isolation, node capacity and status, number of snapshots, and replica filling efficiency.

[0054] Optionally, a rate limiter can be used to maintain each storage node. During the replication expansion process, the allocation rate of storage nodes selected for multiple data shards is controlled so that the selection of storage nodes for multiple data shards is staggered and resources are allocated evenly.

[0055] Optionally, the method for adaptive adjustment of replicas based on graph databases provided in this application embodiment further includes: when a disk failure of a storage node is detected, resulting in data loss, automatically reducing the replicas of the affected shards; when the number of replicas of a shard is identified as lower than the expected replica value in subsequent detection, automatically triggering the replica expansion process, selecting a new storage node, and issuing a Raft group to the new storage node to complete the automatic data recovery.

[0056] Optionally, the method for adaptive adjustment of replicas based on graph databases provided in this application embodiment further includes: constructing a single-replica cluster in a data import scenario; wherein, replica expansion is initiated by specifying the desired number of replicas for a single zone or multiple zones, and the metadata service metad completes the distribution adjustment of replicas according to filters and rate limiters.

[0057] Optionally, performing replica scaling operations includes: sending a metadata replica adjustment command to the metadata service metad via the ngctl tool; sending a data replica adjustment command to the metadata service metad via the ngql tool; and having the leader node perform the replica adjustment process.

[0058] Optionally, the method for adaptive replica adjustment based on graph database provided in this application embodiment further includes: when the metadata service metad receives a replica change request, it persists the change information to the local machine; the background scheduling thread of the metadata service metad periodically reads the global configuration, compares the sharding information with the expected replica value, and triggers the replica adjustment process when there is a discrepancy.

[0059] Optionally, the method for adaptive adjustment of replicas based on graph database provided in this application embodiment runs in a graph database system composed of metadata service metad, storage service stored, and query service graphd. The query service graphd periodically obtains the sharded distribution view from the metadata service metad and adjusts the access strategy according to the availability status of the Zone.

[0060] In conclusion, Figure 2 A flowchart illustrating another method for adaptive replication adjustment based on a graph database provided in this embodiment of the invention; as shown below. Figure 2 As shown in the embodiments of this application, the method for adaptive adjustment of replicas based on graph databases is as follows:

[0061] The method for adaptive adjustment of replicas based on graph databases provided in this application mainly includes the following functions:

[0062] Feature 1: High Availability Support: Allows users to customize the number of Zone replicas. When the primary Zone fails, the number of replicas is adjusted to enable read and write capabilities in the backup zone. When the primary Zone recovers, the number of replicas is adjusted again, and the primary Zone regains write capabilities. During this process, the user only needs to adjust the number of replicas; the service itself handles the primary / backup switchover and replica replenishment.

[0063] Function 2: Data Loss: When the upper layer detects data loss in a node, it will immediately shrink the corresponding shard replicas and then adaptively expand the replicas to ensure the number of replicas.

[0064] like Figure 2 As shown, at the entry point, the operations to adjust metad and stored are performed using the ngctl and ngql tools, respectively.

[0065] When adjusting metad, you need to specify the specific members to be changed in the command, as well as whether a forced reset operation is required.

[0066] To adjust stored replicas, the request is submitted to the metad leader node, which then handles the specific replica adjustment.

[0067] For a complete flowchart of the dungeon expansion process, in Figure 2 The text has already been marked:

[0068] After a front-end user initiates a request to change the replica, metad will write the expansion information to the local machine and ensure its persistence.

[0069] metad's background scheduling thread periodically reads the global configuration to check if any changes have been made.

[0070] After confirming the changes, the next step is to iterate through the shard information and compare it to see if any inconsistencies occur.

[0071] If inconsistencies are found, expansion is considered. Before expansion, the optimal storage needs to be found. Since a large number of fragment expansions are involved, this application embodiment requires filters and rate limiters to control the success rate and speed of fragment expansion.

[0072] Once the optimal solution is found, begin issuing commands for replica creation and synchronization requests.

[0073] Finally, metad waits for the heartbeat information reported by stored to confirm whether the expansion is complete, and compares the latest view information with the expected value.

[0074] After the changes are completed, wait for the next configuration change request. This completes a full sharding change process.

[0075] In a preferred example, during a disaster recovery scenario, when the primary zone fails, the built-in tool `ngctl` inputs the command `scalemetad = {…}` to send a request to scale down the replicas of the metad in the standby zone. The built-in tool `ngql` inputs the command `scale replica = {…}` to send a request to scale down the data replicas of the metad after it has scaled down and become the leader. All data shard leaders will be concentrated in the standby zone, providing write services. When the primary zone recovers, the number of metads and data replicas is readjusted using `ngctl` commands. The service internally adaptively adjusts the number of replicas, and the primary zone regains write services.

[0076] In a preferred embodiment, the method for adaptive adjustment of replicas based on a graph database provided in this application includes the following specific steps during the initialization process:

[0077] During the cluster creation phase, the default expected values ​​for each Zone are given during the creation process. This includes the corresponding Zone information, such as priority and number of replicas, as shown below:

[0078] ngctl svcgrp add-zone <cluster><zone_name> -p <priority> -r <replica-factor>

[0079] After the resources are created, the system internally records all zones created in the cluster through a sub-table, the zone table, including the corresponding number of replicas and priority. All creation processes are based on the user's usage scenario, allowing users to customize zone size, master-slave zone partitioning, and individual resource allocation.

[0080] In a preferred embodiment, the process of a primary region failure in the graph database-based replica adaptive adjustment method provided in this application is as follows:

[0081] The command `ngctl scale meta {h1:p1} [force]` performs a scale operation on the meta tag, as follows:

[0082] The entire metad replica is a raft group, so only one member change needs to be performed here. Since the member may no longer be available, a forced reset is used instead of a member change.

[0083] The command `ngql > scale replica {zone1=0, zone2=1} [force]` performs a scale-down operation on the data replicas, as follows:

[0084] Metad internally maintains a global configuration to record expected values. For example, during the initialization phase, if the data is configured to have 3 replicas, the expected value will be recorded as 3 after initialization. At the same time, after Storaged reports complete replica information, it is confirmed that the value matches the expected value, and the initialization is thus completed.

[0085] PartView reflects the latest data view in the storage layer, while StoreView reflects information at the physical layer, such as resource usage of a single machine and snapshot information.

[0086] The scheduler periodically compares the number of replicas in the view with the expected value. When a discrepancy is confirmed, a change instance is created to handle the inconsistency. In the case of scaling down, forced replica scaling down is required, so only a reset request needs to be executed on all shards.

[0087] Once it is confirmed that PartView matches the expected value, it is considered that the expectation has been met, and no more change instances will be generated. At this point, the scaling down is complete.

[0088] In a preferred example, the process of the graph database-based replica adaptive adjustment method provided in this application during operation is as follows:

[0089] In terms of access methods, graphd periodically obtains the latest shard distribution view information from the metad node. When it finds that only the backup zone is available, the zone-based access strategy will fail, which is equivalent to the absence of the concept of zone.

[0090] In a preferred embodiment, the method for adaptive replication based on a graph database provided in this application, during the primary area recovery process, is as follows:

[0091] The command `ngctl scale meta {h1:p1, h2:p2, h3:p3}` is used to perform a meta scale operation.

[0092] The command `ngql > scale replica {zone1=2, zone2=1}` is used to scale up the data replicas.

[0093] Specifically, when data replicas are found to be inconsistent, the corresponding replicas will be expanded according to the number of replicas specified for each Zone.

[0094] Because this involves changes to all shards, a filter and rate limiter are provided to restrict the content and rate at which changed instances are generated, preventing instability to the cluster. The specific execution process is as follows:

[0095] Suppose there is initially 1 storage node with 3 shards and 1 replica. Now, 3 more machines are added, and the expected replica count is adjusted to 3. The scheduler in metad will confirm the inconsistency in replica count and then create a change instance for each shard to execute the change.

[0096] When selecting multiple replicas, factors such as isolation (the same stored node cannot store multiple replicas of the same shard), node capacity / status, number of snapshots, and replica filling efficiency need to be considered. Each factor is organized into a filter, which is used when selecting valid stored nodes and will affect the final selected stored node.

[0097] Because all change instances are executed sequentially during the actual execution phase, the information seen is consistent when all stored nodes are in normal status. For example, when shard 1 selects a stored node, it sees valid nodes 2, 3, and 4. Shard 2 also sees 2, 3, and 4, and so does shard 3. Since the newly added machines are all empty, the sorting rules are consistent. This leads to a problem: if shard 1 selects 2 and 3, shards 2 and 3 will also select 2 and 3 based on what they see, resulting in an empty machine and wasted resources. The fundamental problem here is that the selection by a shard does not affect other shards. The solution is to maintain a rate limiter for each stored node, which can be controlled at the second level. Simultaneously, a stored node is selected each time until the expected value is met. In the same scenario, shard 1 selects 2, shard 2 selects 3, and shard 3 selects 4; this loop repeats, waiting for the N-second limit. Then, shard 1 selects 3, shard 2 selects 4, and shard 3 selects 1. At this point, the new replica position is successfully confirmed, and the replica creation and synchronization process begins.

[0098] Similarly, when it is confirmed that the PartView matches the expected value, it is considered that the expectation has been met, and no more Events will be generated. At this point, the expansion is complete.

[0099] Since the underlying implementation uses Raft, a balance leader operation is required to ensure that the leader appears in the primary partition, and the primary partition provides write services.

[0100] In a preferred embodiment, the method for adaptively adjusting replicas based on a graph database provided in this application proceeds as follows in other scenarios:

[0101] Scenario 1, Derivative:

[0102] Build a single-replica cluster (here, you can set the number of replicas to 1 when creating it using ngctl, and keep other operations unchanged; the derivative operation is generally performed through a derivative tool inside Nebula Graph).

[0103] Start expanding the replica count (the parameter for replica expansion here can be specified for a single zone or multiple zones, indicating the corresponding desired number of replicas). If there is a need for a new zone, a new zone will be created here, and the number of replicas will be specified as 0.

[0104] Once all Zones have been created, the scale replace command will be executed.

[0105] Scenario 2: Disk failure, data loss (3 copies):

[0106] When the storage layer reports, it will detect the loss of fragments. At this time, it will perform a replica reduction operation on the current fragment from 3 replicas to 2 replicas.

[0107] During the next check, metad will detect that the number of shard replicas is less than expected and trigger the replica expansion behavior. Internally, it will select a new storage node and simultaneously issue a Raft group.

[0108] At this point, no manual intervention is required to complete the automatic expansion of the copy.

[0109] In a preferred example, the method for adaptive replica adjustment based on a graph database provided in this application embodiment is used during the cluster creation phase:

[0110] By establishing a Zone setting and then performing real-time sensing, a collaborative foundation for rate control through multi-factor filtering is established, providing closed-loop support for subsequent adaptive adjustments.

[0111] First, perform the Zone configuration operation using the command `ngctl svcgrp add-zone`. <cluster><zone_name> -p <priority> -r <replica-factor>Create Zone information, where -p <priority>Specify the service priority of the Zone (primary zone has higher priority than backup zone), -r <replica-factor>Set the expected replica value for the corresponding Zone. The sub-table "zone table" records the replica count, priority, and resource allocation information of all Zones in the cluster. This expected replica value will serve as the benchmark threshold for all subsequent adjustment operations, directly related to the triggering conditions for real-time sensing, the filtering guidance for multi-factor filtering, and the execution scale for rate control.

[0112] For example, a primary zone (Zone1, priority P1=3) and a backup zone (Zone2, priority P2=2) are created, with the expected replica count configured as Zone1=2, Zone2=1, and a total of 3 replicas. This configuration not only clarifies the replica distribution target under normal conditions but also presets priority rules for adaptive failover in failure scenarios. That is, when the primary zone fails, the backup zone can quickly take over the service based on its own priority and expected replica count; after the primary zone recovers, the system can restore the primary zone's write service permissions according to priority.

[0113] Simultaneously, during cluster initialization, the detection module synchronously starts a real-time monitoring mechanism, periodically acquiring the replica status reported by the storage layer, and initially constructing PartView (shard-replica mapping view) and StoreView (physical resource association view such as node capacity, number of snapshots, liveness status, and load status); the multi-factor filter completes the initial parameter configuration, preset the filtering weights of isolation, node capacity and status, number of snapshots, and replica filling efficiency (isolation 0.3, node capacity and status 0.25, number of snapshots 0.15, replica filling efficiency 0.1); the rate limiter initializes the allocation counter for each stored node, setting the reset period to 1 second and the maximum number of replica allocation requests per second to 2, laying the foundation for balanced resource allocation in subsequent scaling operations.

[0114] In a preferred example, the replica adaptive adjustment method based on a graph database provided in this application embodiment applies to a primary region failure scenario:

[0115] When a partition failure occurs in the primary zone, such as network interruption, power failure, or node failure, the replica adaptive adjustment method based on graph database provided in this application performs multi-factor filtering through real-time perception, and then performs coordinated linkage of rate control to achieve rapid degradation and backup zone takeover without manual intervention. The specific coordination process is as follows:

[0116] Step 1, Real-time Sensing Trigger: By periodically monitoring the heartbeat signals of stored nodes, it was found that the heartbeat of all nodes in Zone 1 of the primary zone was lost. The PartView and StoreView were obtained simultaneously. The PartView showed that the number of replicas of each shard was 3 (Zone 1=2, Zone 2=1). The StoreView confirmed that the two stored nodes in Zone 2 were in a normal state (capacity utilization of 60%, number of snapshots of 2, and load below the threshold). Then, the replica adjustment signal was triggered, and the status information was synchronized to the metad's raft group.

[0117] Step 2, Metadata Replica Collaborative Scaling Down: Users send a metadata replica scaling down request to the metadata service metad's raft group by entering the command `ngctl scale meta{h1:p1} [force]` using the ngctl tool. Since the primary zone node is unavailable, metad performs a forced reset operation instead of a regular member change, executing a raft group member change and selecting the metad node with the best status in Zone 2 to be promoted to leader. That is, this selection process incorporates the core logic of multi-factor filtering, prioritizing nodes with sufficient capacity, fewer snapshots, and faster response rates to ensure the rapid recovery of the metadata service.

[0118] Step 3, Data Replica Collaborative Scaling Down: After the standby metad becomes the leader node, the command `ngql > scale replica {zone1=0, zone2=1} [force]` is used in the ngql tool to send a data replica scaling down request to the leader node. The metad scheduler, based on physical resource information in the StoreView, uses a multi-factor filter to screen available nodes in Zone2: First, isolation is the primary condition, excluding nodes that might exist in the same shard replica set; then, a weighted scoring algorithm is used to comprehensively score the nodes in Zone2 (example node score 0.9) to determine the optimal candidate node; finally, a rate limiter controls the scaling down rate, processing only one shard scaling down operation per second to avoid consistency protocol conflicts caused by simultaneous operations.

[0119] Step 4, Seamless Service Takeover: After the scaling down operation is completed, the leaders of all data shards are concentrated in Zone 2. PartView displays that the number of replicas for each shard is consistent with the adjusted expected replica values ​​(Zone1=0, Zone2=1), and the backup zone officially provides write services. The entire collaboration process takes no more than 30 seconds, significantly reducing the RTO compared to the existing technology's RTO of over 30 minutes. This is primarily due to the closed-loop collaboration that ensures stability through real-time perception for rapid triggering, filtering and selecting the optimal node, and rate control.

[0120] In a preferred example, the graph database-based replica adaptive adjustment method provided in this application embodiment is applied in a primary region recovery scenario:

[0121] Once the primary zone failure is resolved and the nodes return to normal, a multi-stage collaborative scaling operation is performed to restore the primary zone's write service capability. The specific collaborative process is as follows:

[0122] Step 1, Metadata Replica Collaborative Scaling: Send a metadata replica scaling request to metad via the command ngctl scale meta {h1:p1, h2:p2, h3:p3}. metad uses StoreView to detect the node status (capacity, load, number of snapshots, etc.) of the primary zone recovery, filters nodes that meet the conditions through a multi-factor filter, and re-adds them to the raftgroup to restore the number of replicas of the metadata service, ensuring the redundancy and reliability of the control plane.

[0123] Step 2, Collaborative Data Replica Scaling: After the number of metadata replicas is restored, a data replica scaling request is sent via the command `ngql > scalereplica {zone1=2, zone2=1}`. The metad scheduler compares the current PartView (replica count 1) with the expected replica values ​​(Zone1=2, Zone2=1). Upon confirming the discrepancy, a replica change instance is generated, and a collaborative execution mechanism for filtering and rate limiting is initiated.

[0124] Multi-factor filtering: For each shard, isolation is the primary condition to ensure that new replicas of the same shard are not distributed in the same zone or adjacent nodes as existing replicas; then, the capacity utilization rate (must be less than 80%), status (running normally), number of snapshots (≤3), and replica filling efficiency (≥100MB / s) of the primary recovery node are combined to score the optimal candidate nodes (primary nodes are given priority because they have higher priority).

[0125] Rate-controlled allocation: The rate limiter controls the allocation rate based on a counter, with each node receiving a maximum of 2 shard replica requests per second. For example, in a cluster with 3 shards, when the primary region recovers 3 nodes, during expansion, shard 1 first selects primary region node A, shard 2 selects primary region node B, and shard 3 selects primary region node C. After waiting for 1 second for the counter to reset, shard 1 then selects primary region node B, shard 2 selects primary region node C, and shard 3 selects primary region node A, ensuring even resource distribution and avoiding node overload.

[0126] Step 3, Leader Balancing and Cooperative Switchover: Once the number of replicas for all data shards reaches the desired replica count, a leader balancing operation is executed. This operation also involves a multi-factor filter, migrating the leader nodes of data shards to the primary zone nodes based on factors such as node load and the number of leaders. During the filtering process, nodes with fewer leaders and lower loads in the primary zone are prioritized to ensure load balancing across the primary zone nodes. Ultimately, the primary zone resumes write services, and the standby zone resumes its warm standby role. The entire collaborative scaling and switchover process is smooth and uninterrupted.

[0127] In a preferred example, the graph database-based replica adaptive adjustment method provided in this application embodiment is applied in disk failure scenarios:

[0128] When a storage node experiences disk failure leading to data loss (disk failure), the graph database-based replica adaptive adjustment method provided in this application achieves automatic self-healing in a 3-2-3 manner through real-time sensing and a closed-loop coordination of scaling down, filtering, rate limiting, and scaling up. The specific process is as follows:

[0129] Step 1, Real-time Fault Detection and Emergency Scaling Down: Based on the status information reported by the storage layer, a disk failure signal on a specific stored node is detected in real time, confirming the loss of replicas for 10 shards on that node. PartView displays that the replica count of these shards has decreased from 3 to 2. To maintain the normal operation of the consistency protocol for the remaining replicas, the collaborative adjustment module automatically triggers a scaling down operation, officially reducing the replica count of the affected shards to 2. Simultaneously, PartView is updated, and the fault information is synchronized to metad.

[0130] Step 2, Difference Detection and Expansion Trigger: The background scheduling thread of metad periodically reads the global configuration, compares the shard information with the expected replica value, and finds that the number of replicas (2) of the affected shard is lower than the expected replica value (3). It immediately triggers the replica expansion process and generates a replica change instance containing the target number of replicas (3) and the identifier of the shard to be adjusted.

[0131] Step 3, Filtering and Rate Limiting Collaborative Scaling: Metad, based on StoreView, uses a multi-factor filter to select new storage nodes, prioritizing nodes in different zones from failed nodes, with capacity utilization below 70%, fewer snapshots, and high fill efficiency (example score 0.85). A rate limiter controls the scaling rate, allocating one shard replica request per second to each target node to avoid resource contention. Subsequently, Raft groups are distributed to the selected nodes, and data synchronization is completed via the Raft synchronization protocol.

[0132] Step 4, Self-Healing Completion Verification: The detection module monitors the replica synchronization progress in real time and periodically obtains PartView. When it is confirmed that the replica count of all affected shards has recovered to 3, consistent with the expected replica count, the self-healing process ends. The entire process requires no manual intervention, takes no more than 10 minutes, and reduces the risk of data loss to below 0.01%. This is primarily due to the multi-stage closed-loop collaboration of emergency handling through fault detection, followed by synchronous recovery through screening and allocation.

[0133] In a preferred example, the method for adaptively adjusting replicas based on a graph database provided in this application embodiment is applied in a data import scenario:

[0134] In data import scenarios, to improve import speed and reduce the probability of failure, a single-replica cluster is built and efficiently scaled through multi-stage collaboration. The specific process is as follows:

[0135] Step 1, Single-Replica Cluster Construction: When creating the cluster using the ngctl tool, set the expected replica value of all Zones to 1 to build a single-replica cluster, avoiding the impact of multi-replica synchronization on import speed. At this time, the detection module continuously monitors the storage layer status, PartView records the shard distribution of the single replica, and StoreView updates the node capacity usage and load in real time.

[0136] Step 2, Collaborative Scaling Execution: After data import is complete, a scaling request is initiated by specifying the desired replica values ​​for a single or multiple zones (e.g., ngql > scale replica {zone1=2, zone2=1}). After metad generates a replica change instance, it coordinates with multi-factor filters and rate limiters to complete the replica distribution adjustment.

[0137] The filter prioritizes nodes with sufficient capacity, low load, and few snapshots to ensure efficient replica synchronization.

[0138] The rate limiter controls the expansion rate to avoid cluster instability caused by a large number of shards synchronizing simultaneously, ensuring the safety of imported data and rapid completion of redundant configuration.

[0139] In a preferred embodiment, the detailed workflow of the multi-stage collaborative adaptive adjustment mechanism in the graph database-based replica adaptive adjustment method provided in this application is as follows:

[0140] The core of the graph database-based replica adaptive adjustment method provided in this application lies in constructing and running an adaptive system that tightly coordinates multiple stages, including target setting, state awareness, intelligent decision-making and equilibrium control, and closed-loop execution. The following details how each stage collaborates during system operation to achieve intelligent management of graph database replicas.

[0141] Step 1: Definition of each collaborative step and data interface

[0142] Target setting phase: Zone and desired replica value configuration

[0143] Maintain a global configuration table that records the unique identifier, priority, and corresponding expected replica value for each predefined availability zone. This value is set during cluster creation via management commands (such as ADD ZONE) and can be dynamically updated at runtime. For example, the expected replica value for the primary zone (Zone_A) can be set to 2, and for the backup zone (Zone_B) to 1, which forms the target baseline for the entire adaptive adjustment process.

[0144] The output of this step is a mapping relationship (Map).<Zone_ID, Expected_Replica_Count> This serves as the input for the decision-making comparison process.

[0145] State awareness process (dual views: PartView and StoreView):

[0146] The PartView (logical view) is generated as follows: Each storage service periodically (e.g., every second) reports the replica status of all data partitions it is responsible for to the metadata service (metad), including replica ID, node, Raft group role (Leader / Follower), log synchronization progress, etc. Metad aggregates the reported information from all nodes to generate a global PartView, which is essentially a data structure reflecting the actual number and distribution of replicas of each data partition in the cluster at the current moment.

[0147] The specific generation of StoreView (physical view): The storage service simultaneously reports the physical resource status of its node, including disk capacity utilization, memory usage, CPU load, network bandwidth, and the number of snapshot tasks being executed. After aggregation by metad, StoreView is generated, which is a data structure reflecting the health and load capacity of each storage node in the cluster at the current moment.

[0148] The update cycle of the PartView and StoreView views can be dynamically adjusted according to the system load. When a large difference in the number of replicas or a fault event is detected, the reporting and aggregation frequency can be temporarily increased to achieve adaptive perception accuracy.

[0149] Intelligent decision-making and balanced control (coordination between filters and speed limiters):

[0150] The decision trigger is set up so that the background scheduler periodically (e.g., every 5 seconds) compares the actual number of replicas for each shard in PartView with the expected value for the corresponding Zone provided in the target setting phase. Once an inconsistency is found (too many or too few), a replica change instance is generated for that shard.

[0151] A multi-factor filter (intelligent filtering) is used to score all candidate storage nodes for a shard that needs additional replicas. The scoring function considers multiple factors, such as:

[0152] Isolation factor: Prioritize nodes that are not in the same physical rack, switch, or availability zone as existing fragmented replicas to prevent cascading failures.

[0153] Capacity and status factors: Select nodes with sufficient disk space (e.g., utilization <80%), low load (normal CPU / memory utilization) and a "healthy" status.

[0154] Efficiency factor: Avoid selecting nodes that are currently performing a large number of snapshots or data compression to reduce interference with replica synchronization performance.

[0155] The filter outputs an ordered list of candidate nodes sorted in descending order of overall score.

[0156] The rate controller (rate limiter) maintains a lightweight token bucket rate limiter for each storage node. Each token represents a "permission" to add one replica task on that node. Tokens are replenished at a fixed rate (e.g., 1-2 per second), thereby controlling the upper limit of new replica creation tasks that a single node can handle per unit of time.

[0157] The collaborative decision-making process includes the following: when selecting a target node for a shard, it does not directly choose the first node in the filter list. Instead, it checks each node in the list sequentially to see if it has a usable token. It selects the first node with a usable token and consumes one of its tokens. This mechanism ensures that:

[0158] Spatial balance: High-scoring nodes consume tokens quickly due to frequent selection. Even if the filter outputs of subsequent shards are the same, they will switch to the second-best node because the node has no tokens, thus achieving a passive balanced distribution of replicas among high-quality nodes.

[0159] Time smoothing: The token replenishment rate limits the instantaneous pressure of change operations on individual nodes, making cluster load changes smoother and avoiding performance jitter caused by centralized operations.

[0160] Closed-loop execution and self-healing process:

[0161] During normal scaling up and down execution, based on the target node selected in the aforementioned collaborative process, the metadata service issues Raft member change instructions to the corresponding storage node. After execution, the new state will be captured in the next cycle of the state awareness process and will be compared again until PartView matches the expected value, forming a closed loop of execution, feedback, and verification.

[0162] Among them, the collaborative manifestation of disk fault closed-loop self-healing (3-2-3) is: this scenario is a high-level manifestation of the collaborative capabilities of the above-mentioned multiple links.

[0163] Phase 1: Rapid Detection and Response (Security Degradation). When StoreView detects a disk failure on a node (such as an IO error), the system immediately identifies the affected data shards.

[0164] At this point, skipping the complex filter scoring process, and directly aiming to ensure the availability of the remaining replica consistency protocol, a forced scaling down (member removal) request is initiated to the Raft group of the shard, reducing the number of replicas from N (e.g., 3) to N-1 (2). This step prioritizes ensuring the immediate availability of the service.

[0165] Phase 2: Steady-state collaborative recovery (intelligent reconstruction). In subsequent periodic scheduling, the scheduler discovers through PartView that the number of replicas (2) of the shard is lower than the expected value (3), and then generates a standard replica expansion change instance.

[0166] This example will go through the entire process of intelligent decision-making and balanced control, which involves selecting the optimal new node through a filter, obtaining operating permission through a rate limiter, smoothly performing the expansion operation, restoring the number of replicas from 2 to 3, and completing data synchronization.

[0167] By switching between rapid response and steady-state recovery modes, the method for adaptive adjustment of replicas based on graph databases provided in this application demonstrates its adaptive decision-making capability for different fault severity levels, achieving true closed-loop self-healing.

[0168] Step 2, Driving and Scheduling Collaborative Workflows

[0169] The entire multi-stage collaborative workflow is driven by a background scheduling engine. This engine does not operate through simple polling, but rather incorporates state machine logic:

[0170] Listen for configuration changes: Receive user modifications to the desired replica value.

[0171] Trigger global comparison: Compare the new expected value with the current PartView to generate a queue of pending part change instances.

[0172] Scheduled execution: Instances are retrieved from the queue and executed sequentially through the intelligent decision-making, load balancing control, and closed-loop execution processes described above. The rate limiter mechanism ensures that the execution of instances in the queue is staggered and smooth.

[0173] State synchronization and looping: Wait for the next perception cycle and verify the execution result using the updated PartView. If the expectation has been achieved, mark the instance as complete; if not (e.g., the target node operation failed), re-queue the instance or trigger the exception handling process. This process is repeated cyclically, ensuring that the system always converges towards the desired state.

[0174] As can be seen from the detailed workflow described above, the adaptive adjustment of the copy adaptive adjustment method based on graph database provided in this application embodiment is not a simple stacking of individual functions, but rather a deep coupling of multiple links through a carefully designed data flow interface, namely, the PartView and StoreView dual views; decision logic, namely, filters and rate limiters; and feedback loop, namely, execution verification, forming an intelligent collaborative system that can cope with various complex scenarios and has both efficiency and stability.

[0175] The core of the graph database-based replica adaptive adjustment method provided in this application lies in constructing a five-stage collaborative closed loop: Zone setting (baseline layer), real-time perception (trigger layer), multi-factor filtering (filtering layer), rate control (execution layer), and disk fault closed-loop self-healing (bottom layer). Each stage does not operate independently but rather supports each other and dynamically links together.

[0176] The baseline layer (Zone setting) provides goal orientation and priority rules for the entire collaboration, ensuring that adjustments are aligned with business needs;

[0177] The trigger layer (real-time perception) captures state changes through dual views, providing precise trigger signals for collaborative adjustments;

[0178] The filtering layer (multi-factor filtering) filters the optimal node based on multi-dimensional parameters, solving the core problem of "who to assign it to";

[0179] The execution layer (rate control) ensures even resource allocation and prevents filtered nodes from failing due to overload;

[0180] The bottom-level solution (disk fault self-healing) integrates fault handling into a collaborative closed loop, enabling automatic recovery from abnormal scenarios.

[0181] This collaborative closed-loop system effectively addresses the pain points of traditional graph database replica management, such as static rigidity, manual intervention, and resource imbalance. The collaborative effect can be verified through quantitative data: the collaborative rapid scaling down and switching mechanism between metadata and data replicas enables service recovery within minutes or even faster after a primary zone failure. Compared to traditional solutions relying on manual intervention, this significantly reduces the Recovery Time Objective (RTO), reducing the primary zone failure RTO from over 30 minutes to within 30 seconds. Through a closed-loop self-healing process of first scaling down for stability and then intelligently scaling up, data replica redundancy can be efficiently and automatically restored, greatly reducing the risk of permanent data loss due to disk failures. For example, the disk failure self-healing success rate reaches 99.9%, the scaling down / expansion failure rate drops from 30% to below 0.5%, and the storage node load balancing improves by 60%, significantly enhancing the availability, automation level, and resource utilization efficiency of the graph database.

[0182] It should be noted that the specific numerical values ​​involved in the graph database-based replica adaptive adjustment method provided in this application embodiment are only used as optimal examples for illustration, and are intended to achieve the graph database-based replica adaptive adjustment method provided in this application embodiment; no specific limitations are made. In the graph database-based replica adaptive adjustment method provided in this application embodiment, replicas can adaptively adjust, quickly recovering read and write capabilities after a primary area failure without manual intervention, improving availability and data import performance; in the event of fragmented data loss, internal 3-2-3 data recovery can be performed, achieving disk-level fault self-healing; during replica scaling, the next replica creation location is determined based on physical view information, and filters and limiters are used to maintain resource balance, achieving balanced resource usage.

[0183] This invention provides a method for adaptive replica adjustment based on a graph database. During cluster creation, a corresponding expected replica value is set for each predefined Zone. The replica status reported by the storage layer is monitored in real time to obtain the PartView of the current data replica distribution and the StoreView of the physical resource usage. The number of replicas in the PartView is compared with the expected replica value. When the number of replicas differs from the expected replica value, a replica change instance is generated. Based on the StoreView, a target storage node is selected using filters and rate limiters to perform replica scaling up or down operations. When the number of replicas matches the expected replica value, the replica adjustment process ends, thereby improving database availability.

[0184] This invention provides an apparatus for adaptive adjustment of replicas based on a graph database. Figure 3 This is a schematic diagram of a device for adaptive adjustment of replicas based on a graph database, provided in an embodiment of the present invention; as shown. Figure 3 As shown, the apparatus for adaptive adjustment of replicas based on a graph database provided in this application includes:

[0185] The configuration module 30 is used to set the corresponding expected replica value for each predefined Zone during the cluster creation phase; the detection module 32 is used to detect the replica status reported by the storage layer in real time, and obtain the PartView of the current data replica distribution and the StoreView of the physical resource usage status; the comparison module 34 is used to compare the number of replicas in the PartView with the expected replica value; the first adjustment module 36 is used to generate a replica change instance when the number of replicas is different from the expected replica value, and select the target storage node according to the StoreView through filters and rate limiters to perform replica expansion or contraction operations; the second adjustment module 38 is used to end the replica adjustment process when the number of replicas is the same as the expected replica value.

[0186] This invention provides a device for adaptive replica adjustment based on a graph database. During cluster creation, a corresponding expected replica value is set for each predefined Zone. The replica status reported by the storage layer is monitored in real time to obtain a PartView of the current data replica distribution and a StoreView of the physical resource usage. The number of replicas in the PartView is compared with the expected replica value. When the number of replicas differs from the expected replica value, a replica change instance is generated. Based on the StoreView, a target storage node is selected using filters and rate limiters to perform replica scaling up or down operations. When the number of replicas matches the expected replica value, the replica adjustment process ends, thereby improving the availability of the database.

[0187] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. < / priority> < / priority> < / cluster> < / priority> < / cluster>

Claims

1. A method for adaptive replica adjustment based on a graph database, characterized in that, include: During the cluster creation phase, a corresponding expected replica value is set for each predefined Zone; Real-time monitoring of the replica status reported by the storage layer yields a PartView of the current data replica distribution and a StoreView of the physical resource usage status. Compare the number of replicas in the PartView with the expected number of replicas; When the number of replicas differs from the expected replica value, a replica change instance is generated, and the target storage node is selected based on the StoreView through filters and rate limiters to perform replica scaling up or scaling down operations. The replica adjustment process ends when the number of replicas is the same as the expected replica value. The method further includes: When the primary zone fails, a request to reduce the size of the metadata replica is sent to the raft group of the metadata service metad. Based on the metadata replica scaling-down request, a member change operation is performed on the raft group of the metadata service metad, making the metadata service metad in the backup zone the leader node; After the metadata service metad in the backup zone becomes the leader node, it sends a data replica scaling-down request to the leader node. Based on the data replica scaling-down request, the leader node is subjected to a data replica scaling-down operation so that the leaders of all data shards are concentrated in the backup area, and the backup area provides write services. Once the primary region is restored, a metadata replica expansion request is sent to the metadata service metad. The metadata replica expansion request is used to restore the number of replicas of the metadata service metad. After the number of replicas of the metadata service metad is restored, a data replica expansion request is sent to the metadata service metad of the backup zone of the leader node; Perform data shard replication expansion operations based on the data replica expansion request and the desired replica value; Once the number of replicas for all data shards reaches the desired replica count, a balance leader operation is performed, causing the leader of the data shard to appear in the primary region, and the primary region resumes providing write services.

2. The method for adaptive adjustment of replicas based on a graph database according to claim 1, characterized in that, The step of selecting the target storage node based on the StoreView via a filter includes: The filter selects the target storage node for hosting the new replica based on at least one factor, including: isolation, node capacity and status, number of snapshots, and replica fill efficiency.

3. The method for adaptive adjustment of replicas based on a graph database according to claim 2, characterized in that, The scaling up or scaling down of the execution replica includes: Send metadata copy adjustment commands to the metadata service metad using the ngctl tool; Send a data copy adjustment command to the metadata service metad using the ngql tool; The leader node performs replica adjustment processing.

4. The method for adaptive adjustment of replicas based on a graph database according to claim 1, characterized in that, The rate limiter maintains each storage node, wherein during the replica expansion process, the allocation rate of storage nodes selected for multiple data shards is controlled so that the selection of storage nodes for the multiple data shards is staggered sequentially to achieve uniform resource allocation.

5. The method for adaptive replication adjustment based on a graph database according to claim 1, characterized in that, The method further includes: When a disk failure on a storage node is detected, resulting in data loss, the replicas of the affected shards are automatically scaled down. If, during subsequent detection, the number of replicas of the shard is found to be lower than the expected replica value, the replica expansion process is automatically triggered, a new storage node is selected, and a Raft group is sent to the new storage node to complete the automatic data recovery.

6. The method for adaptive replica adjustment based on a graph database according to claim 1, characterized in that, The method further includes: In the data import scenario, a single-replica cluster is constructed; wherein, replica expansion is initiated by specifying the desired number of replicas in a single zone or multiple zones, and the metadata service metad completes the distribution adjustment of replicas according to the filter and the rate limiter.

7. The method for adaptive replica adjustment based on a graph database according to claim 1, characterized in that, The method further includes: When the metadata service metad receives a copy change request, it persists the change information to local storage. The background scheduling thread of the metadata service metad periodically reads the global configuration, compares the shard information with the expected replica value, and triggers the replica adjustment process when there is a discrepancy.

8. The method for adaptive replica adjustment based on a graph database according to claim 1, characterized in that, The method operates in a graph database system consisting of a metadata service metad, a storage service stored, and a query service graphd. The query service graphd periodically obtains sharded distribution views from the metadata service metad and adjusts the access strategy according to the availability status of the Zone.

9. A device for adaptive adjustment of replicas based on a graph database, characterized in that, include: The configuration module is used to set the expected replica value for each predefined Zone during the cluster creation phase; The detection module is used to detect the replica status reported by the storage layer in real time, and obtain the PartView of the current data replica distribution and the StoreView of the physical resource usage status. The comparison module is used to compare the number of replicas in the PartView with the expected replica value; The first adjustment module is used to generate a replica change instance when the number of replicas is different from the expected replica value, and select the target storage node according to the StoreView through filters and rate limiters to perform replica expansion or reduction operations. The second adjustment module is used to end the replica adjustment process when the number of replicas is the same as the expected number of replicas. When the primary zone fails, a metadata replica reduction request is sent to the Raft group of the metadata service metad. Based on this request, a membership change operation is performed on the Raft group of the metadata service metad, making the metadata service metad in the backup zone the leader node. After the backup zone's metadata service metad becomes the leader node, a data replica reduction request is sent to the leader node. Based on this request, a data replica reduction operation is performed on the leader node, so that the leaders of all data shards are concentrated in the backup zone, which provides write services. When the primary zone recovers, a metadata replica expansion request is sent to the metadata service metad to restore the number of replicas in the metadata service metad. After the number of replicas in the metadata service metad is restored, a data replica expansion request is sent to the metadata service metad of the backup zone in the leader node. Based on the data replica expansion request and the desired replica value, a data shard replica expansion operation is performed. When the number of replicas in all data shards reaches the desired replica value, a balance operation is performed. The leader operation causes the leader of the data shard to appear in the primary area, and the primary area resumes providing write services.

Images