Storage and management method, system and electronic device of graph database

Abstract

The application relates to a storage and management method of a graph database, which comprises the following steps: determining a cluster deployment strategy based on a cluster creation instruction, creating corresponding Zones according to the cluster deployment strategy, and determining the number of replicas of each Zone; according to the number of replicas of each Zone and the total number of shards configured by the graph database, polling and allocating shard replicas on the physical machines of each Zone to obtain a shard allocation scheme, and creating corresponding shard replicas on the physical machines according to the shard allocation scheme; in response to a query instruction initiated by a client, obtaining a routing strategy and the latest shard distribution view information from a management node, and enabling a query engine in each Zone to access a storage engine of the Zone based on the routing strategy and the shard distribution view information. Through the application, the problem of low availability during deployment of a graph database service is solved. The graph database service is deployed based on Zones, and the granularity of the Zones can be customized, thereby realizing high availability at the cross-rack / cross-room level and the cross-region level.

Description

Technical Field

[0001] The present application relates to the field of data management, and in particular to storage and management methods, systems, and electronic devices for graph databases. Background Art

[0002] Graph databases are non-relational databases that store data using nodes, edges, and attributes, and are adept at handling complex relationships. In recent years, with the rise of applications such as social networks, recommendation systems, and knowledge graphs, graph database technology has rapidly developed and has become a critical infrastructure in the fields of big data and AI.

[0003] In related technologies, when deployed in multiple locations, graph database services all distribute data according to a unified principle, resulting in inflexible data distribution, low availability, and high latency due to cross-domain access from the service. Furthermore, when resources in a certain location are insufficient, it cannot be expanded independently. During data synchronization, unnecessary replica creation and relocation may occur, occupying excessive resources.

[0004] Currently, no effective solution has been proposed for the problem of low availability when deploying services in related technologies. Summary of the Invention

[0005] The embodiments of the present application provide a storage and management method, system, electronic device, and storage medium for a graph database, so as to at least solve the problem of low availability when deploying graph database services in related technologies.

[0006] In a first aspect, an embodiment of the present application provides a method for storing and managing a graph database, the method comprising:

[0007] Determine the cluster deployment strategy based on the cluster creation instruction, create corresponding zones according to the cluster deployment strategy, and determine the number of replicas for each zone;

[0008] According to the number of replicas in each zone and the total number of shards configured in the graph database, shard replicas are allocated on the physical machines of each zone in a round-robin manner to obtain a shard allocation plan, wherein the shard allocation plan includes the physical machine location of each shard replica, and corresponding shard replicas are created on the physical machines according to the shard allocation plan;

[0009] In response to the query instructions initiated by the client, the routing policy and the latest shard distribution view information are obtained from the management node. The query engine in each zone accesses the zone's storage engine based on the routing policy and the shard distribution view information. The routing policy is a set of rules that the query engine uses to decide how to distribute query requests to storage engine nodes based on the shard distribution and zone topology.

[0010] In some embodiments, the routing policy includes a local priority policy; and the query engine in each zone accesses the storage engine of the zone based on the routing policy and the shard distribution view information, including:

[0011] Priority is given to accessing the storage engine in the same zone as the query engine. If the access fails, the request is re-initiated according to the priority of each zone.

[0012] In some embodiments, the routing policy includes a leader policy; and the query engine in each zone accesses the storage engine of the zone based on the routing policy and the shard distribution view information, including:

[0013] The location information of the target shard leader copy is obtained through the shard distribution view information, where the location information includes the zone where the shard leader copy is located, and the leader copy is accessed based on the location information.

[0014] In some embodiments, the cluster deployment strategy includes a regional division strategy; when the regions involved in the regional division strategy include a primary zone and a backup zone, creating a corresponding zone according to the cluster deployment strategy includes:

[0015] Create two master zone replicas in the master zone, with the priority of the master zone replica being 2;

[0016] A backup zone copy is created in the backup area. The priority of the backup zone copy is 1, where the priority of 1 is lower than 2.

[0017] In some embodiments, the cluster deployment strategy includes a computer room division strategy; when the computer rooms involved in the computer room division strategy include a first computer room, a second computer room, and a third computer room in the same region, creating corresponding zones according to the cluster deployment strategy includes:

[0018] Create a Zone replica for the first data center. The priority of the Zone replica for the first data center is 2.

[0019] Create a Zone copy for the second data center. The priority of the Zone copy for the second data center is 1.

[0020] A third computer room zone copy is created, and the priority of the third computer room zone copy is 1, where the priority of 1 is lower than 2.

[0021] In some embodiments, the cluster deployment strategy includes a rack partitioning strategy; when the racks involved in the rack partitioning strategy include a first rack, a second rack, and a third rack in the same computer room, creating a corresponding zone according to the cluster deployment strategy includes:

[0022] Create a first rack zone copy, the priority of which is 1;

[0023] Create a second rack zone copy, the priority of which is 1.

[0024] A third rack zone replica is created, and the priority of the third rack zone replica is 1.

[0025] In some embodiments, the method further comprises:

[0026] When the target zone load exceeds the preset load threshold, the new machine is added to the cluster and marked as a member of the target zone;

[0027] If a new machine is detected in the target zone, some shard copies will be automatically migrated to the new machine.

[0028] In some embodiments, the method further comprises:

[0029] When the utilization rate of the target zone is lower than a preset utilization rate threshold, determining a machine to be removed from the target zone;

[0030] Detect the target shard copy on the machine to be removed, and determine whether the machine to be removed can be removed and the removal method based on the number of the target shard copies in the cluster. The removal methods include direct removal and removal after synchronizing data to other machines.

[0031] In a second aspect, an embodiment of the present application provides a storage and management system for a graph database, the system comprising:

[0032] A creation module is used to determine a cluster deployment strategy based on the cluster creation instruction, create corresponding zones according to the cluster deployment strategy, and determine the number of replicas for each zone;

[0033] An allocation module is configured to poll and allocate shard replicas on the physical machines of each zone according to the number of replicas of each zone and the total number of shards configured in the graph database, obtain a shard allocation plan, the shard allocation plan including the physical machine location of each shard replica, and create corresponding shard replicas on the physical machines according to the shard allocation plan;

[0034] The query module is used to respond to query instructions initiated by the client and obtain the routing policy and the latest shard distribution view information from the management node. The query engine in each zone accesses the zone's storage engine based on the routing policy and the shard distribution view information. The routing policy is the set of rules that the query engine uses to decide how to distribute query requests to storage engine nodes based on the shard distribution and zone topology.

[0035] In a third aspect, an embodiment of the present application provides a computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the method for storing and managing a graph database as described in the first aspect above is implemented.

[0036] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, it implements the storage and management method of the graph database as described in the first aspect above.

[0037] Compared with related technologies, the storage and management method of the graph database provided in the embodiment of the present application determines the cluster deployment strategy based on the cluster creation instruction, creates the corresponding zone according to the cluster deployment strategy, and determines the number of replicas of each zone; according to the number of replicas of each zone and the total number of shards configured for the graph database, polls and allocates shard replicas on the physical machines of each zone to obtain a shard allocation plan. The shard allocation plan includes the physical machine location of each shard replica. According to the shard allocation plan, the corresponding shard replica is created on the physical machine; in response to the query instruction initiated by the client, the routing strategy and the latest shard distribution view information are obtained from the management node. The query engine in each zone accesses the storage engine of the zone based on the routing strategy and the shard distribution view information. The routing strategy is a set of rules that the query engine uses to decide how to distribute query requests to the storage engine nodes based on the shard distribution and zone topology, which solves the problem of low availability when deploying graph database services. The graph database service is deployed based on the zone, and the granularity of the zone can be customized to achieve high availability across racks / computer rooms and regions. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0039] Figure 1 is a flowchart of a method for storing and managing a graph database according to an embodiment of the present application;

[0040] Figure 2This is a topology diagram of a Zone-based graph database storage and management system according to an embodiment of the present application;

[0041] Figure 3 is a schematic diagram of a shard allocation solution according to an embodiment of the present application;

[0042] Figure 4 This is a structural block diagram of a storage and management system for a graph database according to an embodiment of the present application;

[0043] Figure 5 Schematic diagram of the internal structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is described and illustrated below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely used to explain this application and are not intended to limit this application. Based on the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without making any creative efforts are within the scope of protection of this application.

[0045] Obviously, the drawings described below are merely examples or embodiments of the present application. Those skilled in the art can, without inventive effort, apply the present application to other similar scenarios based on these drawings. Furthermore, it is also understood that, although the effort involved in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, changes in design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as an insufficiency of the content disclosed in this application.

[0046] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments unless there is a conflict.

[0047] Unless otherwise defined, technical or scientific terms used herein shall have the ordinary meaning as understood by persons of ordinary skill in the art to which this application belongs. The terms "a," "an," "an," "the," and similar expressions used herein do not denote quantitative limitations and may refer to either the singular or the plural. The terms "comprise," "include," "have," and any variations thereof, used herein, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or modules (units) is not limited to the listed steps or units but may also include steps or units not listed, or may include other steps or units inherent to the process, method, product, or apparatus. The terms "connected," "connected," "coupled," and similar expressions used herein are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. As used herein, "plurality" means two or more. "And / or" describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" may mean: A exists alone; A and B exist simultaneously; or B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0048] This embodiment provides a method for storing and managing a graph database. Figure 1 is a flowchart of a method for storing and managing a graph database according to an embodiment of the present application. Figure 1 As shown, the process includes the following steps:

[0049] Step S101: Determine a cluster deployment strategy based on a cluster creation instruction, create corresponding zones according to the cluster deployment strategy, and determine the number of replicas for each zone.

[0050] In a distributed system, a zone is a logical partition, which can be a computer room, data center, or geographical location.

[0051] During cluster creation, you need to prepare machine resources and add them using the ngctl tool provided by NebulaGraph. Before adding them, you also need to confirm the cluster deployment method, such as whether you want to deploy only locally or deploy in multiple locations.

[0052] First create the corresponding Zone information, including the corresponding priority and number of copies.

[0053] ngctl svcgrp add-zone <cluster><zone_name> -p <priority> -r <replica-factor>

[0054] When adding a host or service, you also need to mark the zone information where it is located.

[0055] ngctl svcgrp add-host <cluster> -H <host>-z <zone_name> / ngctlsvcgrp add-service <cluster> -H <host>-z<zone_name>

[0056] After resources are created, the system records them in multiple sub-tables. The service group table records the number of replicas and routing policies; the zone table records all zones created within the cluster, including the corresponding number of replicas and priority; and the host table records the cluster and zone.

[0057] Figure 2 This is a topology diagram of a Zone-based graph database storage and management system according to an embodiment of the present application, such as Figure 2 As shown in the figure, Metad refers to the Meta Service, which is responsible for managing the system's metadata; GraphD is a specific software tool mainly used for the management and operation of graph databases.

[0058] A zone represents a rack, a computer room, or an area, such as Shanghai Zone 1 or Nanjing Zone 1. A zone can contain one or more datasets (replicas).

[0059] Replica, represents a complete data set, usually several copies correspond to several Replicas. Figure 2 For example, each shard has 3 replicas, of which the ones marked in red are Leader replicas. For example, in dataset 1, Shard 1 and Shard 3 replicas are Leaders.

[0060] A shard (Part) represents a data shard, and there is no data overlap between parts.

[0061] Peer is a replica set of data shards, distributed in different Replicas.

[0062] Host, which represents a physical machine, is a resource in a zone and is shared by all datasets in the zone.

[0063] It should be noted that users can customize the size of zones, the division of master and slave zones, and resource allocation through cluster creation instructions. Fine-grained data distribution based on zones optimizes distributed graph databases and improves their high availability.

[0064] In some embodiments, the cluster deployment strategy includes a regional division strategy. When the regions involved in the regional division strategy include a primary zone and a backup zone, creating corresponding zones according to the cluster deployment strategy includes:

[0065] Create two master zone replicas in the master zone, with the priority of the master zone replica being 2;

[0066] Create a backup zone copy in the backup area. The priority of the backup zone copy is 1, where priority 1 is lower than priority 2.

[0067] For example, if the primary zone is in Shanghai and Nanjing is the backup zone, create a Zone Shanghai 2 replica with priority 2 and a Zone Nanjing 1 replica with priority 1.

[0068] Zones are divided based on regions. Natural disasters (such as earthquakes and power outages) in a single region do not affect data availability in other zones. User requests are preferentially routed to the query engine (GraphD) and storage engine (StorageD) in the same region, reducing cross-city network latency and making it suitable for cross-city, provincial, and national business.

[0069] In some embodiments, the cluster deployment strategy includes a computer room division strategy. When the computer rooms involved in the computer room division strategy include a first computer room, a second computer room, and a third computer room in the same region, creating corresponding zones according to the cluster deployment strategy includes:

[0070] Create a Zone copy for the first computer room. The priority of the Zone copy for the first computer room is 2.

[0071] Create a Zone copy for the second computer room. The priority of the Zone copy for the second computer room is 1.

[0072] Create a Zone copy for the third computer room. The priority of the Zone copy for the third computer room is 1, where the priority of 1 is lower than that of 2.

[0073] Within a region (e.g., a city), different data centers are divided into independent zones. A single data center failure (such as air conditioning failure or network outage) does not affect services in other data centers within the region. Replicas are distributed across three data centers, meeting requirements similar to the "three locations, five centers" model in the financial industry. Capacity expansion of a single data center is possible without impacting other zones, making it suitable for a multi-active architecture within a city.

[0074] In some embodiments, the cluster deployment strategy includes a rack partitioning strategy; when the racks involved in the rack partitioning strategy include a first rack, a second rack, and a third rack in the same computer room, creating a corresponding zone according to the cluster deployment strategy includes:

[0075] Create a first-rack zone replica. The priority of the first-rack zone replica is 1.

[0076] Create a second rack zone copy with a priority of 1.

[0077] Create a third-rack zone replica with a priority of 1.

[0078] Within the same data center, different racks (or power supply units) are divided into sub-zones to prevent data unavailability caused by single-rack power outages or network switch failures. Inter-rack latency is low, making it suitable for high-frequency queries and latency-sensitive scenarios.

[0079] It should be noted that, depending on the actual scenario, regional division strategies, computer room division strategies, and rack division strategies can be used in combination, such as combining three computer rooms across regions and within the same city.

[0080] Continue to refer to Figure 1 After creating the Zone, execute step S102.

[0081] Step S102: Based on the number of replicas in each zone and the total number of shards configured in the graph database, shard replicas are allocated on the physical machines of each zone in a round-robin manner to obtain a shard allocation plan. The shard allocation plan includes the physical machine location of each shard replica. According to the shard allocation plan, corresponding shard replicas are created on the physical machine.

[0082] In graph databases, a shard is the smallest unit of horizontal data splitting for distributed storage. A complete copy of the data must be maintained in each zone. Each shard has multiple replicas to achieve high availability and load balancing. High availability: Replicas are distributed across different nodes / zones, with automatic failover in the event of a single point of failure. Load balancing: Read requests are distributed across multiple replicas to avoid overloading a single zone or host.

[0083] The number of replicas specified when creating a zone determines the data placement strategy. For example, the master zone can have multiple shard replicas to ensure local majority, while slave zones can have a single replica to ensure disaster recovery and data recovery.

[0084] In this embodiment, users can customize data placement strategies to make data distribution more reasonable and meet isolation requirements in different scenarios.

[0085] Figure 3 Schematic diagram of a shard allocation solution according to an embodiment of the present application. Figure 3 As shown in the figure, the cluster has two zones. Zone1 has a higher priority than Zone2. Zone1 has two shard copies, while Zone2 has a single shard copy, ensuring that each zone has complete copy data. The yellow one is the Leader copy.

[0086] Step S103, responding to the query instruction initiated by the client, obtaining the routing policy and the latest shard distribution view information from the management node, and the query engine in each zone accesses the zone's storage engine based on the routing policy and shard distribution view information. The routing policy is a set of rules that the query engine uses to decide how to distribute query requests to storage engine nodes based on the shard distribution and zone topology.

[0087] In a zone-based graph database cluster, each zone includes a query engine (GraphD) and a storage engine (StorageD) for localized computing and storage. A management node (MetaD) serves as the global coordinator and can be deployed across zones (typically, each zone has at least one MetaD replica).

[0088] GraphD needs to know which StorageD nodes the data is distributed on, so it periodically pulls the latest shard distribution view from MetaD. GraphD periodically sends heartbeat requests to MetaD (for example, every 10 seconds) to obtain the latest shard distribution view. GraphD maintains this view in memory for subsequent query routing.

[0089] When a client initiates a query, GraphD determines the request path based on the routing policy in MetaD.

[0090] In some embodiments, the routing policy includes a local priority policy; the query engine in each zone accesses the storage engine of the zone based on the routing policy and the shard distribution view information, including:

[0091] The local priority strategy (Zone First) prioritizes access to the storage engine in the same zone as the query engine. If the access fails, the request is re-initiated based on the priority of each zone.

[0092] GraphD prioritizes accessing StorageD in the same zone. If the StorageD has the required shard copy, it directly requests the local copy to reduce cross-region network latency and improve query speed.

[0093] If the local StorageD does not have the required shard copy, that is, the access fails, then the shards are accessed one by one according to the Zone priority from high to low until the required shard copy is found.

[0094] In some embodiments, the routing policy includes a leader policy; the query engine in each zone accesses the storage engine of the zone based on the routing policy and the shard distribution view information, including:

[0095] The location information of the target shard leader copy is obtained through the shard distribution view information. The location information includes the zone where the shard leader copy is located. The leader copy is accessed based on the location information.

[0096] In a distributed system, a leader is a node responsible for coordinating and managing specific tasks. The leader strategy (LeaderFirst) prioritizes access to the zone where the leader replica resides. A leader can be located in any zone, and GraphD obtains its location through MetaD's view.

[0097] Choose Zone First or Leader First based on the actual application environment or user instructions.

[0098] Through the above steps, the cluster deployment strategy is determined based on the cluster creation instructions, the corresponding zones are created according to the cluster deployment strategy, and the number of replicas for each zone is determined. Based on the number of replicas in each zone and the total number of shards configured for the graph database, shard replicas are allocated on the physical machines in each zone in a round-robin manner to obtain a shard allocation plan. The shard allocation plan includes the physical machine location of each shard replica. According to the shard allocation plan, the corresponding shard replicas are created on the physical machines. In response to query instructions initiated by the client, the routing policy and the latest shard distribution view information are obtained from the management node. The query engine in each zone accesses the zone's storage engine based on the routing policy and shard distribution view information. The routing policy is the set of rules that the query engine uses to determine how to distribute query requests to the storage engine nodes based on the shard distribution and zone topology. This solves the problem of low availability when deploying graph database services. Graph database services are deployed based on zones, and the granularity of zones can be customized to achieve high availability across racks / computer rooms and regions.

[0099] In some embodiments, the method further comprises:

[0100] When the target zone load exceeds the preset load threshold, the new machine is added to the cluster and marked as a member of the target zone;

[0101] When a new machine is detected in the target zone, some shard copies are automatically migrated to the new machine.

[0102] For example, if Zone-A is overloaded, a new machine, HostX, needs to be added to Zone-A. HostX is added to the cluster and marked as a Zone-A member. This triggers a balancing log, and the system detects the new machine in Zone-A and automatically migrates some shard replicas to HostX. It's important to note that unused machines receive data first to ensure fast load balancing.

[0103] During the migration process, ensure that the number of replicas of each shard in Zone-A remains unchanged (for example, if Zone-A originally had two replicas, it will still have two replicas after expansion, but the distribution will change from [Host1, Host2] to [Host1, HostX]).

[0104] In some embodiments, the method further comprises:

[0105] When the utilization rate of the target zone is lower than the preset utilization rate threshold, the machine to be removed from the target zone is determined;

[0106] Detect the target shard copies on the machine to be removed. Based on the number of target shard copies in the cluster, determine whether the machine to be removed can be removed and how to remove it. Removal methods include direct removal and removal after synchronizing data to other machines.

[0107] For example, if Zone-B has low utilization and HostY (the machine to be removed) needs to be taken offline, check the shard replicas on HostY to ensure that other zones have sufficient replicas (to avoid data loss). If HostY is the only replica of a shard, do not remove it.

[0108] If you decide to remove HostY, migrate the replica on HostY to other machines in the same zone. After confirming that HostY has no data, delete it from the cluster.

[0109] In this embodiment, the number of replicas in each zone is strictly limited, that is, the number of replicas in each zone is fixed. Global balancing can be decomposed into internal balancing in each zone, reusing existing logic without developing new zone-aware algorithms.

[0110] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0111] This embodiment also provides a storage and management system for a graph database, which is used to implement the above-mentioned embodiments and preferred implementation methods. The details that have been explained will not be repeated here. As used below, the terms "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that implements the predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

[0112] Figure 4 This is a structural block diagram of a storage and management system for a graph database according to an embodiment of the present application. Figure 4 As shown, the system includes:

[0113] The creating module 41 is configured to determine a cluster deployment strategy based on a cluster creation instruction, create corresponding Zones according to the cluster deployment strategy, and determine the number of replicas of each Zone.

[0114] The allocating module 42 is configured to allocate shard replicas on the physical machines of each Zone according to the number of replicas of each Zone and the total number of shards configured by the graph database, to obtain a shard allocation scheme, the shard allocation scheme including the physical machine locations of each shard replica, and to create corresponding shard replicas on the physical machines according to the shard allocation scheme.

[0115] The querying module 43 is configured to respond to a query instruction initiated by a client, to obtain a routing strategy and the latest shard distribution view information from a management node, and to cause a query engine in each Zone to access a storage engine of the Zone based on the routing strategy and the shard distribution view information, the routing strategy being a rule set for the query engine to determine how to distribute a query request to the storage engine node according to shard distribution and Zone topology.

[0116] In some embodiments, the routing strategy includes a local priority strategy, and the querying module 43 includes:

[0117] The local priority module is configured to preferentially access a storage engine in the same Zone as the query engine, and to reinitiate a request according to the priority of each Zone if the access fails.

[0118] In some embodiments, the routing strategy includes a Leader strategy, and the querying module 43 includes:

[0119] The Leader querying module is configured to obtain location information of a target shard Leader replica through the shard distribution view information, the location information including a Zone where the shard Leader replica is located, and to access the Leader replica based on the location information.

[0120] In some embodiments, the cluster deployment strategy includes a region division strategy, and in the case that the region division strategy involves a primary region area and a backup region area, the creating module 41 includes:

[0121] The primary region creating module is configured to create two primary region Zone replicas in the primary region area, the priority of the primary region Zone replicas being 2.

[0122] The backup region creating module is configured to create one backup region Zone replica in the backup region area, the priority of the backup region Zone replica being 1, wherein the priority of 1 is lower than the priority of 2.

[0123] In some embodiments, the cluster deployment strategy includes a computer room division strategy. When the computer rooms involved in the computer room division strategy include a first computer room, a second computer room, and a third computer room in the same region, the creation module 41 includes:

[0124] The first computer room creation module is used to create a first computer room Zone copy. The priority of the first computer room Zone copy is 2.

[0125] The second computer room creation module is used to create a second computer room Zone copy. The priority of the second computer room Zone copy is 1.

[0126] The third computer room creation module is used to create a third computer room Zone copy. The priority of the third computer room Zone copy is 1, where the priority of 1 is lower than 2.

[0127] In some embodiments, the cluster deployment strategy includes a rack partitioning strategy, and in the case where the racks involved in the rack partitioning strategy include a first rack, a second rack, and a third rack in the same computer room; the creation module 41 includes:

[0128] The first rack creation module is used to create a first rack zone copy, and the priority of the first rack zone copy is 1.

[0129] The second rack creation module is used to create a second rack zone copy. The priority of the second rack zone copy is 1.

[0130] The third rack creation module is used to create a third rack zone copy. The priority of the third rack zone copy is 1.

[0131] In some embodiments, the system further comprises:

[0132] The machine addition module is used to add new machines to the cluster and mark them as members of the target zone when the load of the target zone is higher than the preset load threshold; when a new machine is detected in the target zone, some shard copies are automatically migrated to the new machine.

[0133] In some embodiments, the method further comprises:

[0134] The machine reduction module is used to determine the machines to be removed from the target zone when the utilization of the target zone is lower than the preset utilization threshold; detect the target shard copies on the machines to be removed, and determine whether the machines to be removed can be removed and the removal method based on the number of target shard copies in the cluster. The removal methods include direct removal and removal after synchronizing data to other machines.

[0135] Through the above system, the creation module 41 determines the cluster deployment strategy based on the cluster creation instruction, creates the corresponding zone according to the cluster deployment strategy, and determines the number of replicas for each zone. The allocation module 42 polls and allocates shard replicas on the physical machines of each zone based on the number of replicas of each zone and the total number of shards configured for the graph database, and obtains a shard allocation plan. The shard allocation plan includes the physical machine location of each shard replica. According to the shard allocation plan, the corresponding shard replica is created on the physical machine. The query module 43 responds to the query instruction initiated by the client and obtains the routing strategy and the latest shard distribution view information from the management node. The query engine in each zone accesses the storage engine of the zone based on the routing strategy and the shard distribution view information. The routing strategy is a set of rules that the query engine uses to decide how to distribute query requests to the storage engine nodes based on the shard distribution and zone topology, solving the problem of low availability when deploying graph database services. The graph database service is deployed based on the zone, and the granularity of the zone can be customized to achieve high availability across racks / computer rooms and regions.

[0136] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0137] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0138] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0139] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0140] S1: Determine the cluster deployment strategy based on the cluster creation instruction, create the corresponding zone according to the cluster deployment strategy, and determine the number of replicas for each zone.

[0141] S2, based on the number of replicas in each zone and the total number of shards configured in the graph database, polls and allocates shard replicas on the physical machines in each zone to obtain a shard allocation plan. The shard allocation plan includes the physical machine location of each shard replica. According to the shard allocation plan, the corresponding shard replica is created on the physical machine.

[0142] S3 responds to query instructions initiated by the client and obtains the routing policy and the latest shard distribution view information from the management node. The query engine in each zone accesses the zone's storage engine based on the routing policy and shard distribution view information. The routing policy is a set of rules that the query engine uses to decide how to distribute query requests to storage engine nodes based on the shard distribution and zone topology.

[0143] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be repeated here.

[0144] In one embodiment, Figure 5 is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application, such as Figure 5 As shown, an electronic device is provided, which may be a server, and its internal structure diagram may be as shown in FIG. Figure 5 As shown. The electronic device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the electronic device is used to store data. The network interface of the electronic device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for storing and managing a graph database is implemented.

[0145] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0146] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0147] Those skilled in the art should understand that the various technical features of the above-described embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0148] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.< / host> < / cluster> < / host> < / cluster> < / priority> < / cluster>

Claims

1. A method for storing and managing a graph database, characterized in that: The method comprises: Determine the cluster deployment strategy based on the cluster creation instruction, create corresponding zones according to the cluster deployment strategy, and determine the number of replicas for each zone; According to the number of replicas in each zone and the total number of shards configured in the graph database, shard replicas are allocated on the physical machines of each zone in a round-robin manner to obtain a shard allocation plan, wherein the shard allocation plan includes the physical machine location of each shard replica, and corresponding shard replicas are created on the physical machines according to the shard allocation plan; In response to the query instructions initiated by the client, the routing policy and the latest shard distribution view information are obtained from the management node. The query engine in each zone accesses the zone's storage engine based on the routing policy and the shard distribution view information. The routing policy is a set of rules that the query engine uses to decide how to distribute query requests to storage engine nodes based on the shard distribution and zone topology.

2. The method according to claim 1, characterized in that The routing policy includes a local priority policy; the query engine in each zone accesses the storage engine of the zone based on the routing policy and the shard distribution view information, including: Priority is given to accessing the storage engine in the same zone as the query engine. If the access fails, the request is re-initiated according to the priority of each zone.

3. The method according to claim 1, characterized in that The routing strategy includes a leader strategy; the query engine in each zone accesses the storage engine of the zone based on the routing strategy and the shard distribution view information, including: The location information of the target shard leader copy is obtained through the shard distribution view information, where the location information includes the zone where the shard leader copy is located, and the leader copy is accessed based on the location information.

4. The method according to claim 1, wherein The cluster deployment strategy includes a regional division strategy; In the case where the regions involved in the regional division strategy include a primary zone and a backup zone, creating a corresponding zone according to the cluster deployment strategy includes: Create two master zone replicas in the master zone, with the priority of the master zone replica being 2; A backup zone copy is created in the backup area. The priority of the backup zone copy is 1, where the priority of 1 is lower than 2.

5. The method according to claim 1, wherein The cluster deployment strategy includes a computer room division strategy; when the computer rooms involved in the computer room division strategy include a first computer room, a second computer room, and a third computer room in the same region, creating a corresponding zone according to the cluster deployment strategy includes: Create a Zone replica for the first data center. The priority of the Zone replica for the first data center is 2. Create a Zone copy for the second data center. The priority of the Zone copy for the second data center is 1. A third computer room zone copy is created, and the priority of the third computer room zone copy is 1, where the priority of 1 is lower than 2.

6. The method according to claim 1, characterized in that The cluster deployment strategy includes a rack partitioning strategy; In a case where the racks involved in the rack partitioning strategy include a first rack, a second rack, and a third rack in the same computer room, creating a corresponding zone according to the cluster deployment strategy includes: Create a first rack zone copy, the priority of which is 1; Create a second rack zone copy, the priority of which is 1. A third rack zone replica is created, and the priority of the third rack zone replica is 1.

7. The method according to claim 1, characterized in that The method further comprises: When the target zone load exceeds the preset load threshold, the new machine is added to the cluster and marked as a member of the target zone; If a new machine is detected in the target zone, some shard copies will be automatically migrated to the new machine.

8. The method according to claim 1, characterized in that The method further comprises: When the utilization rate of the target zone is lower than a preset utilization rate threshold, determining a machine to be removed from the target zone; Detect the target shard copy on the machine to be removed, and determine whether the machine to be removed can be removed and the removal method based on the number of the target shard copies in the cluster. The removal methods include direct removal and removal after synchronizing data to other machines.

9. A storage and management system for a graph database, characterized in that: The system comprises: A creation module is used to determine a cluster deployment strategy based on the cluster creation instruction, create corresponding zones according to the cluster deployment strategy, and determine the number of replicas for each zone; An allocation module is configured to poll and allocate shard replicas on the physical machines of each zone according to the number of replicas of each zone and the total number of shards configured in the graph database, obtain a shard allocation plan, the shard allocation plan including the physical machine location of each shard replica, and create corresponding shard replicas on the physical machines according to the shard allocation plan; The query module is used to respond to query instructions initiated by the client and obtain the routing policy and the latest shard distribution view information from the management node. The query engine in each zone accesses the zone's storage engine based on the routing policy and the shard distribution view information. The routing policy is the set of rules that the query engine uses to decide how to distribute query requests to storage engine nodes based on the shard distribution and zone topology.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the storage and management method of the graph database according to any one of claims 1 to 8 is implemented.

Images