Data processing method and device for distributed graph database

By decoupling the data backup and recovery process according to topological structure information in the distributed graph database, optimizing the selection of target storage nodes, the problems of low data backup and recovery efficiency and poor flexibility in the existing technology are solved, and efficient and flexible data are achieved The solution reduces cross-region network overhead.

CN118377835BActive Publication Date: 2024-11-12ALIPAY (HANGZHOU) INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410814594.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-21
Publication Date
2024-11-12
Estimated Expiration
2044-06-21

AI Technical Summary

Technical Problem

The existing distributed graph database data processing method is inefficient and not flexible enough in the data backup and recovery process, making it difficult to meet the needs of changes in multi-cluster topology, resulting in large overhead across regions, and the backup and recovery process is coupled, affecting the reliability of the system. sex.

Method used

Data backup and data backup are realized by determining the graph data to be backed up and exporting it from the target storage node to the intermediate storage device according to the topological structure information of the first cluster, and then storing it to the storage node of the other cluster according to the topological structure information of the second cluster. The decoupling of recovery, independent of the topology of the original cluster and the target cluster, optimizes the target storage node selection to reduce cross-regional communication overhead.

Benefits of technology

It realizes a more flexible, practical and efficient data backup and recovery process, reduces cross-region network overhead, and improves the reliability and flexibility of the system, adapts to changes in multi-cluster topology.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118377835B_ABST
    Figure CN118377835B_ABST
Patent Text Reader

Abstract

The embodiments of the present specification provide a data processing method and device for a distributed graph database. In the data processing method for a distributed graph database, the graph data of the distributed graph database is stored in the storage nodes of the first cluster in the form of shards, and the graph data to be backed up is determined; at least one target storage node is determined according to the correspondence between the graph data shards and the storage nodes of the first cluster where they are located; several graph data shards in the target storage node are exported to an intermediate storage device; and the graph data exported to the intermediate storage device is then stored in at least one storage node of the second cluster according to the correspondence between the graph data shards and the storage nodes of the second cluster where they will be stored.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of this specification generally relate to the field of computer technology, and more particularly to a data processing method and apparatus for a distributed graph database. Background Art

[0002] With the rapid development of Internet technology and the continuous increase in data scale, the application of distributed graph databases is becoming more and more extensive. Various data processing methods for distributed graph databases (such as data backup and recovery) are of great significance to ensuring the security and reliability of data services. Summary of the invention

[0003] In view of the above, the embodiments of this specification provide a data processing method and device for a distributed graph database. By using this method and device, more flexible, practical and efficient data backup can be achieved.

[0004] According to one aspect of an embodiment of the present specification, a data processing method for a distributed graph database is provided, wherein graph data of the distributed graph database is stored in the form of shards in a storage node of a first cluster, and the data processing method comprises: determining graph data to be backed up, the graph data to be backed up comprises a number of graph data shards; determining at least one target storage node where the number of graph data shards are located from the storage nodes of the first cluster according to first graph topology information, the first graph topology information comprising: a correspondence between graph data shards and the storage nodes of the first cluster where they are located; exporting the number of graph data shards in the target storage node to an intermediate storage device; and storing the graph data exported to the intermediate storage device in at least one storage node of a second cluster according to second graph topology information, the second graph topology information comprising: a correspondence between the graph data shards and the storage nodes of the second cluster where they will be stored.

[0005] According to another aspect of an embodiment of the present specification, there is provided a data processing apparatus for a distributed graph database, wherein graph data of the distributed graph database is stored in the form of shards in a storage node of a first cluster, and the data processing apparatus comprises: a node determination unit, configured to determine graph data to be backed up, wherein the graph data to be backed up comprises a plurality of graph data shards; determining at least one target storage node where the plurality of graph data shards are located from the storage nodes of the first cluster according to first graph topology information, wherein the first graph topology information comprises: a correspondence between graph data shards and the storage nodes of the first cluster where they are located; a data export unit, configured to export the plurality of graph data shards in the target storage node to an intermediate storage device; and a data import unit, configured to store the graph data exported to the intermediate storage device in at least one storage node of a second cluster according to second graph topology information, wherein the second graph topology information comprises: a correspondence between the graph data shards and the storage nodes of the second cluster where they will be stored.

[0006] According to another aspect of an embodiment of the present specification, there is provided a data processing device for a distributed graph database, comprising: at least one processor, and a memory coupled to the at least one processor, the memory storing instructions, which, when executed by the at least one processor, enables the at least one processor to execute the data processing method for a distributed graph database as described above.

[0007] According to another aspect of an embodiment of the present specification, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the data processing method for a distributed graph database as described above is implemented.

[0008] According to another aspect of an embodiment of the present specification, a computer program product is provided, including a computer program, wherein the computer program is executed by a processor to implement the data processing method for a distributed graph database as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] A further understanding of the nature and advantages of the content of this specification may be achieved by referring to the following drawings. In the drawings, similar components or features may have the same reference numerals.

[0010] Figure 1 An exemplary architecture of a data processing method and apparatus for a distributed graph database according to an embodiment of the present specification is shown.

[0011] Figure 2 A flowchart showing an example of a data processing method for a distributed graph database according to an embodiment of the present specification.

[0012] Figure 3 A flowchart showing an example of a process of determining a target node group according to an embodiment of the present specification.

[0013] Figure 4 A schematic diagram showing another example of a data processing method for a distributed graph database according to an embodiment of the present specification.

[0014] Figure 5 A block diagram showing an example of a data processing device for a distributed graph database according to an embodiment of the present specification.

[0015] Figure 6 A schematic diagram showing an example of a data processing device for a distributed graph database according to an embodiment of the present specification. DETAILED DESCRIPTION

[0016] The subject matter described herein will be discussed below with reference to example implementations. It should be understood that the discussion of these implementations is only to enable those skilled in the art to better understand and implement the subject matter described herein, and is not a limitation of the scope of protection, applicability or examples set forth in the claims. The functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the embodiments of this specification. Various processes or components may be omitted, substituted or added to each example as needed. In addition, the features described relative to some examples may also be combined in other examples.

[0017] As used herein, the term "including" and its variations represent open terms, meaning "including but not limited to". The term "based on" means "based at least in part on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other definitions may be included below, whether explicit or implicit. Unless the context clearly indicates otherwise, the definition of a term is consistent throughout the specification.

[0018] The data processing method and device for a distributed graph database according to the embodiments of this specification will be described in detail below in conjunction with the accompanying drawings.

[0019] Figure 1 An exemplary architecture 100 of a data processing method and apparatus for a distributed graph database according to an embodiment of the present specification is shown.

[0020] like Figure 1As shown, the architecture 100 of the distributed graph database may include a platform layer 110, an engine layer 120, and a data storage layer 130. The platform layer 110 can be used to interact with external devices of the distributed graph database. For example, the platform layer 110 can receive various requests initiated by users for the distributed graph database, such as read and write requests, query requests, backup requests, etc. for graph data. The engine layer 120 can be used to save and maintain metadata of the distributed graph database. In some examples, the engine layer 120 can also be used to perform various analytical calculations, such as querying or updating operations based on the data stored in the storage layer 130.

[0021] The storage layer 130 may include multiple storage nodes, such as nodes 131 and 132. In some examples, the graph data stored in the distributed graph database may have multiple copies, and each copy may refer to a complete graph data. In some examples, the copy may be sharded, and at least part of the shards obtained may be stored by multiple storage nodes respectively, so that the graph data shards stored by the multiple storage nodes can be combined into a complete copy.

[0022] It should be understood that Figure 1 All network entities shown in the figure are exemplary. Any other network entities may be involved in the architecture 100 according to specific application requirements.

[0023] Figure 2 A flowchart of a data processing method 200 for a distributed graph database according to an embodiment of the present specification is shown.

[0024] like Figure 2 As shown, at 210, the graph data to be backed up is determined.

[0025] In this embodiment, the graph data to be backed up may include several graph data shards. In this embodiment, the graph data of the distributed graph database may be stored in the storage nodes of the first cluster in the form of shards. Several graph data shards may be determined from the graph data shards stored in the storage nodes of the first cluster to form the graph data to be backed up.

[0026] In some examples, the data content corresponding to each shard can be obtained based on a hash operation on a data identifier (e.g., VID) corresponding to each piece of vertex data and edge data. In some examples, for each copy of graph data, a hash algorithm can be used to convert the data identifier corresponding to each piece of vertex data and edge data of the copy into a hash value, and then the data is divided into different shards according to the obtained hash value, thereby obtaining the data content contained in each shard.

[0027] Figure 3A schematic diagram shows an example 300 in which graph data of a distributed graph database according to an embodiment of the present specification is stored in storage nodes of a first cluster in the form of shards.

[0028] like Figure 3 As shown, the graph data of the distributed graph database may include master copy graph data 310 and slave copy graph data 320, 330. In some examples, the master copy graph data 310 may include several master copy graph data shards (e.g., 311~314), the slave copy graph data 320 may include several slave copy graph data shards (e.g., 321~324), and the slave copy graph data 330 may include several slave copy graph data shards (e.g., 331~334). In some examples, the graph data to be backed up may be composed of master copy graph data shards (e.g., 311~314). In some examples, the graph data to be backed up may be composed of slave copy graph data shards (e.g., 321~324 or 331~334 or 321~323 and 334, etc.). In some examples, the graph data to be backed up may be composed of a portion of primary replica graph data shards and a portion of secondary replica graph data shards (eg, composed of 311 and 322-324 or composed of 311-312, 323, 334, etc.).

[0029] Back to Figure 2 At 220, at least one target storage node where the graph data shards are located is determined from the storage nodes of the first cluster according to the first graph topology information.

[0030] In this embodiment, the first graph topology structure information may include the correspondence between the graph data shards and the storage nodes of the first cluster to which they are located. In some examples, the first graph topology structure information may be "storage node 1 - 1st shard and 3rd shard, storage node 2 - 2nd shard and 4th shard". Among them, storage node 1 and storage node 2 may be storage nodes in the first cluster. In some examples, the above-mentioned first graph topology structure information may be used as one of the metadata of the distributed graph database, which is saved and maintained by the engine layer. In some examples, the graph data shards may be distinguished according to the shard identifier, the data content contained therein, etc. In some examples, the graph data shard metadata may also be used as one of the metadata of the distributed graph database, which is saved and maintained by the engine layer. In some examples, the graph data shard metadata may, for example, include the name, identifier, data size, metadata of the data blocks contained therein, the replica to which the shard belongs, etc.

[0031] In some examples, when the graph data includes primary and secondary graph data, the first graph topology information may be used to indicate the correspondence between the graph data shards of the primary and secondary graph data and the storage nodes of the first cluster where the graph data shards are located. Figure 3As shown, the first graph topology information can be used to indicate the following corresponding relationship: storage node 1 in the first cluster (such as Figure 3 341) and the shard 1 of the primary replica graph data 310 (as shown in Figure 3 311) and shard 2 (as shown in Figure 3 312 in the first cluster), the storage node 2 in the first cluster (as shown in FIG. Figure 3 342) and the shard 3 of the master copy graph data 310 (as shown in Figure 3 313) and shard 4 (as shown in Figure 3 314 in the first cluster), the storage node 3 in the first cluster (as shown in FIG. Figure 3 343) and from the replica graph data 320, shard 1 (as shown in Figure 3 321) and shard 4 (as shown in Figure 3 324 in the first cluster), the storage node 4 in the first cluster (as shown in Figure 3 344) and from the replica graph data 320, shard 2 (as Figure 3 322 in the first cluster), the storage node 5 in the first cluster (as shown in Figure 3 345) and from the replica graph data 320 shard 3 (as Figure 3 323 in the first cluster), the storage node 6 in the first cluster (as shown in Figure 3 346) and from the replica graph data 330, shard 1 (as Figure 3 331) and shard 3 (as shown in Figure 3 333 in the first cluster), the storage node 7 in the first cluster (as shown in Figure 3 347) and from the replica graph data 330, shard 2 (as Figure 3 332) and shard 4 (as shown in Figure 3 The correspondence between them is shown in 334).

[0032] In some examples, the above-mentioned several graph data shards may be slave replica graph data shards, so that the target storage node may be a storage node storing the above-mentioned slave replica graph data shards. In some examples, the target storage node may be storage node 6 and storage node 7 in the first cluster of the above example.

[0033] In some examples, at least one target storage node where several graph data shards are located may also be determined from the storage nodes of the first cluster based on the location information of the storage nodes of the first cluster and the location information of the intermediate storage device.

[0034] In these examples, the location information can be used to indicate the location where the physical machine or intermediate storage device corresponding to the storage node is placed. In some examples, based on the location information, for multiple copies of graph data shards, the storage node closest to the intermediate storage device among the storage nodes where the graph data shards are located can be used as the target storage node.

[0035] In some examples, at least one target storage node where several graph data shards are located may also be determined from the storage nodes of the first cluster according to the load conditions of the storage nodes of the first cluster.

[0036] In these examples, the load condition of the storage node may include but is not limited to at least one of the following: read load condition, write load condition. In some examples, the node load information used to indicate the above load condition can be used as one of the metadata of the distributed graph database, and is saved and maintained by the engine layer. In some examples, for multiple copies of graph data shards, the storage node with a lighter read load among the storage nodes where the graph data shards are located can be determined as the target storage node.

[0037] In some examples, at least one target storage node where the graph data shards are located may also be determined from the storage nodes of the first cluster based on the location information of the storage nodes of the first cluster, the location information of the intermediate storage device, and the load of the storage nodes. In these examples, at least one target storage node may be determined by comprehensively considering the distance between the storage node and the intermediate storage device and the load of the storage node.

[0038] In one example, each storage node in the first cluster only stores the graph data shards of the specified replica, and does not store graph data shards across replicas. The load level of each slave replica graph data can be determined based on the first graph topology information and the load conditions of the corresponding storage nodes. For example, for each slave replica graph data, the load conditions of the storage nodes are comprehensively calculated in units of replicas according to the load conditions of the storage nodes where the graph data shards of the slave replica graph data are located, and the load levels of each slave replica graph data are obtained. Then, the target slave replica graph data is determined based on the determined load level. For example, the slave replica graph data with the lightest load level can be determined as the target slave replica graph data. For another example, the target slave replica graph data can be determined randomly or in combination with other factors from several slave replica graph data with lighter load levels. If the various graph data shards of the determined target slave replica graph data all meet the backup conditions, the various graph data shards of the target slave replica graph data can be determined as several graph data shards included in the graph data to be backed up. If the determined target contains graph data shards that do not meet the backup conditions from the graph data shards of the replica graph data, a replacement graph data shard can be selected from other replicas, so that the target is determined to be several graph data shards included in the graph data to be backed up from the graph data shards of the replica graph data that meet the backup conditions and the replacement graph data shards. In some examples, the graph data shards that do not meet the backup conditions may be, for example, graph data shards stored on a storage node whose load exceeds a threshold. It can be understood that the data content contained in the replacement graph data shards at least covers the data content contained in the above-mentioned graph data shards that do not meet the backup conditions.

[0039] Through the above method, this solution can further optimize the method of determining the target storage node with the help of the location information of the storage node and the intermediate storage device and the load condition of the storage node, so as to make full use of computing resources and reduce cross-regional communication overhead.

[0040] Back to Figure 2 , at 230, several graph data shards in the target storage node are exported to the intermediate storage device.

[0041] In some examples, the intermediate storage device may be a remote storage device, such as a Network File System (NFS), an Object Storage Service (OSS), etc. In some examples, the data shard-related information may be removed when exporting several shards in the target storage node to the intermediate storage device, thereby obtaining a complete copy of the graph data. In some examples, the above-mentioned data shard-related information may include, for example, information for marking the start and end positions of the data shards.

[0042] At 240 , the graph data exported to the intermediate storage device is stored in at least one storage node of the second cluster according to the second graph topology structure information.

[0043] In this embodiment, the second graph topology information may include a correspondence between the graph data shards and the storage nodes of the second cluster where they will be stored. The first cluster and the second cluster may be the same cluster or different clusters. In some examples, when the second cluster is different from the first cluster, the number of storage nodes in the second cluster may be the same as or different from the number of storage nodes in the first cluster.

[0044] In some examples, the number of graph data shards in the second graph topology information is the same as the number of graph data shards in the first graph topology information. The number of storage nodes in the second cluster in the second graph topology information is different from the number of storage nodes in the first cluster in the first graph topology information. In these examples, several graph data shards in the graph data to be backed up can be redeployed to at least one storage node in the second cluster according to the second graph topology information.

[0045] In some examples, the number of graph data shards in the second graph topology information is different from the number of graph data shards in the first graph topology information. The number of storage nodes in the second cluster in the second graph topology information is the same as or different from the number of storage nodes in the first cluster in the first graph topology information. In these examples, the graph data to be backed up that is exported to the intermediate storage device may be resharded, for example, resharded on the basis of several graph data shards exported to the intermediate storage device, or several graph data shards exported to the intermediate storage device may be first integrated into complete graph data and then resharded. Thus, the resharded graph data shards in the graph data to be backed up may be redeployed to at least one storage node in the second cluster according to the second graph topology information.

[0046] Figure 4 A schematic diagram showing yet another example of a data processing method 400 for a distributed graph database according to an embodiment of the present specification.

[0047] like Figure 4 As shown, the first topology information can be used to indicate the corresponding relationship between the graph data shard and the storage node of the first cluster 430 where the graph data shard is located. In one example, storage node 1 stores the primary copy graph data 1 ( Figure 4 In some examples, the distribution of graph data shards of different replicas on corresponding storage nodes may also be different. For example, storage node 4 stores the slave replica graph data 2 ( Figure 4 The storage node 5 stores the shard 3 and shard 5 of the replica graph data 2, and the storage node 6 stores the shard 4 and shard 6 of the replica graph data 2. The above-mentioned shards can be obtained by sharding the graph data. In some examples, the above-mentioned first topology structure information can be saved and maintained by the engine layer 420.

[0048] The platform layer 410 can receive a data backup request. In some examples, the primary copy graph data 1 can be determined as the graph data to be backed up. Afterwards, at least one target storage node (e.g., storage node 1 to storage node 3) where several graph data shards (e.g., shard 1 to shard 6) of the primary copy graph data 1 are located can be determined from the storage nodes of the first cluster 430 based on the first graph topology information. Afterwards, several graph data shards (e.g., shard 1 to shard 6) of the primary copy graph data 1 can be exported to the intermediate storage device 440 so that a complete copy of the graph data is stored in the intermediate storage device 440.

[0049] Next, the platform layer 410 may also receive a data recovery request for the saved copy of the graph data. In some examples, the second graph topology information may be determined by the above data recovery request. Among them, the second graph topology information may include the correspondence between the graph data shards and the storage nodes of the second cluster where they will be stored. In some examples, the second graph topology information may indicate the correspondence between the storage node n+1 in the second cluster 450 and the shards 1, 3, and 5 of the graph data, and the correspondence between the storage node n+2 and the shards 2, 4, and 6 of the graph data 1. The graph data exported to the intermediate storage device 440 may be stored in at least one storage node of the second cluster 450 according to the second graph topology information.

[0050] It can be understood that since the number of storage nodes and the second graph topology information included in the second cluster are not necessarily the same as the number of storage nodes and the first graph topology information included in the first cluster, the distribution of data shards imported into the various storage nodes of the second cluster may be the same as or different from the distribution of data shards indicated by the first graph topology information.

[0051] In some implementations, the intermediate storage device 440 may include cloud storage. The cloud storage may have a regional identifier for the region where the service node is located. At least one target storage node may be determined based on the region indicated by the location information of each storage node of the first cluster 430 and the region indicated by the location information of the intermediate storage device 440. In one example, if the region indicated by the location information of the intermediate storage device 440 is A, the region indicated by the location information of the storage nodes 1, 3, and 5 of the first cluster 430 is A, and the region indicated by the location information of the storage nodes 2, 4, and 6 is B, then when the sharding of the master replica graph data 1 stored by the storage nodes 1 to 3 is the same as the sharding of the slave replica graph data 2 stored by the storage nodes 4 to 6, the storage nodes 1, 3, and 5 may be determined as target storage nodes.

[0052] Through the above method, the cross-region network overhead in the process of data backup and data recovery can be effectively reduced.

[0053] use Figure 1-Figure 4 The data processing method for a distributed graph database disclosed in the document completely exports the determined graph data to be backed up from the target storage node determined according to the correspondence between the graph data shards and the storage nodes of the first cluster where they are located to an intermediate storage device, and then stores the graph data exported to the intermediate storage device in the storage nodes of the second cluster according to the correspondence between the graph data shards and the storage nodes of the second cluster where they will be stored, thereby innovatively proposing to decouple the data export (backup) process and the data import (restore) process through the saved complete data copy. Compared with the traditional method of directly copying each data shard according to the cluster topology of the current distributed system and then restoring it to a new cluster, the topological structure of the cluster (i.e., the second cluster) used for data import in this solution is independent of the topological structure of the original cluster (i.e., the first cluster) during the data import process, and it is not necessary to require the topological structures of the clusters before and after to be completely consistent according to the requirements of data copying, thereby providing a more flexible, practical and efficient solution for data processing.

[0054] Figure 5 A block diagram of an example of a data processing device 500 for a distributed graph database according to an embodiment of the present specification is shown. Figure 2-Figure 4 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0055] like Figure 5 As shown, the data processing device 500 for a distributed graph database may include a node determination unit 510 , a data export unit 520 , and a data import unit 530 .

[0056] The node determination unit 510 is configured to determine the graph data to be backed up, the graph data to be backed up includes a plurality of graph data shards; according to the first graph topology information, determine at least one target storage node where the plurality of graph data shards are located from the storage nodes of the first cluster, the first graph topology information including: the correspondence between the graph data shards and the storage nodes of the first cluster where they are located. The operation of the node determination unit 510 can refer to the above Figure 2 The operations of 210 to 220 are described.

[0057] In some examples, the graph data of the distributed graph database includes master-copy graph data and slave-copy graph data. The master-copy graph data includes several master-copy graph data shards. The slave-copy graph data includes several slave-copy graph data shards. At least one target storage node where the several graph data shards are located is determined from the storage nodes of the first cluster as a storage node for storing the slave-copy graph data shards.

[0058] In some examples, the node determination unit 510 is further configured to determine at least one target storage node where the plurality of graph data shards are located from the storage nodes of the first cluster based on location information of the storage nodes of the first cluster and location information of the intermediate storage device.

[0059] In some examples, the node determination unit 510 is further configured to: determine at least one target storage node where the plurality of graph data shards are located from the storage nodes of the first cluster according to the load conditions of the storage nodes of the first cluster.

[0060] The data export unit 520 is configured to export the plurality of graph data shards in the target storage node to an intermediate storage device. The operation of the data export unit 520 may refer to the above Figure 2 The operation of 230 is described.

[0061] The data import unit 530 is configured to store the graph data exported to the intermediate storage device in at least one storage node of the second cluster according to the second graph topology structure information, wherein the second graph topology structure information includes: the correspondence between the graph data shards and the storage nodes of the second cluster where they will be stored. The operation of the data import unit 530 can refer to the above Figure 2 The operation of 240 is described.

[0062] In some examples, the number of graph data shards in the second graph topology information is the same as the number of graph data shards in the first graph topology information, and the number of storage nodes in the second cluster in the second graph topology information is different from the number of storage nodes in the first cluster in the first graph topology information.

[0063] In some examples, the number of graph data shards in the second graph topology information is different from the number of graph data shards in the first graph topology information, and the number of storage nodes in the second graph topology information is the same as or different from the number of storage nodes in the first graph topology information.

[0064] It should be noted that the operations of the node determination unit 510, the data export unit 520 and the data import unit 530 can refer to the above Figure 2-Figure 4 The corresponding operations described in the embodiments will not be repeated here.

[0065] Reference above Figures 1 to 5 , an embodiment of a data processing method and device for a distributed graph database according to an embodiment of this specification is described.

[0066] The data processing device for a distributed graph database in the embodiments of this specification can be implemented in hardware, software, or a combination of hardware and software. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of the device in which it is located reading the corresponding computer program instructions in the memory into the memory and running them. In the embodiments of this specification, the data processing device for a distributed graph database can be implemented, for example, using an electronic device.

[0067] Figure 6 A schematic diagram showing an example of a data processing device 600 for a distributed graph database according to an embodiment of the present specification.

[0068] like Figure 6 As shown, the data processing device 600 for a distributed graph database may include at least one processor 610, a memory (e.g., a non-volatile memory) 620, a memory 630, and a communication interface 640, and the at least one processor 610, the memory 620, the memory 630, and the communication interface 640 are connected together via a bus 650. At least one processor 610 executes at least one computer-readable instruction stored or encoded in the memory (i.e., the above-mentioned element implemented in the form of software).

[0069] In one embodiment, computer executable instructions are stored in a memory, which when executed cause at least one processor 610 to: determine graph data to be backed up, the graph data to be backed up comprising a number of graph data shards; determine at least one target storage node where the number of graph data shards are located from the storage nodes of the first cluster according to first graph topology information, the first graph topology information comprising: a correspondence between graph data shards and the storage nodes of the first cluster where they are located; export the number of graph data shards in the target storage node to an intermediate storage device; and store the graph data exported to the intermediate storage device in at least one storage node of the second cluster according to second graph topology information, the second graph topology information comprising: a correspondence between the graph data shards and the storage nodes of the second cluster where they will be stored.

[0070] It should be understood that the computer executable instructions stored in the memory, when executed, cause at least one processor 610 to perform the above combined operations in various embodiments of the present specification. Figure 1-Figure 4 Describes the various operations and functions.

[0071] According to one embodiment, a program product such as a computer-readable medium is provided. The computer-readable medium may have instructions (i.e., the above-mentioned elements implemented in the form of software), which, when executed by a computer, causes the computer to perform the above-mentioned combination of various embodiments of this specification. Figure 1-Figure 4 Describes the various operations and functions.

[0072] Specifically, a system or device equipped with a readable storage medium can be provided, on which software program codes that implement the functions of any of the above-mentioned embodiments are stored, and a computer or processor of the system or device can read and execute instructions stored in the readable storage medium.

[0073] In this case, the program code itself read from the machine-readable medium can realize the function of any one of the above-mentioned embodiments, and thus the machine-readable code and the machine-readable storage medium storing the machine-readable code constitute a part of the present invention.

[0074] The computer program code required for the operation of each part of this specification can be written in any one or more programming languages, including object-oriented programming languages, such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB, NET and Python, conventional procedural programming languages ​​such as C language, Visual Basic 2003, Perl, COBOL2002, PHP and ABAP, dynamic programming languages ​​such as Python, Ruby and Groovy, or other programming languages. The program code can be run on the user's computer, or run on the user's computer as an independent software package, or run partly on the user's computer and partly on a remote computer, or all on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any network form, such as a local area network (LAN) or a wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service, such as software as a service (SaaS).

[0075] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Optionally, the program code may be downloaded from a server computer or a cloud via a communication network.

[0076] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0077] Not all steps and units in the above-mentioned processes and system structure diagrams are necessary, and some steps or units can be ignored according to actual needs. The execution order of each step is not fixed and can be determined according to needs. The device structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some units may be implemented by the same physical entity, or some units may be implemented by multiple physical entities, or some components in multiple independent devices may be implemented together.

[0078] The term "exemplary" as used throughout this specification means "serving as an example, instance, or illustration" and does not imply "preferred" or "advantageous" over other embodiments. The detailed description includes specific details for the purpose of providing an understanding of the described techniques. However, the techniques may be implemented without these specific details. In some instances, in order to avoid obscuring the concepts of the described embodiments, well-known structures and devices are shown in block diagram form.

[0079] The optional implementation modes of the embodiments of the present specification are described in detail above in conjunction with the accompanying drawings. However, the embodiments of the present specification are not limited to the specific details in the above implementation modes. Within the technical concept of the embodiments of the present specification, various simple modifications can be made to the technical solutions of the embodiments of the present specification, and these simple modifications all belong to the protection scope of the embodiments of the present specification.

[0080] The above description of the contents of this specification is provided to enable any person of ordinary skill in the art to implement or use the contents of this specification. Various modifications to the contents of this specification will be apparent to those of ordinary skill in the art, and the general principles defined herein may be applied to other variations without departing from the scope of protection of the contents of this specification. Therefore, the contents of this specification are not limited to the examples and designs described herein, but are consistent with the widest range of principles and novel features disclosed herein.

Claims

1. A data processing method for a distributed graph database, wherein the graph data of the distributed graph database is stored in the storage nodes of a first cluster in the form of shards. include: Determine graph data to be backed up, where the graph data to be backed up includes a plurality of graph data shards; Determine at least one target storage node where the plurality of graph data shards are located from the storage nodes of the first cluster according to the first graph topology information, wherein the first graph topology information includes: a correspondence between the graph data shards and the storage nodes of the first cluster where the graph data shards are located; Exporting the plurality of graph data shards in the target storage node to an intermediate storage device; and The graph data exported to the intermediate storage device is stored in at least one storage node of the second cluster according to second graph topology structure information, wherein the second graph topology structure information includes: a correspondence between the graph data shards and the storage nodes of the second cluster where they will be stored, the number of graph data shards in the second graph topology structure information is different from the number of graph data shards in the first graph topology structure information, and the number of storage nodes in the second graph topology structure information is different from the number of storage nodes in the first graph topology structure information.

2. The data processing method according to claim 1, in, The graph data of the distributed graph database includes master-copy graph data and slave-copy graph data, wherein the master-copy graph data includes a plurality of master-copy graph data shards, and the slave-copy graph data includes a plurality of slave-copy graph data shards. At least one target storage node where the plurality of graph data shards are located is determined from the storage nodes of the first cluster as a storage node for storing the slave replica graph data shards.

3. The data processing method according to claim 1, in, The data processing method further includes: According to the location information of the storage nodes of the first cluster and the location information of the intermediate storage device, at least one target storage node where the plurality of graph data shards are located is determined from the storage nodes of the first cluster.

4. The data processing method according to claim 1, in, The data processing method further includes: According to the load conditions of the storage nodes of the first cluster, at least one target storage node where the plurality of graph data shards are located is determined from the storage nodes of the first cluster.

5. A data processing device for a distributed graph database, wherein the graph data of the distributed graph database is stored in the form of shards in the storage nodes of the first cluster, and the data processing device include: A node determination unit is configured to determine graph data to be backed up, wherein the graph data to be backed up includes a plurality of graph data shards; Determine at least one target storage node where the plurality of graph data shards are located from the storage nodes of the first cluster according to the first graph topology information, wherein the first graph topology information includes: a correspondence between the graph data shards and the storage nodes of the first cluster where the graph data shards are located; a data export unit, configured to export the plurality of graph data shards in the target storage node to an intermediate storage device; and A data import unit is configured to store the graph data exported to the intermediate storage device in at least one storage node of the second cluster according to second graph topology structure information, wherein the second graph topology structure information includes: a correspondence between the graph data shards and the storage nodes of the second cluster where they will be stored, the number of graph data shards in the second graph topology structure information is different from the number of graph data shards in the first graph topology structure information, and the number of storage nodes in the second graph topology structure information is different from the number of storage nodes in the first graph topology structure information.

6. The data processing device according to claim 5, in, The graph data of the distributed graph database includes master-copy graph data and slave-copy graph data, wherein the master-copy graph data includes a plurality of master-copy graph data shards, and the slave-copy graph data includes a plurality of slave-copy graph data shards. At least one target storage node where the plurality of graph data shards are located is determined from the storage nodes of the first cluster as a storage node for storing the slave replica graph data shards.

7. The data processing device according to claim 5, in, The node determination unit is further configured to: According to the location information of the storage nodes of the first cluster and the location information of the intermediate storage device, at least one target storage node where the plurality of graph data shards are located is determined from the storage nodes of the first cluster.

8. The data processing device according to claim 5, in, The node determination unit is further configured to: According to the load conditions of the storage nodes of the first cluster, at least one target storage node where the plurality of graph data shards are located is determined from the storage nodes of the first cluster.

9. A data processing device for a distributed graph database, include: At least one processor, a memory coupled to the at least one processor, and a computer program stored on the memory, wherein the at least one processor executes the computer program to implement the data processing method for a distributed graph database as described in any one of claims 1 to 4.