A method and system for preventing hanging edges in a distributed graph database

Abstract

The present application relates to a method and system for preventing hanging edges in a distributed graph database, wherein the method comprises: creating a point ref‑key data storage format and an edge ref‑key data storage format, wherein the point ref‑key and the edge ref‑key are respectively associated with the corresponding point data‑key and the edge data‑key; based on the point ref‑key and the edge ref‑key, operating the graph data stored in the distributed graph database, wherein the operation comprises inserting an edge, and inserting an edge specifically starts a transactional mechanism to insert a forward edge into the distributed graph database, and after completion, inserting a reverse edge into the distributed graph database. Through the present application, the problem of hanging edges that are prone to appear in distributed graph databases is solved, and the design of the ref‑key is implemented to ensure that hanging edges will not appear, and the final consistency of the forward and reverse edges is ensured.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a method and system for preventing hanging edges in a distributed graph database. Background Art

[0002] With the rapid development of big data and artificial intelligence, ultra-large-scale relationship networks are gradually being widely used in the fields of social recommendation, risk control, Internet of Things, blockchain, and security control. As one of the technical cornerstones of all these applications, the storage and computing platform of large-scale distributed relationship networks has attracted more and more attention from academia and industry. This type of relationship network is usually based on the graph theory in data structure. There are two core elements that constitute the graph: points (vertex or node, also known as nodes) and the attributes on the points, and edges (edge, also known as associations or relationships) and the attributes on the edges. For example, a point can correspond to an individual in a social network, and its attributes can be email addresses, account numbers, etc.; an edge can correspond to a friend relationship or a transfer relationship in a social network. The edge has directionality, and the attributes of the edge can be the transfer amount, etc.

[0003] A distributed graph database is a database that is good at processing data structures such as graphs. A mature distributed graph database needs to support ultra-large-scale data networks, with the number of nodes reaching tens to hundreds of billions, and the number of edges reaching trillions. Even if only the nodes and edges are stored, it is far greater than the capacity of a general server. Therefore, there needs to be a way to cut the data in the graph and store it on different logical partitions. OLAP (Online Analytical Processing) type graph processing software often uses point cutting to cut data, while OLTP (Online Transaction Processing) type distributed graph databases mostly use edge cutting. Among them, the edge cutting method is prone to hanging edges, that is, edges with one or two endpoints missing, which is unreasonable in graph theory. This is because in a distributed environment, the starting point, end point and edge are usually stored on two or more servers. .

[0004] Currently, no effective solution has been proposed for the problem of hanging edges that are prone to occur in distributed graph databases in related technologies. Summary of the invention

[0005] The embodiments of the present application provide a method and system for preventing hanging edges in a distributed graph database, so as to at least solve the problem that hanging edges are prone to appear in distributed graph databases in related technologies.

[0006] In a first aspect, an embodiment of the present application provides a method for preventing hanging edges in a distributed graph database, the method comprising:

[0007] In the distributed graph database, a point ref-key data storage format is created according to the tag information of the point, and an edge ref-key data storage format is created according to the point ID information corresponding to the edge, wherein the point ref-key and the edge ref-key are associated with the corresponding point data-key and edge data-key respectively;

[0008] Based on the point ref-key and the edge ref-key, operations are performed on the graph data stored in the distributed graph database, wherein the operations include inserting edges, and the inserting edges specifically starts a transactional mechanism to insert reverse edges into the distributed graph database, and after completion, the forward edges are also inserted into the distributed graph database.

[0009] In some embodiments, inserting the reverse edge into the distributed graph database and, upon completion, inserting the forward edge into the distributed graph database also includes:

[0010] Check whether the starting point ref-key exists. If not, return an error. If yes, lock the starting point ref-key.

[0011] Check whether the target point ref-key exists. If not, return an error and unlock the starting point ref-key. If yes, lock the target point ref-key.

[0012] Insert a pre-written forward edge into the edge data-key of the data shard at the starting point, and insert a reverse edge into the edge data-key of the data shard at the target point,

[0013] The pre-written forward edge is deleted from the edge data-key of the data shard at the starting point, and then the forward edge is inserted.

[0014] In some embodiments, the operation further includes Tag information of the insertion point, and the Tag information of the insertion point is specifically:

[0015] Lock the point that needs to be operated, write the Tag information into the point data-key, and write the Tag information into the point ref-key.

[0016] In some embodiments, the operation further includes reading Tag information of the point, and the Tag information of the point is specifically:

[0017] In case of reading a point or a tag of a point, the point or the tag is directly read from the point ref-key;

[0018] When reading the Tag attribute of a point, read the point ref-key to confirm whether there is a corresponding Tag. If not, return an error. If so, query the associated point data-key based on the point ref-key and read the Tag attribute from the point data-key.

[0019] In some embodiments, the operation further includes obtaining the out-degree and in-degree of the point, and the obtaining the out-degree and in-degree of the point is specifically:

[0020] Find the forward edge and reverse edge corresponding to the point, and obtain the out-degree and in-degree of the point from the edge ref-key of the forward edge and the edge ref-key of the reverse edge.

[0021] In some embodiments, the operation further includes deleting a point or an edge, and the deleting a point or an edge is specifically:

[0022] Mark the corresponding field in the vertex or edge ref-key as deleted;

[0023] Start the garbage collection mechanism to delete the point or edge data-key associated with the point or edge ref-key.

[0024] In some embodiments, marking a corresponding field in a vertex or edge ref-key as deleted includes:

[0025] If the deletion is a deletion of an edge, the edge ref-key of the edge is marked as deleted;

[0026] If the deletion is to delete the Tag information of the point, the corresponding Tag field in the point ref-key of the point is marked as deleted;

[0027] If the deletion is a deletion point, determine whether the point is an orphan node. If so, mark the point ref-key of the point as deleted. If not, mark all forward edge ref-keys and reverse edge ref-keys of the point as deleted, and also mark the point ref-key of the point as deleted.

[0028] In some embodiments, deleting the point or edge data-key associated with the point or edge ref-key includes:

[0029] Scan each vertex or edge data-key in sequence and check whether the vertex or edge data-key exists in the associated vertex or edge ref-key. If not, delete the vertex or edge data-key.

[0030] In some of the embodiments, the method includes: storing the ref-key and the data-key separately, and allocating resident memory for the ref-key.

[0031] In a second aspect, an embodiment of the present application provides a system for preventing hanging edges in a distributed graph database, the system comprising a data construction module and a data operation module;

[0032] The data construction module creates a point ref-key data storage format according to the tag information of the point in the distributed graph database, and creates an edge ref-key data storage format according to the point ID information corresponding to the edge, wherein the point ref-key and the edge ref-key are associated with the corresponding point data-key and edge data-key respectively;

[0033] The data operation module operates the graph data stored in the distributed graph database based on the point ref-key and the edge ref-key, wherein the operation includes inserting an edge, and the inserting an edge specifically starts a transactional mechanism to insert a reverse edge into the distributed graph database, and after completion, inserts a forward edge into the distributed graph database.

[0034] Compared with the related art, the embodiment of the present application provides a method and system for preventing hanging edges in a distributed graph database, which creates a point ref-key data storage format according to the tag information of the point in the distributed graph database, and creates an edge ref-key data storage format according to the point ID information corresponding to the edge, wherein the point ref-key and the edge ref-key are respectively associated with the corresponding point data-key and edge data-key; based on the point ref-key and the edge ref-key, operations are performed on the graph data stored in the distributed graph database, wherein the operations include inserting edges, and inserting edges specifically starts a transactional mechanism to insert the forward edge into the distributed graph database, and after completion, the reverse edge is also inserted into the distributed graph database, which solves the problem of hanging edges that are prone to appear in the distributed graph database, and implements the design of ref-key to ensure that hanging edges will not appear; and based on locking in memory and writing the reverse edge first and then the forward edge, the final consistency of the forward edge and the reverse edge is ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0036] Figure 1 This is a diagram of how Nebula Graph stores data.

[0037] Figure 2 is a flowchart of the steps of a method for preventing hanging edges in a distributed graph database according to an embodiment of the present application;

[0038] Figure 3 is a schematic diagram of a process of inserting an edge according to an embodiment of the present application;

[0039] Figure 4 is a schematic diagram of the process of inserting Tag information of a point according to an embodiment of the present application;

[0040] Figure 5 is a structural block diagram of a system for preventing hanging edges in a distributed graph database according to an embodiment of the present application;

[0041] Figure 6 It is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application.

[0042] Description of the drawings: 51. Data construction module; 52. Data operation module. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0044] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0045] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0046] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0047] A distributed graph database is a database that is good at processing graph data structures. It can complete graph traversal operations in milliseconds. Points and edges constitute the graph structure, which is a data structure that must exist in every graph. Both the breadth-first traversal and depth-first traversal algorithms in the graph must use the graph structure. The attributes on the points and the attributes on the edges constitute the graph attributes. Graph attributes are additional data structures added for the convenience of users. They may or may not be used during graph traversal.

[0048] A mature distributed graph database needs to support ultra-large-scale data networks, with the number of nodes reaching tens to hundreds of billions, and the number of edges reaching trillions. Even if only nodes and edges are stored, the capacity is far greater than that of a general server. Therefore, there needs to be a way to cut the data in the graph and store it on different logical partitions. OLAP (Online Analytical Processing) type graph processing (graph computing) software often uses point cutting to cut data, while OLTP (Online Transaction Processing) type distributed graph databases mostly use edge cutting.

[0049] Take Nebula Graph distributed graph database as an example. Figure 1 This is a schematic diagram of Nebula Graph's data storage method. Figure 1 As shown in the figure, points are stored in different data shards, and a physical edge is stored as two logical edges. These two edges are stored in one data shard with the starting point and the destination point respectively. Table 1 is an example table of the point data storage format, and Table 2 is an example table of the edge data storage format.

[0050] Table 1

[0051]

[0052]

[0053] Table 2

[0054]

[0055] It can be seen that when modeling graph elements in Nebula Graph, two points and one edge are modeled as four independent keys, that is, two points plus two edges divided by one edge.

[0056] Therefore, there may be problems with dangling edges, such as when inserting data, only the edge is inserted but not the vertex. When deleting data, the edge key is not deleted successfully, but the vertex key is deleted successfully. At this time, the edge key that was missed to be deleted becomes a junk key.

[0057] Furthermore, it will also make it slow to query the in-degree of a certain point. For example, you must scan the prefix of the point or create an index to calculate the in-degree. The edge data corresponding to the same point is distributed as multiple sst (Static Sorted Table) files on the hard disk. It will be very slow to count the out-edges of the super node because a large number of sst files need to be read.

[0058] Furthermore, in actual use, most graph calculations and graph traversals only require the graph structure (i.e., key information), and do not require the attributes of nodes and edges (i.e., graph attributes). Due to this modeling approach, a lot of useless data will be returned when scanning by row, which will significantly reduce query performance.

[0059] In summary, the present application proposes a method and system for preventing hanging edges in a distributed graph database, which are used to solve the above-mentioned problems existing in the distributed graph database.

[0060] The present application embodiment provides a method for preventing hanging edges in a distributed graph database. Figure 2 is a flowchart of the steps of a method for preventing hanging edges in a distributed graph database according to an embodiment of the present application, such as Figure 2 As shown, the method comprises the following steps:

[0061] Step S202, in the distributed graph database, a point ref-key data storage format is created according to the tag information of the point, and an edge ref-key data storage format is created according to the point ID information corresponding to the edge, wherein the point ref-key and the edge ref-key are associated with the corresponding point data-key and edge data-key respectively;

[0062] Specifically, two new data storage formats are added: point ref-key (point reference key) and edge ref-key (edge ​​reference key), where point ref-key and edge ref-key are associated with corresponding point data-key and edge data-key respectively. It should be noted that data-key is the original data in the distributed graph database, and ref_key is equivalent to a newly designed pointer. Table 3 is an example table of the description of the point ref-key data storage format, and Table 4 is an example table of the description of the edge ref-key data storage format. Table 3

[0063]

[0064] Table 4

[0065]

[0066] Preferably, data-key is placed in block-based RocksDB instance, and ref-key is placed in plain-table based RocksDB instance. The purpose is to speed up the reading speed of ref-key. Block-based table and plain table are storage types in RocksDB.

[0067] Step S204, based on the point ref-key and edge ref-key, operates on the graph data stored in the distributed graph database, where the operation includes inserting the edge. Inserting the edge specifically starts the transactional mechanism to insert the reverse edge into the distributed graph database. After completion, the forward edge is also inserted into the distributed graph database.

[0068] Specifically, Figure 3 is a schematic diagram of the process of inserting an edge according to an embodiment of the present application, such as Figure 3 As shown;

[0069] Check whether the starting point ref-key exists in the memory. If not, return an error. If yes, lock the starting point ref-key.

[0070] Check whether the target point ref-key exists in the memory. If not, return an error and unlock the starting point ref-key. If yes, lock the target point ref-key.

[0071] Insert the pre-written forward edge (e') into the edge data-key of the data shard at the starting point, and insert the reverse edge (-e) into the edge data-key of the data shard at the target point. Preferably, the Count field is incremented by 1.

[0072] Delete the pre-written forward edge (e') in the edge data-key of the data shard at the starting point, and then insert the forward edge (e). Preferably, the Count field is increased by 1.

[0073] It should be noted that the above process uses TOSS (Transaction on Storage Side) technology to prevent the situation where one logical edge is successfully inserted while another logical edge fails to be inserted. The process of inserting edges after starting TOSS ensures the transactionality between multiple machines in the distributed cluster.

[0074] Through steps S202 to S204 in the embodiment of the present application, the problem of dangling edges that are prone to appear in distributed graph databases is solved, and the design of ref-key is implemented to ensure that dangling edges do not appear; in addition, based on locking in memory and writing the reverse edge first and then the forward edge, the ultimate consistency of the forward edge and the reverse edge is ensured.

[0075] In some embodiments, the operation further includes inserting tag information of the point:

[0076] Lock the point that needs to be operated, write the Tag information into the point data-key, and write the Tag information into the point ref-key.

[0077] Specifically, Figure 4 is a schematic diagram of the process of inserting Tag information of a point according to an embodiment of the present application, such as Figure 4 As shown, the point (VertexID) to be operated is locked in the memory, the Tag information is written into the point data-key, the Tag information is written into the point ref-key, and the CF Tag in the memory is updated.

[0078] In some embodiments, the operation further includes reading tag information of the point:

[0079] When reading a point or a tag of a point (without reading specific attributes), the point or tag is directly read from the point ref-key;

[0080] When reading the Tag attribute of a point, read the point ref-key to confirm whether the corresponding Tag exists. If not, an error is returned. If so, query the associated point data-key based on the point ref-key and read the Tag attribute from the point data-key.

[0081] Furthermore, reading edges is also based on the same steps, which will not be described in detail here.

[0082] In some embodiments, the operation further includes obtaining the out-degree and in-degree of the point:

[0083] Find the forward edge and reverse edge corresponding to the point, and obtain the out-degree and in-degree of the point from the edge ref-key of the forward edge and the edge ref-key of the reverse edge.

[0084] Specifically, according to the prefix (Type+PartID+VertexID), find the forward edge and reverse edge corresponding to the point (there is a field called Edge-type in ref-key), and obtain the out-degree and in-degree of the point from the Count field of the forward edge ref-key and the reverse edge ref-key.

[0085] Preferably, after obtaining the out-degree and in-degree of a point, determine whether the point is a supernode, and add up the counts of various edge types to get the sum of the out-degree and in-degree of the point. Generally, it can be set that when the accumulated value exceeds 10,000, the point is marked as a supernode. This marking action can be persistent (written to hard disk) or pure memory (cached after each calculation).

[0086] In some embodiments, the operation further comprises deleting a vertex or an edge:

[0087] Mark the corresponding field in the vertex or edge ref-key as deleted; start the garbage collection mechanism and delete the vertex or edge data-key associated with the vertex or edge ref-key.

[0088] Specifically, if the deletion operation is to delete an edge, the edge ref-key of the edge is marked as deleted; the deletion of the associated edge data-key can be performed through a later GC (Garbage Collection).

[0089] If the deletion operation is to delete the Tag information of the point, the corresponding Tag field in the point ref-key of the point is marked as deleted; the garbage collection mechanism is started to delete the point data-key associated with the point ref-key.

[0090] Preferably, a point can have multiple tags. If the tag to be removed is not the last one (such as Tag1), directly remove the Tag1+Ref1 field in the value field of the point ref-key; the specific attribute part (point data-key) is handed over to GC for execution. If the tag to be removed is the last one, it is equivalent to deleting this point and all the edges around it.

[0091] If the deletion operation is to delete a point, determine whether the point is an orphan node. If so, mark the point ref-key of the point as deleted. If not, mark all forward edge ref-keys and reverse edge ref-keys of the point as deleted, and also mark the point ref-key of the point as deleted. Start the garbage collection mechanism and delete the associated data-key.

[0092] Preferably, if a vertex has no outgoing and incoming edges, it is an orphan node. Note that in graph theory, orphan nodes are legal. You can check whether there is a corresponding edge ref-key (the two share the same prefix) based on the prefix of the vertex ref-key to determine whether it is an orphan node.

[0093] Preferably, if the deleted point is a non-orphan node, logically, deleting the non-orphan node must delete the edges around it, otherwise dangling edges will be caused.

[0094] Basic implementation: You can first delete the edges around a point one by one, and then delete the point.

[0095] Optimization implementation 1: Group all edges around the point according to the partition (data shard) number, and perform batch operations in units of partition.

[0096] Optimization implementation 2: Determine whether the node is a super node. If so, in addition to grouping by partition number, each CountN operation forms a batch. For example, deleting 1 million in a batch will cause serious network and hard disk pressure; every 1,000 batches of pressure are dispersed into multiple times. Avoid long-term network and hard disk preemption. CountN is the maximum number of edges per row set in the Ref-key above. At this time, the single batch reading efficiency is the highest.

[0097] In some embodiments, deleting a vertex or edge data-key associated with a vertex or edge ref-key includes:

[0098] Scan each vertex or edge data-key in sequence and check whether the vertex or edge data-key exists in the associated vertex or edge ref-key. If not, delete the vertex or edge data-key.

[0099] Specifically, GC (Garbage Collection) is implemented through Rocksdb's customized compaction task. The basic steps are as follows: when compaction starts, each data-key is scanned in sequence and checked whether the data-key exists in the associated ref-key. If not, the data-key is discarded; otherwise, the next data-key is operated.

[0100] In some of these embodiments, the method includes storing the ref-key separately from the data-key (e.g., placing the data-key in a block-based RocksDB instance and the ref-key in a plain-table based RocksDB instance), and allocating resident memory for the ref-key.

[0101] Specifically, to ensure query efficiency, the ref-key needs to reside in memory and be stored separately from the data-key.

[0102] Table 5 is an example table of estimating the memory space occupied by edge ref-key. Taking 1 billion edges as an example, the estimation formula for the memory space occupied by edge ref-key is:

[0103] Number of replicas * 2 (forward and reverse edges) * 1G (1 billion data volumes) * number of bytes per row / CountN.

[0104] Table 5

[0105] CountN Bytes per row Memory Estimation 10 96 57.6GB 50 416 49.92GB 100 816 48.96GB

[0106] Preferably, CountN can be set to 50 to reduce memory usage. A single read of a single ref-key can obtain 50 edges corresponding to the point. The measured performance is close to that of CountN=10. However, frequent splitting is not required. The performance and memory usage are roughly balanced.

[0107] In some embodiments, the method further comprises a compatible upgrade of the data:

[0108] In order to be compatible with the original data, the general upgrade steps are as follows: each partition is upgraded separately (so it can be executed concurrently in the distributed database), and its existing data-key is scanned in sequence to construct its corresponding ref-key.

[0109] Specifically, if a data-key is a point, the construction method is: first ensure that the ref-key corresponding to the point does not exist, then read the point information, construct a new point ref-key, and when there are multiple tags, add them to the "CFTag" of the existing ref-key.

[0110] If a data-key is an edge, the construction method is: first ensure that the ref-key corresponding to the starting point or destination point exists, and the ref-key of the edge does not exist; then read the edge information and construct a new edge ref-key; when there are multiple edges, add the destination point to the "CF Edge" of the existing ref-key, and then update the "Count".

[0111] When all data-keys in the partition are scanned, the partition is marked as completed. When all partitions are completed, the historical data compatibility upgrade of the distributed graph database is completed.

[0112] Preferably, in the above steps, if new data is continuously written during the above scanning and upgrading process, the scanning will be incomplete. Therefore, before starting the scan, take a snapshot. Complete the upgrade of this snapshot; thereafter, replay the WAL (write-ahead-log) log of all write operations after the snapshot. Perform hot upgrades through periodic snapshots + replaying WAL. Furthermore, if a failure occurs, it can also be restored through periodic snapshots + replaying WAL.

[0113] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0114] The embodiment of the present application provides a system for preventing hanging edges in a distributed graph database. Figure 5 is a structural block diagram of a system for preventing hanging edges in a distributed graph database according to an embodiment of the present application, such as Figure 5 As shown, the system includes a data construction module 51 and a data operation module 52;

[0115] The data construction module 51 creates a point ref-key data storage format according to the tag information of the point in the distributed graph database, and creates an edge ref-key data storage format according to the point ID information corresponding to the edge, wherein the point ref-key and the edge ref-key are associated with the corresponding point data-key and edge data-key respectively;

[0116] The data operation module 52 operates on the graph data stored in the distributed graph database based on the point ref-key and the edge ref-key, wherein the operation includes inserting the edge. Inserting the edge specifically starts the transactional mechanism to insert the reverse edge into the distributed graph database. After completion, the forward edge is also inserted into the distributed graph database.

[0117] Through the data construction module 51 and the data operation module 52 in the embodiment of the present application, the problem of dangling edges that are prone to appear in distributed graph databases is solved, and the design of ref-key is implemented to ensure that dangling edges will not appear; and based on locking in memory and writing the reverse edge first and then the forward edge, the ultimate consistency of the forward edge and the reverse edge is ensured.

[0118] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0119] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0120] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0121] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0122] In addition, in combination with the method for preventing hanging edges in a distributed graph database in the above embodiment, the embodiment of the present application can provide a storage medium for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, any one of the methods for preventing hanging edges in a distributed graph database in the above embodiment is implemented.

[0123] In one embodiment, a computer device is provided, which may be a terminal. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for preventing hanging edges in a distributed graph database is implemented. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball, or touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0124] In one embodiment, Figure 6 is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application, such as Figure 6 As shown, an electronic device is provided, which may be a server, and its internal structure diagram may be as shown in Figure 6 As shown. The electronic device includes a processor, a network interface, an internal memory and a non-volatile memory connected through an internal bus, wherein the non-volatile memory stores an operating system, a computer program and a database. The processor is used to provide computing and control capabilities, the network interface is used to communicate with an external terminal through a network connection, the internal memory is used to provide an environment for the operation of the operating system and the computer program, the computer program is executed by the processor to implement a method for preventing hanging edges in a distributed graph database, and the database is used to store data.

[0125] Those skilled in the art will understand that Figure 6 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0126] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0127] Those skilled in the art should understand that the technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0128] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for preventing hanging edges in a distributed graph database, characterized in that: The method comprises: In the distributed graph database, a point ref-key data storage format is created according to the tag information of the point, and an edge ref-key data storage format is created according to the point ID information corresponding to the edge, wherein the point ref-key and the edge ref-key are associated with the corresponding point data-key and edge data-key respectively; Based on the point ref-key and the edge ref-key, the graph data stored in the distributed graph database is operated, wherein the operation includes inserting an edge, and the inserting edge specifically starts a transactional mechanism, checks whether a starting point ref-key exists, and if not, returns an error, and if so, locks the starting point ref-key; checks whether a target point ref-key exists, and if not, returns an error and unlocks the starting point ref-key, and if so, locks the target point ref-key; Insert a pre-written forward edge into the edge data-key of the data shard at the starting point, insert a reverse edge into the edge data-key of the data shard at the target point, delete the pre-written forward edge in the edge data-key of the data shard at the starting point, and then insert the forward edge.

2. The method according to claim 1, characterized in that The operation also includes Tag information of the insertion point, and the Tag information of the insertion point is specifically: Lock the point that needs to be operated, write the Tag information into the point data-key, and write the Tag information into the point ref-key.

3. The method according to claim 1, characterized in that The operation also includes reading the tag information of the point, and the tag information of the point is specifically: In case of reading a point or a tag of a point, the point or the tag is directly read from the point ref-key; When reading the Tag attribute of a point, read the point ref-key to confirm whether there is a corresponding Tag. If not, return an error. If so, query the associated point data-key based on the point ref-key and read the Tag attribute from the point data-key.

4. The method according to claim 1, characterized in that: The operation further includes obtaining the out-degree and in-degree of the point, and the out-degree and in-degree of the point are specifically: Find the forward edge and reverse edge corresponding to the point, and obtain the out-degree and in-degree of the point from the edge ref-key of the forward edge and the edge ref-key of the reverse edge.

5. The method according to claim 1, characterized in that The operation also includes deleting a point or an edge, and the deleting point or an edge is specifically: Mark the corresponding field in the vertex or edge ref-key as deleted; Start the garbage collection mechanism to delete the point or edge data-key associated with the point or edge ref-key.

6. The method according to claim 5, characterized in that Marking the corresponding field in the vertex or edge ref-key as deleted includes: If the deletion is a deletion of an edge, the edge ref-key of the edge is marked as deleted; If the deletion is to delete the Tag information of the point, the corresponding Tag field in the point ref-key of the point is marked as deleted; If the deletion is a deletion point, determine whether the point is an orphan node. If so, mark the point ref-key of the point as deleted. If not, mark all forward edge ref-keys and reverse edge ref-keys of the point as deleted, and also mark the point ref-key of the point as deleted.

7. The method according to claim 5, characterized in that Deleting the point or edge data-key associated with the point or edge ref-key includes: Scan each vertex or edge data-key in sequence and check whether the vertex or edge data-key exists in the associated vertex or edge ref-key. If not, delete the vertex or edge data-key.

8. The method according to claim 1, characterized in that: The method includes: storing the ref-key and the data-key separately, and allocating a resident memory for the ref-key.

9. A system for preventing hanging edges in a distributed graph database, characterized in that: The system includes a data construction module and a data operation module; The data construction module creates a point ref-key data storage format according to the tag information of the point in the distributed graph database, and creates an edge ref-key data storage format according to the point ID information corresponding to the edge, wherein the point ref-key and the edge ref-key are associated with the corresponding point data-key and edge data-key respectively; The data operation module operates the graph data stored in the distributed graph database based on the point ref-key and the edge ref-key, wherein the operation includes inserting an edge, and the inserting an edge specifically starts a transactional mechanism, checks whether a starting point ref-key exists, and if not, returns an error; if so, locks the starting point ref-key; checks whether a target point ref-key exists, and if not, returns an error and unlocks the starting point ref-key; if so, locks the target point ref-key; inserts a pre-written forward edge into the edge data-key of the data shard of the starting point, inserts a reverse edge into the edge data-key of the data shard of the target point, deletes the pre-written forward edge in the edge data-key of the data shard of the starting point, and then inserts the forward edge.

Images