A graph data construction method and query method based on KV database

Abstract

The present invention relates to a graph data construction method and query method based on a KV database. The steps of the method include: mapping the node data in the attribute graph model into a KV structure; mapping the relationship data in the attribute graph model into a KV structure; based on the KV structure, storing the attribute graph data in the KV database as the basis to obtain a KV graph database. The present invention combines the characteristics of graph data storage and query, designs a graph data storage method based on KV storage, stores complex graph data in a simple KV database, and can effectively improve storage efficiency; on this basis, designs a conversion from graph query to KV query, which can improve query efficiency while ensuring correctness and integrity; compared with the native storage of existing graph data, the present invention stores graph data based on the KV database, has a simpler structure, and is easier to implement distributed expansion.

Description

Technical Field

[0001] The present invention relates to the fields of graph data, big data technology, database technology, etc., and proposes a graph database construction method and a query method based on a KV database. Background Art

[0002] With the advent of the big data era, traditional relational databases are unable to effectively manage and analyze large-scale, complex relational data. Compared to relational models, graph data models offer intuitive expression, flexible structure, and superior performance when processing complex relational data. Thanks to these advantages, graph databases based on these models have seen rapid growth in recent years and are widely used in fields such as financial services, biomedicine, and social networks.

[0003] KV database refers to a Key-value database, which is a non-relational database that uses a simple key-value method to store data. A key-value database stores data as a collection of key-value pairs, where the key serves as a unique identifier. The key-value distributed storage system has fast query speed, stores large amounts of data, and supports high concurrency. It is very suitable for queries by primary key, but cannot perform complex conditional queries. The graph data model models data with nodes and edges, describes the data model of the relationship between entities, and uses attributes to extend the characteristics of nodes and edges. The Seek operation in the KV database locates the first matching record based on the complete key or partial key as a prefix, and places the cursor on the record. The Scan operation in the KV database starts at the cursor and sequentially traverses each subsequent record until the termination condition is met.

[0004] Most existing graph databases use native graph data storage, meaning they employ underlying storage solutions specifically designed for graph data. These storage solutions are complex to design and difficult to optimize and scale in a distributed manner for large-scale data storage. Summary of the Invention

[0005] The present invention aims to provide a graph database construction method and query method based on a KV database. KV databases use a simple key-value model and support distributed storage. Therefore, the present invention designs a rational construction method within the KV database, stores complex graph data within a simple KV database, and designs query methods to support graph queries. This allows graph data to be stored and queried within the KV database, resolving the aforementioned issues with native graph databases.

[0006] The technical solutions of the present invention are as follows:

[0007] In a first aspect, the present invention provides a method for constructing a graph database based on a KV database, the steps of which include:

[0008] 1) Map the node data in the property graph model into a KV structure;

[0009] 2) Map the relational data in the property graph model into a KV structure;

[0010] 3) Based on the KV structure obtained in steps 1) to 2), the attribute graph data is stored in the KV database to obtain a KV graph database.

[0011] Furthermore, mapping the node data in the property graph model into a KV structure includes:

[0012] 1) The node storage structure includes two KV data tables: node and nodeLabel. Each data table contains two parts: Key and Value.

[0013] 2) The key portion of the node table is 12 bytes in total. The first 4 bytes store the labelId (label id) corresponding to the node's label, and the last 8 bytes store the node id. The value portion has a variable length and is the serialized binary data of the node's attributes.

[0014] 3) The key part of the nodeLabel table is 12 bytes in total. The first 8 bytes store the node id, and the last 4 bytes store the labelId corresponding to the node label. The value part is empty.

[0015] 4) For single-label nodes, each node stores one row in each of the above two tables;

[0016] 5) For multi-label nodes, store the number of rows of labels in each of the two tables;

[0017] 6) For unlabeled nodes, use the specified labelId representing no label as its label storage.

[0018] Furthermore, mapping the relational data in the property graph model into a KV structure includes:

[0019] 1) The relational storage structure includes four KV data tables: relation (relationship table), in (input table), out (output table), and typeRelation (relationship type table). Each data table contains two parts: Key and Value.

[0020] 2) The relation table directly stores relation data. Its key is the relationId (relation id) that uniquely represents a relation and is 8 bytes long. Its value is the binary data of the serialized relation attributes.

[0021] 3) The in table records the incoming direction of the relationship; its key part is 20 bytes in total, the first 8 bytes store the id of the end node of the directed relationship, the middle 4 bytes store the typeId (type id) corresponding to the type of the relationship, and the last 8 bytes store the id of the starting node of the directed relationship; its value part stores the relationId of the relationship;

[0022] 4) The out table records the outbound direction of the relationship; its key part is 20 bytes in total, the first 8 bytes store the id of the starting node of the directed relationship, the middle 4 bytes store the typeId corresponding to the type of the relationship, and the last 8 bytes store the id of the end node of the directed relationship; its value part stores the relationId of the relationship;

[0023] 5) The typeRelation table records the relationship type. The key part is 12 bytes in total. The first 4 bytes store the typeId corresponding to the relationship type, and the last 8 bytes store the relationId of the relationship.

[0024] 6) In the above relational storage structure, all relationships are unidirectional, and bidirectional relationships are considered as two relationships in opposite directions;

[0025] 7) In this relational storage structure, each relation has exactly one relation type. For a graph data relation, one row of data needs to be stored in each of the four KV tables.

[0026] In a second aspect, the present invention provides a method for performing data query on a KV graph database, which converts a graph query into a KV query.

[0027] Furthermore, the graph query includes:

[0028] 1) Query the node according to the node ID;

[0029] 2) Query nodes based on their labels;

[0030] 3) Query the relationship based on the relationship ID;

[0031] 4) Query relationships based on relationship type;

[0032] 5) Query the relationship based on the relationship starting node;

[0033] 6) Query the relationship based on the relationship termination node.

[0034] Furthermore, querying nodes in the KV graph database based on node IDs includes:

[0035] ① Use the node ID as the first 8-byte prefix of the nodeLabel table key, use this prefix to perform a Seek operation in the nodeLabel table, and parse out the nodeId and labelId in the key of the first record found by Seek;

[0036] ② Use the parsed labelId and nodeId to form the key of the node table, perform a seek operation, and deserialize the obtained record value to obtain the query result.

[0037] Furthermore, querying nodes based on their labels in the KV graph database includes:

[0038] ① Use labelId corresponding to label as the first 4 bytes prefix of node table key, and use this prefix to perform Seek operation in node table;

[0039] ②Execute the Scan operation on the node table until the prefix does not match. During the process, the value of each record is deserialized to obtain the query result.

[0040] Furthermore, in the KV graph database, a relationship is queried based on the relationship ID, including: using the relationship ID as the key of the relation table, performing a seek operation, and deserializing the obtained record value to obtain the query result.

[0041] Furthermore, in the KV graph database, relationships are queried based on relationship type, including:

[0042] ① Use the typeId corresponding to the relationship type as the first 4-byte prefix of the typeRelation table Key, and use this prefix to perform a Seek operation in the typeRelation table;

[0043] ② Perform a Scan operation on the typeRelation table until the prefix does not match. During the process, record the last 8 bytes of the key as the relation id;

[0044] ③ The query result is obtained by executing the operation of querying the relationship based on the relationship ID in 3).

[0045] Furthermore, querying a relationship based on the relationship start node includes:

[0046] ① Use the starting node ID as the first 8-byte prefix of the out table key and use this prefix to perform a Seek operation in the out table;

[0047] ② Perform a Scan operation on the out table until the prefix does not match, parsing each row of Value to obtain the relation ID;

[0048] ③ The query result is obtained by executing the operation of querying the relationship based on the relationship ID in 3).

[0049] Furthermore, querying a relationship based on the relationship termination node includes:

[0050] ① Use the starting node ID as the first 8-byte prefix of the in table key and use this prefix to perform a Seek operation in the in table;

[0051] ② Perform a Scan operation on the in table until the prefix does not match, parsing each row value to obtain the relationship ID;

[0052] ③ The query result is obtained by executing the operation of querying the relationship based on the relationship ID in 3).

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] The present invention combines the characteristics of graph data storage and query to design a graph data storage method based on KV storage, which can effectively improve storage efficiency. On this basis, a conversion from graph query to KV query is designed, which can improve query efficiency while ensuring correctness and integrity. Based on the above design, a method for storing graph data in a KV database is implemented. Compared with the native storage of existing graph data, the present invention stores graph data based on a KV database, which has a simpler structure and is easier to implement distributed expansion. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a sample data diagram of property graph data.

[0056] Figure 2 This is a schematic diagram of node storage in the KV graph database.

[0057] Figure 3 This is a diagram of the relational storage of the KV graph database. DETAILED DESCRIPTION

[0058] The present invention will be further described below with reference to specific embodiments and accompanying drawings.

[0059] 1. Based on Figure 1 The property graph data shown is stored in the KV graph database:

[0060] Example data of property graph data is as follows Figure 1 As shown, it contains 3 nodes and 3 relationships.

[0061] The nodes in the KV graph database are stored as follows Figure 2As shown in the example, there are two tables: node and nodeLabel. In the example data, the node with ID 3 has both the labels Women and Student. Therefore, two rows are required to store each node.

[0062] The relationship storage in the KV graph database is as follows Figure 3 As shown, there are four tables: relation, typeRelation, in, and out.

[0063] 2. Based on Figure 2 、 Figure 3 The graph storage shown here gives an example of its query in the KV graph database:

[0064] 1) Example 1. Querying a node based on its node ID (nodeId=3)

[0065] ① Use nodeId(3) as a prefix to perform a Seek operation in the nodeLabel table, match the third row of data, and parse out the 8-byte data labelId=1 after the key of the row of data.

[0066] ② Combine labelId(1) and nodeId(3) into a key, perform a Seek operation on the node table, match the second row of data, read the binary data in the Value section, and deserialize it to obtain the query result.

[0067] 2) Example 2. Querying nodes based on their label (label = Women)

[0068] ① Use the labelId (1) corresponding to Women as a prefix to perform a Seek operation in the node table to match the first row of data.

[0069] ②Scan the node table until the prefix does not match, that is, stop at row 3. During the Scan process, read the value of the matching row and deserialize the binary data to obtain the query result.

[0070] 3) Example 3. Querying a relationship based on relation ID (relationId=1)

[0071] Use relationId(1) as a prefix to perform a Seek operation in the relation table, match the first row, read the binary data in the Value part, and deserialize it to obtain the query result.

[0072] 4) Example 4. Querying relationships based on relationship type (type = friend)

[0073] ① Use typeId(2) corresponding to friend as a prefix to perform a Seek operation in the typeRelation table to match the second row.

[0074] ② Scan the typeRelation table until the prefix does not match, which means the table is terminated. During this process, the relationId (2, 3) in the key of the matching row is parsed.

[0075] ③ Use relationId(2,3) as a prefix to perform a Seek operation in the relation table, match rows 2 and 3, read the binary data in the Value part, and deserialize it to obtain the query result.

[0076] 5) Example 5. Querying a relationship based on its starting node (startId=1)

[0077] ① Use startId(1) as a prefix to perform a Seek operation in the out table to match the first row.

[0078] ② Scan the out table until the prefix does not match, that is, stop at row 3. During this process, parse the value of the matching row and obtain relationId(1,2).

[0079] ③ Use relationId(1,2) as a prefix to perform a Seek operation in the relation table, match rows 1 and 2, read the binary data in the Value part, and deserialize it to obtain the query result.

[0080] 6) Example 6. Querying a relationship based on its end node (endId = 2)

[0081] ① Use endId(2) as a prefix to perform a Seek operation in the in table to match the first row.

[0082] ② Scan the in table until the prefix does not match, that is, stop at row 3. During this process, parse the value of the matching row and obtain relationId(1,3).

[0083] ③ Use relationId(1,3) as a prefix to perform a Seek operation in the relation table, match rows 1 and 3, read the binary data in the Value part, and deserialize it to obtain the query result.

[0084] The present invention can be used in fields such as graph data management and application. For example:

[0085] 1) For the problem of large-scale attribute graph data storage in the field of graph data storage, the method of the present invention can be used to establish a KV graph database for large-scale complex attribute graph data, thereby solving the problem of difficulty in storing large-scale graph data in this field.

[0086] 2) For the problem of efficient query of large-scale graph data in the field of graph data query, the method of the present invention can be used to establish a KV graph database for attribute graph data, and then use the established KV graph database to realize efficient query of nodes, relationships, and attributes, thereby solving the problem of low graph query efficiency in this field.

[0087] Based on the same inventive concept, another embodiment of the present invention provides an electronic device (computer, server, smart phone, etc.), which includes a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing each step in the method of the present invention.

[0088] Based on the same inventive concept, another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, CD), which stores a computer program. When the computer program is executed by a computer, it implements the various steps of the method of the present invention.

[0089] The specific embodiments of the present invention disclosed above are intended to facilitate understanding and implementation of the present invention. Those skilled in the art will appreciate that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the embodiments disclosed in this specification; the scope of protection of the present invention shall be determined by the scope defined in the claims.

Claims

1. A method for constructing graph data based on a KV database, characterized in that: The following steps are involved: Map the node data in the property graph model into a KV structure; Map the relational data in the property graph model into a KV structure; Based on the KV structure, the attribute graph data is stored in the KV database to obtain the KV graph database; Mapping the node data in the property graph model to a KV structure includes: 1) The node storage structure includes two KV data tables: node and nodeLabel. Each data table contains two parts: Key and Value. 2) The key portion of the node table is 12 bytes in total. The first 4 bytes store the labelId corresponding to the node label, and the last 8 bytes store the node id. The value portion has a variable length and is the serialized binary data of the node attributes. 3) The key part of the nodeLabel table is 12 bytes in total. The first 8 bytes store the node id, and the last 4 bytes store the labelId corresponding to the node label. The value part is empty. 4) For single-label nodes, each node stores one row in each of the above two tables; 5) For multi-label nodes, store the number of rows of labels in each of the two tables; 6) For unlabeled nodes, use the specified labelId representing no label as its label storage; Mapping the relational data in the property graph model to a KV structure includes: 1) The relational storage structure includes four KV data tables: relation, in, out, and typeRelation. Each data table contains two parts: Key and Value. 2) The relation table directly stores relation data. Its key is the relationId that uniquely represents a relation and is 8 bytes long. Its value is the binary data of the serialized relation attributes. 3) The in table records the incoming direction of the relationship; its key part is 20 bytes in total, the first 8 bytes store the ID of the end node of the directed relationship, the middle 4 bytes store the typeId corresponding to the type of the relationship, and the last 8 bytes store the ID of the starting node of the directed relationship; its value part stores the relationId of the relationship; 4) The out table records the outbound direction of the relationship; its key part is 20 bytes in total, the first 8 bytes store the id of the starting node of the directed relationship, the middle 4 bytes store the typeId corresponding to the type of the relationship, and the last 8 bytes store the id of the end node of the directed relationship; its value part stores the relationId of the relationship; 5) The typeRelation table records the relationship type. The key part is 12 bytes in total. The first 4 bytes store the typeId corresponding to the relationship type, and the last 8 bytes store the relationId of the relationship. 6) All relationships in the above relational storage structure are unidirectional relationships, and bidirectional relationships are considered as two relationships in opposite directions; 7) In the relational storage structure, each relation has only one relation type; for a graph data relation, one row of data is stored in each of the four KV data tables.

2. A data query method based on a KV graph database, characterized in that: Utilizing the KV graph database obtained by the method of claim 1, converting graph queries into KV queries; The graph query includes: 1) Query the node according to the node ID; 2) Query nodes based on their labels; 3) Query the relationship based on the relationship ID; 4) Query relationships based on relationship type; 5) Query the relationship based on the relationship starting node; 6) Query the relationship based on the relationship termination node; The querying of the node according to the node ID includes: ① Use the node ID as the first 8-byte prefix of the nodeLabel table key, use this prefix to perform a Seek operation in the nodeLabel table, and parse out the nodeId and labelId in the key of the first record found by Seek; ② Use the parsed labelId and nodeId to form the key of the node table, perform a seek operation, and deserialize the obtained record value to obtain the query result; The querying of nodes according to node labels includes: ① Use labelId corresponding to label as the first 4 bytes prefix of node table key, and use this prefix to perform Seek operation in node table; ② Execute the Scan operation on the node table until the prefix does not match. During the process, the value of each record is deserialized to obtain the query result. The querying of a relationship based on a relationship ID includes: using the relationship ID as a key of the relation table, performing a seek operation, and deserializing the obtained record value to obtain the query result; The querying of a relationship based on a relationship type includes: ① Use the typeId corresponding to the relationship type as the first 4-byte prefix of the typeRelation table Key, and use this prefix to perform a Seek operation in the typeRelation table; ② Perform a Scan operation on the typeRelation table until the prefix does not match. During the process, record the last 8 bytes of the key as the relation id; ③ Execute the operation of querying the relationship based on the relationship ID to obtain the query result; The querying of a relationship based on a relationship starting node includes: ① Use the starting node ID as the first 8-byte prefix of the out table key and use this prefix to perform a Seek operation in the out table; ② Perform a Scan operation on the out table until the prefix does not match, parsing each row of Value to obtain the relation ID; ③ Execute the operation of querying the relationship based on the relationship ID to obtain the query result; The querying of the relationship according to the relationship termination node includes: ① Use the starting node ID as the first 8-byte prefix of the in table key and use this prefix to perform a Seek operation in the in table; ② Perform a Scan operation on the in table until the prefix does not match, parsing each row value to obtain the relationship ID; ③ Use the relationship ID to perform the operation of querying the relationship and obtain the query result.

3. An electronic device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to claim 1 or 2.

4. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method according to claim 1 or 2 is implemented.

Images