Graph database system and graph query method

Abstract

One or more embodiments of the invention provide a graph database system and a graph query method, nodes and edges in a graph are organized into key-value pairs, and the key-value pairs are stored in an SSTable file; the SSTable file comprises a first data block and a second data block; the edge attribute is stored in the second data block; a key in the first key-value pair comprises a node identifier, and a value comprises a node attribute; the key in the second key value pair comprises a starting point identifier and an ending point identifier of the edge, and the value comprises storage position information of the edge attribute; the first key value pair and the second key value pair with the same node identifier and the same edge starting point identifier are aggregated and stored in the first data block; the method comprises the steps of obtaining a target key; querying in the SSTable file based on a target node identifier in the target key to determine a target first data block for aggregating and storing a key value pair containing the target node identifier; and querying in the target first data block based on the target key to determine a target value corresponding to the target key, and determining a graph query result based on the target value.

Description

Technical Field

[0001] One or more embodiments of the present application relate to the technical field of storage systems, and in particular, to a graph database system and a graph query method. Background Art

[0002] A graph is a data structure that stores and manages data in a graphical structure. In a graph, nodes, edges, and properties are used to store data. This storage method is very suitable for expressing complex relationships between entities. In a graph, nodes represent entities such as people, places, events, etc., and each node can have multiple properties to describe the specific information of the entity. Edges are used to represent the relationships between nodes, such as "know", "belong to", "be located at", etc., and edges can also contain properties to describe the characteristics of the relationships, such as the strength of the relationship, the establishment time, etc. Properties are data fields attached to nodes or edges to store specific information, such as a person's name, age, or the strength of the relationship, the establishment time, etc.

[0003] Graphs are very suitable for expressing and processing data with complex relationships, and therefore usually can provide relatively high query performance. Correspondingly, for files used to store graph data such as nodes, edges, and properties, how to design the format of such files so that such files are graph-aware, thereby being able to retain the relatively high query performance of the graph, and thus improving the efficiency of graph query in such files has become a highly concerned issue. Summary of the Invention

[0004] One or more embodiments of the present application provide the following technical solutions:

[0005] The present application provides a graph database system for storing graph data, where the graph data includes nodes and edges; wherein:

[0006] An SSTable file for storing sorted string tables is stored in the storage space of the graph database system; the SSTable file includes at least one first data block for storing key-value pairs corresponding to the nodes and the edges, and a second data block for storing edge attributes of the edges;

[0007] The key-value pairs include first key-value pairs and second key-value pairs; the key in the first key-value pair includes the node identifier of the node, and the value includes the node attributes of the node; the key in the second key-value pair includes the start identifier and end identifier of the edge, and the value includes the storage location information of the edge attributes in the SSTable file;

[0008] In the first data block, various target first key-value pairs are aggregated and stored, as well as target second key-value pairs whose starting point identifier is the same as the node identifier in the target first key-value pairs.

[0009] This application also provides a graph query method. Among them, the nodes and edges in the graph are stored in a sorted string table SSTable file. The SSTable file includes at least one first data block for storing key-value pairs corresponding to the nodes and the edges, and a second data block for storing the edge attributes of the edges. The nodes and the edges are respectively organized into first key-value pairs and second key-value pairs and stored in the first data block. The key in the first key-value pair includes the node identifier of the node, and the value includes the node attributes of the node. The key in the second key-value pair includes the starting point identifier and the ending point identifier of the edge, and the value includes the storage location information of the edge attribute in the SSTable file. Various target first key-value pairs, and target second key-value pairs whose starting point identifier is the same as the node identifier in the target first key-value pairs, are aggregated and stored in the first data block.

[0010] The method includes:

[0011] Obtain a target key for querying the graph.

[0012] Query in the SSTable file based on the target node identifier in the target key to determine a target first data block for aggregating and storing the first key-value pairs and second key-value pairs containing the target node identifier.

[0013] Query in the target first data block based on the target key to determine a target value corresponding to the target key.

[0014] If the target value is the node attribute, determine the node attribute as the graph query result.

[0015] If the target value is the storage location information, determine the edge attribute corresponding to the target key in the second data block included in the SSTable file according to the storage location information, and determine the edge attribute corresponding to the target key as the graph query result.

[0016] This application also provides an electronic device, including:

[0017] A processor;

[0018] A memory for storing instructions executable by the processor;

[0019] Wherein, the processor realizes the steps of the method as described in any one of the above by running the executable instructions.

[0020] The present application also provides a computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method described in any one of the above are implemented.

[0021] In the above technical solution, the nodes and edges in the graph can be stored in an SSTable file, and the SSTable file can include at least one first data block for storing key-value pairs corresponding to the nodes and edges, and a second data block for storing the edge attributes of the edges; the nodes and edges can be respectively organized into first key-value pairs and second key-value pairs and stored in the first data block. Among them, the key in the first key-value pair corresponding to a node can include the node identifier of the node, and the value can include the node attributes of the node. The key in the second key-value pair corresponding to an edge can include the start point identifier and the end point identifier of the edge, and the value can include the storage location information of the edge attributes of the edge in the SSTable file; each target first key-value pair and the target second key-value pair whose start point identifier is the same as the node identifier in the target first key-value pair can be aggregated and stored in the first data block. Accordingly, when the target key for querying the graph is obtained, the target first data block for aggregating and storing the first key-value pair and the second key-value pair containing the target node identifier can be first queried in the SSTable file based on the target node identifier in the target key, and then the target value corresponding to the target key can be queried in the target first data block based on the target key; if the target value is node attributes, the node attributes can be determined as the graph query result; if the target value is storage location information, the edge attributes corresponding to the target key can be determined in the second data block included in the SSTable file according to the storage location information, and the edge attributes corresponding to the target key can be determined as the graph query result.

[0022] By adopting the above method, the optimization of the key-value storage system is realized by using the characteristics of graph topology and graph query, specifically including adjusting and improving aspects such as the SSTable file format and indexing method in the key-value storage system, so that the SSTable file for storing graph data in the key-value storage system is graph-aware, thereby reducing the complexity of the graph query operation of the graph storage engine based on the key-value storage system in the key-value storage system and improving the graph query efficiency, so as to ensure that the graph storage engine has relatively high graph query performance. Description of the Drawings

[0023] The drawings required for the description of the exemplary embodiments will be described below, where:

[0024] Figure 1 is a schematic diagram of a key-value storage system shown in an exemplary embodiment of the present application.

[0025] Figure 2It is a schematic diagram of a graph storage file shown in an exemplary embodiment of the present application.

[0026] Figure 3 It is a flowchart of a graph query method shown in an exemplary embodiment of the present application.

[0027] Figure 4 It is a schematic structural diagram of a device shown in an exemplary embodiment of the present application.

[0028] Figure 5 It is a block diagram of a graph query device shown in an exemplary embodiment of the present application. Detailed implementation manners

[0029] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all the implementation manners consistent with one or more embodiments of the present application. On the contrary, they are only examples consistent with some aspects of one or more embodiments of the present application.

[0030] It should be noted that in other embodiments, the steps of the corresponding methods are not necessarily executed in the order shown and described in the present application. In some other embodiments, the steps included in the method may be more or less than those described in the present application. In addition, a single step described in the present application may be decomposed into multiple steps for description in other embodiments; and multiple steps described in the present application may also be combined into a single step for description in other embodiments.

[0031] A storage system generally refers to the combination of hardware and software in the entire computer system for storing data. It includes not only physical storage media such as hard disk drives (HDDs), solid state drives (SSDs), tape libraries, etc., but also file systems for managing these media, storage services at the operating system level, etc. In a database environment, the storage system is responsible for providing persistent storage space, ensuring that data can be safely and reliably stored on physical media and can be accessed efficiently.

[0032] A storage engine is a component in a database management system (DBMS) responsible for managing how data is stored, retrieved, and updated. It determines the physical storage structure of data on disk and supported functions such as transaction processing, concurrency control, etc.

[0033] The storage engine relies on the underlying storage system to implement actual data reading and writing operations. Without the physical storage capabilities provided by the storage system, the storage engine cannot complete the persistent storage of data. Moreover, the storage engine optimizes performance by understanding or controlling the characteristics of the storage system; for example, some storage engines may adjust their index structures or caching policies according to the characteristics of the storage medium to improve data access efficiency.

[0034] The storage system can be regarded as the underlying storage infrastructure of the storage engine. The storage system is responsible for providing reliable storage capabilities to ensure the security and availability of data, while the storage engine implements more advanced data management and access functions on this basis.

[0035] Graph Storage refers to a data storage solution specifically designed for graph data structures, aiming to efficiently store and manage a large number of graph data such as interconnected nodes, edges, and attributes.

[0036] Graph storage systems usually optimize operations on nodes, edges, and their attributes to support fast querying, traversing, and other graph-related computational tasks. In a graph storage system, graph data is usually stored in the form of files, for example, persistently stored on disk in the form of files; these files constitute the basic storage units of graph data in the graph storage system.

[0037] The graph storage engine built on top of the graph storage system, that is, the graph storage engine with the graph storage system as the underlying storage, is responsible for the specific data access and storage logic corresponding to the graph storage system, including storage structure (such as file format), indexing method, transaction processing, concurrency control, etc.

[0038] In practical applications, graph storage systems can generally be divided into two categories: Native Graph Storage systems and Non-Native Graph Storage systems.

[0039] A Native Graph Storage system is a storage system specifically designed for graphs, which directly organizes data in the form of nodes, edges, and attributes. The core features of this storage model include: nodes and edges are directly represented as independent objects at the physical storage level, with their own identifiers (IDs), sets of attributes, and references to other related objects, rather than being indirectly represented through relational tables or key-value pairs; nodes usually contain a list of references to all their adjacent nodes (i.e., the target nodes pointed to by the edges connected to them), and edges can also directly link to the two nodes they connect, enabling fast graph traversal operations, that is, accessing or processing all nodes and edges in the graph according to a certain strategy, such as jumping from one node to another along the edges; providing consistency and persistence guarantees for graph data.

[0040] For native graph storage systems, they are usually optimized by leveraging the characteristics of graph topology and graph queries. Among them, graph topology refers to the connection method and structural characteristics between nodes and edges in a graph, and graph query refers to the process of querying graph data in a storage system. That is to say, according to the characteristics of the graph data structure and common graph query patterns, aspects such as the file format and indexing method corresponding to the storage system are adjusted and improved, making the files used to store graph data in the native graph storage system graph-aware. In this case, for a graph storage engine built on a native graph storage system, it can reduce the complexity of graph query operations in the native graph storage system, improve graph query efficiency, and thus ensure that the graph storage engine has relatively high graph query performance.

[0041] Non-graph native storage systems usually convert graphs into other forms of data structures for storage, such as tables in relational databases or key-value pairs in key-value storage systems.

[0042] Non-graph native storage systems usually do not have the ability to directly map nodes and edges, nor do they specifically optimize graph traversal operations. That is to say, the files used to store graph data in non-graph native storage systems are usually not graph-aware, resulting in poor query performance of the graph storage engine built on the non-graph native storage system, that is, the complexity of graph query operations in the non-graph native storage system for the graph storage engine is high, and the graph query efficiency is low.

[0043] A key-value store is a simple and efficient database model that uses key-value pairs to store data. Each record in a key-value store has a unique key and a value associated with that key. The key is a unique identifier used to locate a specific value stored in the database. Usually, the key is in the form of a string, but it can also be other types of data. The value is the actual data item associated with the key. It can be any form of data, such as a string, a number, a binary object, etc. In some cases, the value can be a complex data structure, such as a JSON document or a serialized object.

[0044] The design of a key-value store is very simple. It usually only provides basic operations such as put (insert), get / exact lookup, range lookup or seek, and delete, without complex schema definitions or relationship constraints, which makes them easy to implement, deploy, and maintain. Due to the simple data structure, key-value stores usually have high read and write performance, capable of providing extremely high read and write throughput and low-latency response times.

[0045] Key-value storage systems have excellent scalability and flexibility. Specifically, most key-value storage systems are designed as distributed systems, supporting horizontal scaling to adapt to larger data volumes and higher throughput requirements. Additionally, since the format and content of the values in key-value pairs are usually unrestricted, one can freely choose how to organize the data to form a storage data structure according to actual needs. Key-value storage systems can also provide different consistency guarantees, ranging from strong consistency to eventual consistency, depending on the specific application scenarios and technical implementations.

[0046] Due to the good scalability and flexibility of key-value storage systems, in practical applications, graph data can be converted into a key-value pair data structure and stored in a key-value storage system, thus transforming the key-value storage system into a graph storage system. Building a graph storage engine based on a key-value storage system can utilize the existing complete key-value storage system ecosystem and infrastructure.

[0047] However, a graph storage system built based on a key-value storage system is a non-native graph storage system. This means that the files used to store graph data in such storage systems are actually not graph-aware. Therefore, the graph query performance of a graph storage engine built based on a key-value storage system will be greatly affected.

[0048] In this application, the key-value storage system is optimized by leveraging the characteristics of graph topology and graph queries. A graph-aware SSTable file format is proposed, which is obtained by adjusting and improving the original SSTable file format in the key-value storage system. Accordingly, adjustments and improvements are also made in aspects such as the indexing method, enabling the graph storage engine built based on the optimized key-value storage system to have relatively high query performance.

[0049] Specifically, one or more embodiments of the present application provide a technical solution for implementing graph queries. In this technical solution, the nodes and edges in the graph can be stored in an SSTable file, which can include at least one first data block for storing key-value pairs corresponding to the nodes and edges, and a second data block for storing the edge attributes of the edges; the nodes and edges can be respectively organized into first key-value pairs and second key-value pairs and stored in the first data block. Among them, the key in the first key-value pair corresponding to a node can include the node identifier of the node, and the value can include the node attributes of the node. The key in the second key-value pair corresponding to an edge can include the start point identifier and the end point identifier of the edge, and the value can include the storage location information of the edge attributes of the edge in the SSTable file; each target first key-value pair, and the target second key-value pair whose start point identifier is the same as the node identifier in the target first key-value pair, can be aggregated and stored in the first data block. Accordingly, when obtaining the target key for querying the graph, first, based on the target node identifier in the target key, query in the SSTable file to determine the target first data block for aggregating and storing the first key-value pair and the second key-value pair containing the target node identifier, and then, based on the target key, query in the target first data block to determine the target value corresponding to the target key; if the target value is node attributes, the node attributes can be determined as the graph query result; if the target value is storage location information, the edge attributes corresponding to the target key can be determined in the second data block included in the SSTable file according to the storage location information, and the edge attributes corresponding to the target key can be determined as the graph query result.

[0050] By adopting the above method, the optimization of the key-value storage system is realized by utilizing the characteristics of graph topology and graph queries, specifically including adjusting and improving aspects such as the SSTable file format and indexing method in the key-value storage system, so that the SSTable file for storing graph data in the key-value storage system is graph-aware, thereby reducing the complexity of graph query operations of the graph storage engine based on the key-value storage system in the key-value storage system, improving the graph query efficiency, and thus ensuring that the graph storage engine has relatively high graph query performance.

[0051] In the present application, a graph database system can refer to a system for storing, managing, and querying graph data structures.

[0052] In the storage space of the above graph database system, SSTable (Sorted String Table) files can be stored, and these SSTable files are used to store nodes and edges in the graph. Among them, the SSTable file can include at least one data block (which can be called the first data block) for storing key-value pairs corresponding to nodes and edges, and a data block (which can be called the second data block) for storing edge attributes of edges.

[0053] Nodes in the above graph can be organized into first key-value pairs corresponding to the nodes, and edges in the graph can be organized into second key-value pairs corresponding to the edges; the first key-value pairs and the second key-value pairs can be stored in the above first data block. Among them, the key in the first key-value pair can include the node identifier of the node, and the value can include the node attributes of the node; the key in the second key-value pair can include the start point identifier and the end point identifier of the edge, and the value can include the storage location information of the edge attributes of the edge in the above SSTable file.

[0054] Each first key-value pair can be used as a target first key-value pair in turn, and the second key-value pair with the start point identifier the same as the node identifier in the target first key-value pair can be used as the target second key-value pair; the target first key-value pair and the target second key-value pair can be aggregated and stored in the above first data block.

[0055] It should be noted that the above graph database system can specifically include a key-value storage system for storing graph data in the data structure of key-value pairs, that is, the key-value storage system provides the storage space of the graph database system. The graph database system can also include a storage engine for converting graph data into the data structure of key-value pairs and making it meet specific storage requirements. Among them, the storage engine can be a graph storage engine built based on the key-value storage system, that is, a graph storage engine with the key-value storage system as the underlying storage.

[0056] The following takes the key-value storage system and the graph storage engine built based on the key-value storage system as examples for detailed description.

[0057] Please refer to Figure 1 , Figure 1 which is a schematic diagram of a key-value storage system shown in an exemplary embodiment of the present application.

[0058] As Figure 1 shown, the graph storage engine can use the data structure of LSM-Tree (Log Structured Merge Tree) to organize data in the key-value storage system.

[0059] The LSM-Tree is a data structure optimized for handling a large number of write operations. In its design, it is more inclined to append new data to the end of a file rather than update it in-place. The LSM-Tree first appends all new write operations to one or more ordered data structures in memory and then, at an appropriate time, writes this data to persistent storage on disk in batches and sequentially. This data structure can reduce random write operations on disk, thereby reducing the overhead of random writes, and can maintain query efficiency through a background merge process.

[0060] The LSM-Tree uses a multi-layer structure to manage data. These layers start with the MemTable in memory and then gradually migrate to multiple levels on the hard disk. Each level contains one or more SSTable files. As the number of levels increases, the amount of data in each level also gradually increases, and the size of each level is usually several times that of the previous level.

[0061] Specifically, when data is first written, it is first stored in a memory data structure called the MemTable. The MemTable is in memory and is a collection of sorted key-value pairs. All newly inserted data will first accumulate here. Since it is memory-based operation, the write speed is very fast, and lookup operations can be performed quickly.

[0062] After the current MemTable reaches a certain size, it will be frozen and converted to an immutable state, that is, an Immutable MemTable, waiting to be written to disk as an immutable file, that is, an SSTable file. This method reduces frequent disk write operations, thereby improving write performance.

[0063] The SSTable file will be written to disk to form the first level (L0). The SSTable is a read-only file format that contains key-value pairs arranged in order. To speed up lookups, each SSTable can have associated metadata information (e.g., Bloom Filter and index), which can be used to quickly locate the block that may contain the target key.

[0064] Over time, more and more SSTable files will accumulate in the L0 level. To maintain the efficient operation of the system and good query performance, the system will periodically perform a compaction operation to merge the data in the L0 level to the next level (L1), while removing duplicate or expired data items. The compaction process will continue downward until the data reaches the bottom layer. The compaction operation is usually asynchronous and does not block the write path.

[0065] It should be noted that Layer L0 is the layer closest to the memory and usually contains SSTable files that have just been flushed from the MemTable in the memory to the disk. The SSTable files in Layer L0 may not be sorted by key because they are generated independently from different MemTables. When the number of SSTable files in Layer L0 exceeds a certain threshold, Minor Compaction will be triggered to merge and sort these files, and then migrate the results to Layer L1.

[0066] Starting from Layer L1, the SSTable files in each layer are sorted by key. The number and size of SSTable files in each layer usually increase layer by layer. For example, the SSTable files in Layer L1 may be larger than those in Layer L0, and the SSTable files in Layer L2 are larger than those in Layer L1. When the number or total size of SSTable files in a certain layer exceeds the preset threshold, Major Compaction will be triggered to merge a part of the files in this layer with the files in the next layer to generate larger SSTable files, and then migrate the results to the next layer.

[0067] When the graph storage engine queries a key-value storage system using an LSM-Tree, it needs to start from the MemTable in the memory and then sequentially check the SSTable files at all levels. Since the SSTable files in each layer are sorted by key, efficient algorithms such as binary search can be used to quickly locate the data.

[0068] Specifically, when performing a query, it will first look for the data in the MemTable in the memory.

[0069] If the required data cannot be found in the MemTable, it will continue to look for it in the SSTable files at all levels that already exist on the disk in sequence (if there are Immutable MemTables, first look for it in the Immutable MemTables and then in the SSTable files). The impact of compaction will also be considered during the query process, that is, all relevant SSTable files need to be traversed to ensure that the latest data version is obtained.

[0070] Before accessing an SSTable file, its associated Bloom Filter can be used to determine whether the SSTable file is likely to contain the target key. If the Bloom Filter indicates that the target key does not exist in the SSTable file, the SSTable file can be skipped, thus avoiding unnecessary disk I / O operations.

[0071] Finally, the query process combines the results from the MemTable, Immutable MemTables, and individual SSTables, and returns the latest and unique key-value pairs.

[0072] For example, assume that in a key-value storage system using an LSM-Tree, there is one MemTable, one SSTable file in level L0, and two SSTable files in level L1 (in the order of SSTable file 1 and SSTable file 2); the key space ranges from key_0 to key_9. Then, when querying for the key key_5, first look in the MemTable. Assume that key_5 is not found; continue to look in the SSTable file in level L0. Assume that key_5 is found in the SSTable file in level L0, and its corresponding value is value_old; continue to look in SSTable file 1 in level L1. Assume that key_5 is not found; continue to look in SSTable file 2 in level L1. Assume that key_5 is also found in SSTable file 2 in level L1, and its corresponding value is value_new, and it is determined to be the updated value according to the timestamp or other mechanisms. In this case, the latest value value_new corresponding to key_5 is finally returned.

[0073] As can be seen from the above, the key-value pairs stored in the original SSTable file are arranged in the order of keys. Therefore, when using the original SSTable file to store graph data, the graph data needs to be converted into key-value pair form data for storage. That is, the original SSTable file is graph-agnostic.

[0074] In a Figure 1 key-value storage system as shown, the original graph-agnostic SSTable file can be replaced with an adjusted and improved graph-aware SSTable file (which can be called a Graph-Aware SSTable file).

[0075] Please refer to Figure 2 , Figure 2 which is a schematic diagram of a graph storage file shown in an exemplary embodiment of the present application.

[0076] As Figure 2 shown, the above graph storage file (i.e., the file for storing graph data) can be an adjusted and improved graph-aware Graph-Aware SSTable file based on the original SSTable file.

[0077] In the original SSTable file, it usually contains a Data Block and an Index Block. The Data Block is the most basic data storage unit in the SSTable file, which is used to store the data in the form of actual key-value pairs, and the stored key-value pairs are arranged in the order of keys. The Index Block is used to store the meta-information of each Data Block, such as the key range (the range of the minimum key and the maximum key) of the DataBlock and the offset of the Data Block in the SSTable file (which can indicate the starting position). It can accelerate the search process and enable quick positioning to the Data Block where the target key is located.

[0078] In addition to the Data Block and the Index Block, a complete SSTable file may also contain other auxiliary components, such as a Footer, a Bloom Filter, and Metadata. The Footer is located at the end of the file and contains the offsets of the Index Block and other metadata information (such as the Bloom Filter) in the SSTable file, which is used to quickly find the IndexBlock. The Bloom Filter is a probabilistic data structure used to quickly determine whether a certain key exists in the SSTable file, reducing unnecessary disk I / O operations. Metadata contains other auxiliary information, such as the size of the file, the creation time, the version number, etc.

[0079] For a Graph-Aware SSTable file as Figure 2 shown, first of all, it should be noted that the Graph-Aware SSTable file stores key-value pairs (which can be respectively called the first key-value pairs and the second key-value pairs) organized by the nodes and edges in the graph. Specifically, the nodes in the graph can be organized into the first key-value pairs corresponding to the nodes, and the edges in the graph can be organized into the second key-value pairs corresponding to the edges.

[0080] In practical applications, in order to facilitate the search for specific key-value pairs in the above-mentioned first data block, all the key-value pairs stored in the first data block can be arranged in the order of keys (usually in Lexicographical Order, that is, the dictionary order). When there are multiple first data blocks, the key-value pairs stored in two adjacent first data blocks are also arranged in the order of keys.

[0081] Taking a node as an example, the key in the first key-value pair organized by this node can include the node identifier of the node, and the value can include the node attributes of the node (specifically, it can be the encoded node attribute data).

[0082] To accelerate the filtering query of edges and improve the efficiency of graph query and graph traversal, edge attributes can be separately stored in a key-value storage system. In this case, the above-mentioned Graph-Aware SSTable file can be used to store the data blocks (which can be called the second data blocks) of the edge attributes of the edges in the graph. Among them, the data block is similar to the DataBlock in the original SSTable file; there can be one and only one second data block, and all edge attributes are stored in this second data block; or similar to the first data block, the amount of edge attribute data stored in each second data block does not exceed a preset threshold.

[0083] Specifically, taking an edge as an example, the edge attribute of this edge (specifically, it can be the encoded edge attribute data) can be stored in the above-mentioned second data block; the key in the second key-value pair organized by this edge can include the node identifier of the SourceNode (source node / starting node) of this edge (abbreviated as the starting point identifier) and the node identifier of the Destination Node (destination node / ending node) (abbreviated as the ending point identifier), and the value includes the storage location information of the edge attribute of this edge in the Graph-Aware SSTable file.

[0084] When the key-value pair composed of the key in the above-mentioned second key-value pair and the above-mentioned edge attribute is stored in the above-mentioned second data block, it means that the key can be used to query in the second data block. Therefore, the storage location information of the edge attribute in the above-mentioned Graph-Aware SSTable file is specifically the storage location information of the second data block where the edge attribute is located in this Graph-Aware SSTable file.

[0085] When only the above-mentioned edge attribute is stored in the above-mentioned second data block to save storage resources, the storage location information of the edge attribute in the above-mentioned Graph-Aware SSTable file is the storage location information of the edge attribute itself in this Graph-Aware SSTable file, that is, the storage location information of the edge attribute in the second data block.

[0086] In practical applications, to facilitate the search for specific edge attributes in the above-mentioned second data block, all edge attributes stored in the second data block can be arranged in the order of the keys in the corresponding second key-value pairs.

[0087] In practical applications, graphs can be divided into directed graphs (Directed Graph / Digraph) and undirected graphs (Undirected Graph). In a directed graph, each edge has a specific direction, pointing from one node to another. Therefore, each edge in a directed graph can be organized into a second key-value pair. In an undirected graph, the edges have no direction, which means that the two nodes connected by each edge are reachable in both directions. Therefore, for an undirected graph, each undirected edge can be regarded as composed of two directed edges. For example, assume that in an undirected graph, an edge connects node A and node B. Then this edge can be regarded as an edge pointing from node A to node B and an edge pointing from node B to node A. Thus, this edge can be organized into two second key-value pairs. In one second key-value pair, the key contains the node identifier of the starting point with the starting point identifier being node A and the ending point identifier being node B. In the other second key-value pair, the key contains the node identifier of the starting point with the starting point identifier being node B and the ending point identifier being node A.

[0088] The above first key-value pair can be as shown in Table 1 below:

[0089] Table 1

[0090]

[0091] In some embodiments, the key in the first key-value pair organized by a node, in addition to including the node identifier of this node, may also include the node type of this node. The specific content of the key and value in the first key-value pair can be set according to actual needs, and this application does not impose special restrictions on this.

[0092] The above second key-value pair can be as shown in Table 2 below:

[0093] Table 2

[0094]

[0095] In some embodiments, the key in the second key-value pair organized by an edge, in addition to including the starting point identifier and ending point identifier of this edge, may also include the node type of the Source Node of this edge (abbreviated as the starting point type), and / or the creation timestamp of this edge. The specific content of the key and value in the second key-value pair can be set according to actual needs, and this application does not impose special restrictions on this.

[0096] In the above Graph-Aware SSTable file, it may contain at least one data block (which can be called the first data block) for storing the above first key-value pair and the above second key-value pair. Among them, the data block is similar to the Data Block in the original SSTable file.

[0097] It should be noted that all the first key-value pairs in all the first key-value pairs (which can be called the target first key-value pairs), as well as the second key-value pairs whose starting point identifiers are the same as the node identifiers in the target first key-value pairs (which can be called the target second key-value pairs), can be aggregated and stored in the above-mentioned first data block. For example, assuming that the node identifier included in the key of the target first key-value pair is node identifier A, the starting point identifier included in the key of the target second key-value pair should also be node identifier A, and these target first key-value pairs and target second key-value pairs can be aggregated and stored in the first data block.

[0098] The fact that the node identifier in the above-mentioned target first key-value pair is the same as the starting point identifier in the above-mentioned target second key-value pair means that the edge corresponding to the target second key-value pair starts from the node corresponding to the target first key-value pair. Therefore, by aggregating and storing the target first key-value pair and the target second key-value pair, the common access to the node and its connected edges can be achieved, thereby accelerating the query of the node and its connection points and improving the efficiency of graph query and graph traversal.

[0099] Correspondingly, the above-mentioned first key-value pairs and the above-mentioned second key-value pairs in the above-mentioned Graph-Aware SSTable file can specifically be arranged in the order of the minimum keys in each group of aggregated and stored first key-value pairs and second key-value pairs. For example, the above-mentioned target first key-value pair and the above-mentioned target second key-value pair are a group of aggregated and stored first key-value pairs and second key-value pairs. Since the data content included in the key of the first key-value pair is less than the data content included in the key of the second key-value pair, based on the lexicographical order, the minimum key in this group of aggregated and stored target first key-value pairs and target second key-value pairs is usually the key of the target first key-value pair.

[0100] Aggregating and storing the above-mentioned target first key-value pair and the above-mentioned target second key-value pair in the above-mentioned first data block can specifically be aggregating and storing the target first key-value pair and the target second key-value pair in the same first data block.

[0101] For example, assume that the node identifier included in the key of the first key-value pair 1 is node identifier A, the node identifier included in the key of the first key-value pair 2 is node identifier B, the start identifiers included in the keys of the second key-value pairs 1-1 and 1-2 are both node identifier A, and the start identifiers included in the keys of the second key-value pairs 2-1, 2-2, 2-3, 2-4, 2-5, and 2-6 are all node identifier B. Also assume that the lexicographical order of the keys is: the key of the first key-value pair 1 < the key of the second key-value pair 1-1 < the key of the second key-value pair 1-2, the key of the first key-value pair 2 < the key of the second key-value pair 2-1 < the key of the second key-value pair 2-2 < the key of the second key-value pair 2-3 < the key of the second key-value pair 2-4 < the key of the second key-value pair 2-5 < the key of the second key-value pair 2-6, and the key of the first key-value pair 1 < the key of the first key-value pair 2. Then, (the first key-value pair 1, the second key-value pair 1-1, the second key-value pair 1-2) can be stored in one first data block, and (the first key-value pair 2, the second key-value pair 2-1, the second key-value pair 2-2, the second key-value pair 2-3, the second key-value pair 2-4, the second key-value pair 2-5, the second key-value pair 2-6) can be stored in another first data block.

[0102] Similar to the Data Block in the original SSTable file, the size of each first data block may have a certain limit to improve the query efficiency of the first data block; when this limit is reached, new key-value pairs will be written into the next first data block. That is, the data volume of the first key-value pairs and the second key-value pairs stored in each first data block is not greater than a preset threshold. Here, the data volume can refer to the number of key-value pairs or the data size of the key-value pairs, and this application does not have special restrictions on this. Correspondingly, aggregating and storing the above-mentioned target first key-value pairs and the above-mentioned target second key-value pairs in the first data block can specifically be aggregating and storing the target first key-value pairs and the target second key-value pairs in one or consecutive multiple first data blocks.

[0103] The first key-value pair and the second key-value pair containing different node identifiers can be stored in different first data blocks. For example, assume that the node identifier contained in the key of the first key-value pair 1 is node identifier A, the node identifier contained in the key of the first key-value pair 2 is node identifier B, the starting point identifiers contained in the keys of the second key-value pair 1-1 and the second key-value pair 1-2 are both node identifier A, the starting point identifiers contained in the keys of the second key-value pair 2-1, the second key-value pair 2-2, the second key-value pair 2-3, the second key-value pair 2-4, the second key-value pair 2-5, and the second key-value pair 2-6 are all node identifier B, and assume that the number of key-value pairs stored in a first data block is not greater than 5, and assume that the lexicographical order of the keys is the key of the first key-value pair 1 < the key of the second key-value pair 1-1 < the key of the second key-value pair 1-2, the key of the first key-value pair 2 < the key of the second key-value pair 2-1 < the key of the second key-value pair 2-2 < the key of the second key-value pair 2-3 < the key of the second key-value pair 2-4 < the key of the second key-value pair 2-5 < the key of the second key-value pair 2-6, and the key of the first key-value pair 1 < the key of the first key-value pair 2. Then, (the first key-value pair 1, the second key-value pair 1-1, the second key-value pair 1-2) can be stored in one first data block, (the first key-value pair 2, the second key-value pair 2-1, the second key-value pair 2-2, the second key-value pair 2-3, the second key-value pair 2-4) can be stored in another first data block, and (the second key-value pair 2-5, the second key-value pair 2-6) can be stored in yet another first data block.

[0104] The first key-value pair and the second key-value pair containing different node identifiers can also be stored in the same first data block. Continuing with the above example, (the first key-value pair 1, the second key-value pair 1-1, the second key-value pair 1-2, the first key-value pair 2, the second key-value pair 2-1) can be stored in one first data block, and (the second key-value pair 2-2, the second key-value pair 2-3, the second key-value pair 2-4, the second key-value pair 2-5, the second key-value pair 2-6) can be stored in another first data block.

[0105] In addition, in the above Graph-Aware SSTable file, an index (which can be called a first-level index) created based on the node identifiers in the above first key-value pairs can also be included to facilitate determining the first key-value pairs and second key-value pairs for aggregated storage that contain specific node identifiers.

[0106] It should be noted that the above first-level index can be used to indicate the first data blocks where the first key-value pairs and second key-value pairs containing each node identifier are located. Among them, the first key-value pairs and second key-value pairs containing the same node identifier are the first key-value pairs and second key-value pairs for aggregated storage.

[0107] Specifically, the above first-level index may include: the correspondence between each node identifier and the storage location information of the first data block where the first key-value pair and the second key-value pair containing the node identifier are located in the above Graph-Aware SSTable file. Among them, the storage location information of the first data block in the Graph-Aware SSTable file may include the offset of the first data block in the Graph-Aware SSTable file (which can indicate the starting position); in addition, the storage location information may also include other auxiliary information, such as the size of the first data block, so as to facilitate determining the termination position of the first data block.

[0108] For example, assume that the first data block 1 stores (the first key-value pair 1, the second key-value pair 1-1, the second key-value pair 1-2, the first key-value pair 2, the second key-value pair 2-1), and the first data block 2 stores (the second key-value pair 2-2, the second key-value pair 2-3, the second key-value pair 2-4, the second key-value pair 2-5, the second key-value pair 2-6). Then, since the first data block where the first key-value pair 1, the second key-value pair 1-1, and the second key-value pair 1-2 containing the node identifier A are located is the first data block 1, and the first data block where the first key-value pair 2, the second key-value pair 2-1, the second key-value pair 2-2, the second key-value pair 2-3, the second key-value pair 2-4, the second key-value pair 2-5, and the second key-value pair 2-6 containing the node identifier B are located includes the first data block 1 and the first data block 2, the first-level index created based on the node identifiers in the first key-value pair 1 and the first key-value pair 2 can be as shown in Table 3 below:

[0109] Table 3

[0110]

[0111] The storage location information of the first data block 1 in the above Table 3 is the storage location information of the first data block 1 in the above Graph-Aware SSTable file, and the storage location information of the first data block 2 is the storage location information of the first data block 2 in the Graph-Aware SSTable file.

[0112] Furthermore, the above first-level index may include: the correspondence between each node identifier and the storage location information of the first data block where the first key-value pair and the second key-value pair containing the node identifier are located in the above Graph-Aware SSTable file. Correspondingly, the correspondence between the node identifier and the storage location information in the first-level index can be arranged in the order of the first key-value pairs containing each node identifier in the first data block. Specifically, when the key-value pairs stored in the first data block are arranged in the order of the keys, the correspondence between the node identifier and the storage location information in the first-level index can be arranged in the order of the smallest key among the first key-value pairs and the second key-value pairs containing each node identifier.

[0113] For example, assume that the node identifier included in the key of the first key-value pair 1 is node identifier A, the node identifier included in the key of the first key-value pair 2 is node identifier B, and the node identifier included in the key of the first key-value pair 3 is node identifier C. Also assume that the key of the first key-value pair 1 < the key of the first key-value pair 2 < the key of the first key-value pair 3, and assume that the first key-value pair and the second key-value pair containing node identifier A are aggregated and stored in the first data block 1, the first key-value pair and the second key-value pair containing node identifier B are aggregated and stored in the first data block 1 and the first data block 2, and the first key-value pair and the second key-value pair containing node identifier C are aggregated and stored in the first data block 3. Then, in the above first-level index, the arrangement order of the correspondence between the node identifier and the storage location information can be: the correspondence between node identifier A and the storage location information of the first data block 1, the correspondence between node identifier B and the storage location information of the first data block 1, the correspondence between node identifier C and the storage location information of the first data block 3. Since the first data block where the first key-value pair and the second key-value pair containing node identifier A are located is the first data block 1, the first data block where the first key-value pair and the second key-value pair containing node identifier C are located is also the first data block 1, and the first data block where the first key-value pair and the second key-value pair containing node identifier C are located is the first data block 3, the first-level index created based on the node identifiers in the first key-value pair 1, the first key-value pair 2, and the first key-value pair 3 can be as shown in Table 4 below:

[0114] Table 4

[0115]

[0116] In the above case, the first data block in the above Graph-Aware SSTable file for aggregating and storing the first key-value pair and the second key-value pair containing the same node identifier can be determined according to the arrangement order of the correspondence between the node identifiers and the storage location information in the above first index. Specifically, for a node identifier, according to the corresponding relationship containing the node identifier and the next corresponding relationship of the corresponding relationship, the first data blocks in the above Graph-Aware SSTable file within the range starting from the first data block indicated by the storage location information contained in the corresponding relationship and ending at the second data block indicated by the storage location information contained in the next corresponding relationship of the corresponding relationship can be determined as the first data block in the above Graph-Aware SSTable file for aggregating and storing the first key-value pair and the second key-value pair containing the node identifier. Continuing with the above example, since the next corresponding relationship of the corresponding relationship between node identifier B and the storage location information of the first data block 1 is the corresponding relationship between node identifier C and the storage location information of the first data block 3, the first data blocks within the range from the first data block 1 to the first data block 3 can be determined as the first data block for aggregating and storing the first key-value pair and the second key-value pair containing node identifier B, that is, the first key-value pair and the second key-value pair containing node identifier B must be stored in the first data block 1 and the second data block 2, and the first key-value pair and the second key-value pair containing node identifier B may be stored in the first data block 3.

[0117] To reduce the lookup time complexity of the above first-level index, thereby accelerating the query of nodes and their connection points and improving the efficiency of graph query and graph traversal, the first-level index can be a hash index. That is, specifically, the first-level index can include: the correspondence between the hash value of each node identifier and the storage location information of the first data block where the first key-value pair and the second key-value pair containing the node identifier are located in the above Graph-Aware SSTable file.

[0118] Alternatively, the above first-level index can also follow the index block in the original SSTable file. At this time, specifically, the first-level index can include: the correspondence between the range of node identifiers in the first key-value pairs contained in each first data block and the storage location information of the first data block in the above Graph-Aware SSTable file.

[0119] It should be noted that for the graph composed of nodes and edges corresponding to the key-value pairs stored in the above-mentioned Graph-Aware SSTable file, if the graph is a sparse graph (i.e., the number of edges is relatively small), it means that a first data block in the Graph-Aware SSTable file may contain multiple first key-value pairs. Therefore, the first-level index in the Graph-Aware SSTable file can be the index block in the SSTable file; if the graph is a dense graph (i.e., the number of edges is relatively large, for example, the number of edges is close to the square of the number of nodes), it means that the first key-value pairs in the Graph-Aware SSTable file may be scattered in discontinuous first data blocks. Therefore, the first-level index in the Graph-Aware SSTable file can be a hash index.

[0120] In practical applications, Cuckoo Hashing can be used to create the above-mentioned first-level index based on the node identifiers in the above-mentioned first key-value pairs.

[0121] In order to facilitate the determination of the first data block for aggregating and storing the first key-value pairs and the second key-value pairs containing the same node identifier, thereby accelerating the query of nodes and their connection points and improving the efficiency of graph query and graph traversal, when the above-mentioned target first key-value pairs and the above-mentioned target second key-value pairs are aggregately stored in a plurality of consecutive first data blocks, an index (which can be called a second-level index) created based on the key in the target second key-value pair can also be stored in the first first data block among these plurality of first data blocks.

[0122] It should be noted that the above-mentioned second-level index can be used to indicate the first data block where the above-mentioned target second key-value pair is located. In this case, the first-level index can specifically be used to indicate the first first data block where the first key-value pairs and the second key-value pairs containing each node identifier are located. That is, at this time, only the first-level index needs to be used to find the first first data block where the first key-value pairs and the second key-value pairs containing the node identifier in the target key are located, and the remaining first data blocks can be found through the second-level index in the first first data block.

[0123] Specifically, the second-level index can include: the correspondence between the key range of the target second key-value pair in each of the first data blocks included in the above-mentioned plurality of consecutive first data blocks and the storage location information of this first data block in the above-mentioned Graph-Aware SSTable file. Among them, the storage location information can include the offset of this first data block in the above-mentioned Graph-Aware SSTable file (which can indicate the starting position); in addition, the storage location information can also include other auxiliary information, such as the size of this first data block, so as to facilitate the determination of the termination position of this first data block.

[0124] In practical applications, for the Nth (N is a natural number) of the above-mentioned consecutive first data blocks, the key range of the target second key-value pairs in the Nth first data block can be represented by the smallest key and the largest key among these target second key-value pairs, or can be represented by the largest key in the target second key-value pairs in the Nth first data block and the largest key in the target second key-value pairs in the (N + 1)th first data block. That is, the key range of the target second key-value pairs in a first data block in the above-mentioned secondary index can specifically be the range of the smallest key and the largest key in the target second key-value pairs in this first data block, or can only be the largest key in the target second key-value pairs in this first data block.

[0125] For example, assume that the first data block 1 stores (first key-value pair 1, second key-value pair 1-1, second key-value pair 1-2, first key-value pair 2, second key-value pair 2-1), and the first data block 2 stores (second key-value pair 2-2, second key-value pair 2-3, second key-value pair 2-4, second key-value pair 2-5, second key-value pair 2-6). Then it indicates that when the target first key-value pair is the first key-value pair 2 and the target second key-value pairs include the second key-value pair 2-1, the second key-value pair 2-2, the second key-value pair 2-3, the second key-value pair 2-4, the second key-value pair 2-5, and the second key-value pair 2-6, the target first key-value pair and the target second key-value pairs are aggregated and stored in two consecutive first data blocks, namely the first data block 1 and the first data block 2. In this case, since the largest key in the target second key-value pairs in the first data block 1 is the key in the second key-value pair 2-1, and the largest key in the target second key-value pairs in the first data block 2 is the key in the second key-value pair 2-6, the secondary index created based on the keys in the target second key-value pairs can be as shown in Table 5 below:

[0126] Table 5

[0127]

[0128] Moreover, the data stored in the first data block 1 can specifically be (first key-value pair 1, second key-value pair 1-1, second key-value pair 1-2, first key-value pair 2, second key-value pair 2-1, the secondary index corresponding to the first key-value pair 2).

[0129] Please, on the basis of Figure 1 and Figure 2 , refer to Figure 3 , Figure 3 which is a flowchart of a graph query method shown in an exemplary embodiment of the present application.

[0130] In this embodiment, as mentioned above, the nodes and edges in the graph can be stored in an SSTable file (specifically, such as Figure 2The Graph-Aware SSTable file shown), the SSTable file may include at least one first data block for storing key-value pairs corresponding to nodes and edges in the graph, and a second data block for storing edge attributes of edges in the graph.

[0131] Nodes in the above graph can be organized into first key-value pairs corresponding to the nodes, and edges in the graph can be organized into second key-value pairs corresponding to the edges, and stored in the above first data block. Among them, the key in a first key-value pair may include the node identifier of a node, and the value may include the node attributes of this node; the key in a second key-value pair may include the start identifier and end identifier of an edge, and the value may include the storage location information of the edge attributes of this edge in the above SSTable file.

[0132] For a first key-value pair among all first key-value pairs, this first key-value pair can be used as the target first key-value pair. The target first key-value pair, and the second key-value pair whose start identifier is the same as the node identifier in the target first key-value pair (which can be called the target second key-value pair), are aggregated and stored in the above first data block.

[0133] It should be noted that the above graph query method can be implemented by a graph storage engine built based on a key-value storage system for graph query in the key-value storage system.

[0134] As Figure 3 shown, the above graph query method may include the following steps:

[0135] Step 302: Obtain a target key for querying the graph.

[0136] In this embodiment, through the Get operation and Seek operation, a query can be made for the above graph based on a key (which can be called the target key), that is, execute the Get(Key) operation or Seek(Key) operation, where the Key is the target key.

[0137] Step 304: Query in the SSTable file based on the target node identifier in the target key to determine a target first data block for aggregating and storing the first key-value pair and the second key-value pair containing the target node identifier.

[0138] In this embodiment, as described above, whether it is the above first key-value pair or the above second key-value pair, the key therein contains a node identifier. Therefore, first, the node identifier (which can be called the target node identifier) can be extracted from the above target key. Subsequently, based on the target node identifier, a query can be made in the above SSTable file, so that a first data block (which can be called the target first data block) for aggregating and storing the first key-value pair and the second key-value pair containing the target node identifier can be determined.

[0139] In some embodiments, the above SSTable file may further include a first-level index created based on the node identifier in the first key-value pair. The first-level index can be used to indicate the first data block where the first key-value pair and the second key-value pair containing each node identifier are located, that is, the first data block that aggregately stores the corresponding first key-value pair and second key-value pair.

[0140] Since the first-level index is an index created based on the node identifier and used to indicate the first data block where the first key-value pair and the second key-value pair containing each node identifier are located, by querying the first-level index, it is possible to determine the above first data block for aggregately storing the first key-value pair and the second key-value pair containing the target node identifier.

[0141] In some embodiments, when the above target key is obtained, it is possible to first query in the corresponding LSM-Tree based on the target key to determine the SSTable file in the LSM-Tree for storing the key-value pair containing the target key, and then query in the above first-level index included in the determined SSTable file based on the above target node identifier in the target key.

[0142] As described above, it is possible to determine whether the SSTable file may contain the target key through its associated Bloom Filter. That is, it is possible to determine the above SSTable file for storing the key-value pair containing the above target key through the Bloom Filter associated with each SSTable file in the LSM-Tree included in the above key-value storage system.

[0143] Step 306: Query in the target first data block based on the target key to determine the target value corresponding to the target key.

[0144] In this embodiment, it is possible to query in the above target first data block based on the above target key. Since the first data block is used to actually store the above first key-value pair and the above second key-value pair, by querying the target first data block, it is possible to determine the target value corresponding to the target key, and thus it is possible to determine the graph query result based on the target value.

[0145] Step 308: If the target value is the node attribute, determine the node attribute as the graph query result.

[0146] Step 310: If the target value is the storage location information, determine the edge attribute corresponding to the target key in the second data block included in the SSTable file according to the storage location information, and determine the edge attribute corresponding to the target key as the graph query result.

[0147] In this embodiment, as described above, the SSTable file may further include the second data block for storing edge attributes; the value in the second key-value pair may include the storage location information of the edge attributes of the corresponding edge in the SSTable file.

[0148] When determining the graph query result based on the above target value, different operations may be performed according to the type of the target value. Specifically, if the target value is the value in the first key-value pair, that is, the node attribute, the node attribute (i.e., the target value itself) may be directly determined as the graph query result; if the target value is the value in the second key-value pair, that is, the storage location information of the edge attributes in the SSTable file, the value corresponding to the target key may be determined in the second data block included in the SSTable file according to the storage address information in the target value, that is, the edge attribute, and the edge attribute corresponding to the target key may be determined as the graph query result.

[0149] In some embodiments, as described above, the data volume of the first key-value pair and the second key-value pair stored in the first data block is not greater than a preset threshold. Accordingly, the target first key-value pair and the target second key-value pair may be aggregated and stored in one first data block, or may be aggregated and stored in a plurality of consecutive first data blocks.

[0150] In some embodiments, when the target first key-value pair and the target second key-value pair are aggregated and stored in a plurality of consecutive first data blocks, the first first data block among the plurality of consecutive first data blocks may further be used to store a secondary index created based on the key in the target second key-value pair.

[0151] The secondary index may be used to indicate the first data block where the target second key-value pair is located. Accordingly, the primary index may specifically be used to indicate the first first data block where the first key-value pair and the second key-value pair including each node identifier are located.

[0152] When determining the graph query result based on the above target value, if the target value is the secondary index, the query may be further performed in the secondary index based on the target key. Since the secondary index is an index created based on the key in the target second key-value pair and used to indicate the first data block where the target second key-value pair is located, the first data block corresponding to the target key may be determined by querying the secondary index. Subsequently, the query may be performed in the first data block corresponding to the target key based on the target key to determine the value corresponding to the target key, and the graph query result may be determined based on the value corresponding to the target key.

[0153] Similarly to the foregoing, when determining the graph query result based on the value corresponding to the foregoing target key, since the secondary index hit by the target key at this time is created based on the key in a specific second key-value pair, the value corresponding to the target key should be the storage location information of the edge attribute in the foregoing SSTable file. Therefore, according to the storage address information in this value, the value corresponding to the target key, that is, the edge attribute, can be determined in the foregoing second data block included in the SSTable file, and the edge attribute corresponding to the target key is determined as the graph query result.

[0154] It should be noted that the foregoing secondary index can exist independently in the foregoing first first data block, that is, it is not included as the value in a key-value pair in this first data block. In this case, when the foregoing target key does not match the keys in each key-value pair included in this first data block, the secondary index in this first data block can be directly determined as the foregoing target value corresponding to the target key.

[0155] Alternatively, the foregoing secondary index can be included as the value in a key-value pair in the foregoing first first data block, and the key in this key-value pair can be the key range of the foregoing target second key-value pair. In this case, when the foregoing target key hits the key range in this first data block (at this time the target key does not match the keys in each key-value pair included in this first data block), the secondary index in this first data block can be determined as the foregoing target value corresponding to the target key.

[0156] In some embodiments, as described above, the foregoing secondary index may include: the correspondence between the key ranges of the target second key-value pairs in each of the foregoing consecutive first data blocks and the storage location information of this first data block in the foregoing SSTable file.

[0157] Therefore, when further querying in the foregoing secondary index based on the foregoing target key to determine the first data block corresponding to the target key, specifically, a binary search can be further performed in this secondary index based on the target key to determine the storage location information of the first data block corresponding to the target key in the foregoing SSTable file, and according to this storage location information, the first data block corresponding to the target key is determined in this SSTable file.

[0158] In some embodiments, as described above, the foregoing primary index may include: the correspondence between each node identifier and the storage location information of the first first data block where the first key-value pair and the second key-value pair including this node identifier are located in the foregoing SSTable file. Correspondingly, the correspondence between the node identifier and the storage location information in the primary index can be arranged in the order of the first key-value pairs including each node identifier in the first data block.

[0159] In the above case, according to the arrangement order of the correspondence between the node identifiers and the storage location information in the above first index, the target first data block in the above SSTable file for aggregating and storing the first key-value pair and the second key-value pair including the above target node identifier can be determined. Specifically, first, a query can be performed in the above first-level index included in the SSTable file based on the target node identifier to determine the corresponding relationship (which can be called the target corresponding relationship) including the target node identifier, and the next corresponding relationship of the target corresponding relationship. Then, in the SSTable file, the first data blocks within the range from the first data block indicated by the storage location information included in the target corresponding relationship to the first data block indicated by the storage location information included in the next corresponding relationship of the target corresponding relationship are determined as the target first data blocks.

[0160] In some embodiments, for the above key-value storage system, the specific format of the SSTable files included in the LSM-Tree therein can be selected according to the actual application scenarios and requirements.

[0161] Taking the above SSTable file as an example, when the subgraph composed of the nodes and edges corresponding to the key-value pairs stored in the SSTable file is a sparse subgraph, the above first-level index in the SSTable file can be the index block in the SSTable file. And when the subgraph composed of the nodes and edges corresponding to the key-value pairs stored in the SSTable file is a dense subgraph, the above first-level index in the SSTable file can be the hash index.

[0162] For another example, in the case of a graph analysis scenario, storing edge attributes by column is more convenient for access. Therefore, the edge attributes can be separately stored in the above second data block.

[0163] It should be noted that the specific formats of the SSTable files included in the same LSM-Tree can also be different. In this case, when merging the SSTable files, the SSTable files with the same format can be merged.

[0164] In the above technical solution, the nodes and edges in the graph can be stored in an SSTable file, which can include at least one first data block for storing key-value pairs corresponding to the nodes and edges, and a second data block for storing the edge attributes of the edges; the nodes and edges can be respectively organized into first key-value pairs and second key-value pairs and stored in the first data block. Among them, the key in the first key-value pair corresponding to a node can include the node identifier of the node, and the value can include the node attributes of the node. The key in the second key-value pair corresponding to an edge can include the start point identifier and the end point identifier of the edge, and the value can include the storage location information of the edge attributes of the edge in the SSTable file; each target first key-value pair, and the target second key-value pair whose start point identifier is the same as the node identifier in the target first key-value pair, can be aggregated and stored in the first data block. Accordingly, when obtaining the target key for querying the graph, first, based on the target node identifier in the target key, query in the SSTable file to determine the target first data block for aggregating and storing the first key-value pair and the second key-value pair containing the target node identifier, and then, based on the target key, query in the target first data block to determine the target value corresponding to the target key; if the target value is node attributes, the node attributes can be determined as the graph query result; if the target value is storage location information, the edge attributes corresponding to the target key can be determined in the second data block included in the SSTable file according to the storage location information, and the edge attributes corresponding to the target key can be determined as the graph query result.

[0165] By adopting the above method, the optimization of the key-value storage system is realized by using the characteristics of graph topology and graph query, specifically including adjusting and improving aspects such as the SSTable file format and indexing method in the key-value storage system, so that the SSTable file for storing graph data in the key-value storage system is graph-aware, thereby reducing the complexity of the graph query operation of the graph storage engine based on the key-value storage system in the key-value storage system and improving the graph query efficiency, thus ensuring that the graph storage engine has relatively high graph query performance.

[0166] Corresponding to the embodiment of the foregoing method, the present application also provides an embodiment of a device.

[0167] Please refer to Figure 4 , Figure 4It is a schematic structural diagram of a device shown in an exemplary embodiment of the present application. At the hardware level, the device includes a processor 402, an internal bus 404, a network interface 406, a memory 408, and a non-volatile memory 410. Of course, it may also include other required hardware. One or more embodiments of the present application can be implemented in a software manner. For example, the processor 402 reads the corresponding computer program from the non-volatile memory 410 into the memory 408 and then runs it. Of course, in addition to the software implementation, one or more embodiments of the present application do not exclude other implementation manners, such as a logic device or a combination of software and hardware. That is to say, the execution subject of the following processing flow is not limited to each logic module, and can also be hardware or a logic device.

[0168] Please refer to Figure 5 , Figure 5 It is a block diagram of a graph query device shown in an exemplary embodiment of the present application.

[0169] The above graph query device can be applied to Figure 5 the device shown in the figure to implement the technical solution of the present application. Among them, the nodes and edges in the figure are stored in the sorted string table SSTable file; the SSTable file includes at least one first data block for storing key-value pairs corresponding to the nodes and the edges, and a second data block for storing the edge attributes of the edges; the nodes and the edges are respectively organized into first key-value pairs and second key-value pairs and stored in the first data block; the key in the first key-value pair includes the node identifier of the node, and the value includes the node attributes of the node; the key in the second key-value pair includes the start identifier and end identifier of the edge, and the value includes the storage location information of the edge attributes of the edge in the SSTable file; each target first key-value pair, and the target second key-value pair whose start identifier is the same as the node identifier in the target first key-value pair, are aggregated and stored in the first data block.

[0170] The above graph query device may include:

[0171] An acquisition module 502, which acquires a target key for querying the graph;

[0172] A first query module 504, which queries in the SSTable file based on the target node identifier in the target key to determine a target first data block for aggregating and storing the first key-value pair and the second key-value pair containing the target node identifier;

[0173] A second query module 506, which queries in the target first data block based on the target key to determine a target value corresponding to the target key;

[0174] Determine module 508. If the target value is the node attribute, determine the node attribute as the graph query result. If the target value is the storage location information, according to the storage location information, determine the edge attribute corresponding to the target key in the second data block included in the SSTable file, and determine the edge attribute corresponding to the target key as the graph query result.

[0175] In some embodiments, the SSTable file further includes a first-level index created based on the node identifier in the first key-value pair. The first-level index is used to indicate the first data block where the first key-value pair and the second key-value pair containing each node identifier are stored.

[0176] The query in the SSTable file based on the target node identifier in the target key to determine the target first data block for aggregating and storing the first key-value pair and the second key-value pair containing the target node identifier includes:

[0177] Query in the first-level index included in the SSTable file based on the target node identifier in the target key to determine the target first data block for aggregating and storing the first key-value pair and the second key-value pair containing the target node identifier.

[0178] In some embodiments, the SSTable file is the SSTable file in the Log-Structured Merge Tree (LSM-Tree).

[0179] The device further includes:

[0180] A third query module. Before querying in the SSTable file based on the target node identifier in the target key, query in the LSM-Tree based on the target key to determine the SSTable file for storing the key-value pair containing the target key.

[0181] In some embodiments, the data volume of the first key-value pair and the second key-value pair stored in each first data block is not greater than a preset threshold. The target first key-value pair and the target second key-value pair are aggregated and stored in one or a continuous plurality of first data blocks.

[0182] In some embodiments, when the target first key-value pair and the target second key-value pair are aggregated and stored in a continuous plurality of first data blocks, the first first data block in the continuous plurality of first data blocks is further used to store a second-level index created based on the key in the target second key-value pair. The second-level index is used to indicate the first data block where the target second key-value pair is located. The first-level index is used to indicate the first first data block where the first key-value pair and the second key-value pair containing each node identifier are located.

[0183] The determining module 508 is further configured to:

[0184] If the target value is the secondary index, query in the secondary index based on the target key to determine a first data block corresponding to the target key;

[0185] Query in the first data block corresponding to the target key to determine storage location information corresponding to the target key;

[0186] According to the storage location information corresponding to the target key, determine edge attributes corresponding to the target key in the second data block included in the SSTable file, and determine the edge attributes corresponding to the target key as the graph query result.

[0187] In some embodiments, the secondary index includes: the correspondence between the key range of the target second key-value pair in each of the consecutive first data blocks and the storage location information of the first data block in the SSTable file;

[0188] The querying in the secondary index based on the target key to determine a first data block corresponding to the target key includes:

[0189] Perform a binary search in the secondary index based on the target key to determine the storage location information of the first data block corresponding to the target key in the SSTable file, and according to the storage location information, determine the first data block corresponding to the target key in the SSTable file.

[0190] In some embodiments, the primary index includes: the correspondence between each node identifier and the storage location information of the first data block in the SSTable file where the first key-value pair and the second key-value pair containing the node identifier are located; the correspondence in the primary index is arranged in the order of the first key-value pair in the first data block;

[0191] The querying in the primary index included in the SSTable file based on the target node identifier in the target key to determine a target first data block for aggregating and storing the first key-value pair and the second key-value pair containing the target node identifier includes:

[0192] Query in the primary index included in the SSTable file based on the target node identifier in the target key to determine a target correspondence relationship containing the target node identifier and the next correspondence relationship of the target correspondence relationship;

[0193] Determine the first data blocks within the range from the first data block indicated by the storage location information included in the target correspondence to the first data block indicated by the storage location information included in the next correspondence in the SSTable file as the target first data blocks for aggregating and storing the first key-value pair and the second key-value pair including the target node identifier.

[0194] In some embodiments, when the subgraph composed of nodes and edges corresponding to the key-value pairs stored in the SSTable file is a sparse subgraph, the first-level index in the SSTable file is the index block in the SSTable file; when the subgraph composed of nodes and edges corresponding to the key-value pairs stored in the SSTable file is a dense subgraph, the first-level index in the SSTable file is a hash index.

[0195] In some embodiments, the key in the first key-value pair further includes the node type of the node.

[0196] In some embodiments, the key in the second key-value pair further includes the start type of the edge, and / or the establishment timestamp of the edge.

[0197] For the device embodiments, they basically correspond to the method embodiments. Therefore, for the relevant parts, refer to the partial descriptions of the method embodiments. The device embodiments described above are only illustrative. The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical modules, that is, they may be located in one place, or may be distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the technical solution of this application.

[0198] The systems, devices, modules or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, and the specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver device, a game console, a tablet computer, a wearable device, or a combination of any several of these devices.

[0199] In a typical configuration, a computer includes one or more processors (CPUs), an input / output interface, a network interface, and a memory.

[0200] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0201] Computer-readable media include permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0202] It should be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0203] The above specific embodiments of the present application are described. Other embodiments are within the scope of the present application. In some cases, the actions or steps recorded in the present application can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the process depicted in the accompanying drawings does not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0204] The terms used in one or more embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of the present application. The singular forms "a", "the" and "said" are also intended to include the plural forms unless the context clearly dictates otherwise. The term "and / or" means any and all possible combinations of one or more of the associated listed items.

[0205] The description of the terms "one embodiment", "some embodiments", "example", "specific example", or "a mode of implementation" etc. used in one or more embodiments of the present application means that the specific features or characteristics described in connection with the embodiment are included in at least one embodiment of the present application. The schematic description of these terms does not necessarily refer to the same embodiment. Moreover, the specific features or characteristics described may be combined in a suitable manner in one or more embodiments of the present application. In addition, different embodiments and the specific features or characteristics in different embodiments may be combined without contradiction.

[0206] It should be understood that although the terms first, second, third, etc. may be used in one or more embodiments of the present application to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0207] The above description is only the preferred embodiment of one or more embodiments of the present application and is not intended to limit one or more embodiments of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of the present application shall be included within the scope protected by one or more embodiments of the present application.

[0208] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of the relevant data need to comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entrances are provided for the user to choose to authorize or refuse.

Claims

1. A graph database system for storing graph data, the graph data including nodes and edges; wherein: An SSTable file is stored in the storage space of the graph database system; the SSTable file includes at least one first data block for storing key-value pairs corresponding to the nodes and the edges, and a second data block for storing edge attributes of the edges; The key-value pairs include first key-value pairs and second key-value pairs; the key in the first key-value pair includes the node identifier of the node, and the value includes the node attributes of the node; the key in the second key-value pair includes the start point identifier and the end point identifier of the edge, and the value includes the storage location information of the edge attributes of the edge in the SSTable file; In the first data block, various target first key-value pairs, and target second key-value pairs whose start point identifiers are the same as the node identifiers in the target first key-value pairs are aggregated and stored.

2. The graph database system according to claim 1, wherein the SSTable file further includes a first-level index created based on the node identifiers in the first key-value pairs; the first-level index is used to indicate the first data blocks where the first key-value pairs and the second key-value pairs containing each node identifier are located.

3. The graph database system according to claim 1, wherein a Log-Structured Merge Tree (LSM-Tree) is stored in the storage space of the graph database system; the SSTable file is the SSTable file in the LSM-Tree.

4. The graph database system according to claim 2, wherein the data volume of the first key-value pairs and the second key-value pairs stored in each first data block does not exceed a preset threshold; the target first key-value pairs and the target second key-value pairs are aggregated and stored in one or a continuous plurality of first data blocks.

5. The graph database system according to claim 4, when the target first key-value pair and the target second key-value pair are aggregately stored in a plurality of consecutive first data blocks, the first first data block among the plurality of consecutive first data blocks further stores a secondary index created based on the key in the target second key-value pair; wherein, The second-level index is used to indicate the first data block where the target second key-value pair is located; the first-level index is used to indicate the first first data block where the first key-value pairs and the second key-value pairs containing each node identifier are located.

6. The graph database system according to claim 5, wherein the secondary index includes: The corresponding relationship between the key ranges of the target second key-value pairs in each of the continuous plurality of first data blocks and the storage location information of the first data block in the SSTable file.

7. The graph database system according to claim 2, wherein the primary index includes: The corresponding relationship between each node identifier and the storage location information of the first first data block where the first key-value pair and the second key-value pair containing the node identifier are located in the SSTable file; The corresponding relationship in the first-level index is arranged in the order of the first key-value pairs in the first data block.

8. The graph database system according to claim 2, when the subgraph composed of the nodes and edges corresponding to the key-value pairs stored in the SSTable file is a sparse subgraph, the first-level index in the SSTable file is an index block in the SSTable file; when the subgraph composed of the nodes and edges corresponding to the key-value pairs stored in the SSTable file is a dense subgraph, the first-level index in the SSTable file is a hash index.

9. A method for graph query; wherein, The nodes and edges in the figure are stored in a sorted string table SSTable file; the SSTable file includes at least one first data block for storing key-value pairs corresponding to the nodes and the edges, and a second data block for storing the edge attributes of the edges; the nodes and the edges are respectively organized into first key-value pairs and second key-value pairs and stored in the first data block; The key in the first key-value pair includes the node identifier of the node, and the value includes the node attributes of the node; the key in the second key-value pair includes the start point identifier and the end point identifier of the edge, and the value includes the storage location information of the edge attributes in the SSTable file; each target first key-value pair, and a target second key-value pair whose start point identifier is the same as the node identifier in the target first key-value pair are aggregated and stored in the first data block; The method includes: Obtaining a target key for querying the figure; Querying in the SSTable file based on the target node identifier in the target key to determine a target first data block for aggregating and storing the first key-value pair and the second key-value pair containing the target node identifier; Querying in the target first data block based on the target key to determine a target value corresponding to the target key; If the target value is the node attribute, determining the node attribute as the figure query result; If the target value is the storage location information, determining, according to the storage location information, the edge attribute corresponding to the target key in the second data block included in the SSTable file, and determining the edge attribute corresponding to the target key as the figure query result.

10. The method according to claim 9, wherein the SSTable file further includes a first-level index created based on the node identifier in the first key-value pair; the first-level index is used to indicate the first data block where the first key-value pair and the second key-value pair containing each node identifier are located; The querying in the SSTable file based on the target node identifier in the target key to determine a target first data block for aggregating and storing the first key-value pair and the second key-value pair containing the target node identifier includes: Querying in the first-level index included in the SSTable file based on the target node identifier in the target key to determine a target first data block for aggregating and storing the first key-value pair and the second key-value pair containing the target node identifier.

11. The method according to claim 9, wherein the SSTable file is the SSTable file in a log-structured merge tree LSM-Tree; Before querying in the SSTable file based on the target node identifier in the target key, the method further includes: Querying in the LSM-Tree based on the target key to determine the SSTable file for storing the key-value pair containing the target key.

12. According to the method described in claim 10, the data volume of the first key-value pairs and the second key-value pairs stored in each first data block is not greater than a preset threshold; the target first key-value pair and the target second key-value pair are stored aggregately in one or a continuous plurality of first data blocks.

13. According to the method described in claim 12, when the target first key-value pair and the target second key-value pair are aggregately stored in a plurality of consecutive first data blocks, the first first data block among the plurality of consecutive first data blocks is further used to store a secondary index created based on the key in the target second key-value pair; the secondary index is used to indicate the first data block where the target second key-value pair is located; The first-level index is used to indicate the first data block where the first key-value pairs and the second key-value pairs containing each node identifier are located; The method further includes: If the target value is the second-level index, perform a query in the second-level index based on the target key to determine the first data block corresponding to the target key; Perform a query in the first data block corresponding to the target key based on the target key to determine the storage location information corresponding to the target key; According to the storage location information corresponding to the target key, determine the edge attribute corresponding to the target key in the second data block included in the SSTable file, and determine the edge attribute corresponding to the target key as the graph query result.

14. The method according to claim 13, wherein the secondary index comprises: The corresponding relationship between the key ranges of the target second key-value pairs in each of the continuous plurality of first data blocks and the storage location information of the first data block in the SSTable file; The performing a query in the second-level index based on the target key to determine the first data block corresponding to the target key includes: Perform a binary search in the second-level index based on the target key to determine the storage location information of the first data block corresponding to the target key in the SSTable file, and determine the first data block corresponding to the target key in the SSTable file according to this storage location information.

15. The method according to claim 10, wherein the primary index includes: The corresponding relationship between each node identifier and the storage location information of the first data block where the first key-value pair and the second key-value pair containing this node identifier are located in the SSTable file; The corresponding relationship in the first-level index is arranged in the order of the first key-value pairs in the first data block; The performing a query in the first-level index included in the SSTable file based on the target node identifier in the target key to determine the target first data block for aggregately storing the first key-value pair and the second key-value pair containing the target node identifier includes: Perform a query in the first-level index included in the SSTable file based on the target node identifier in the target key to determine the target corresponding relationship containing the target node identifier and the next corresponding relationship of the target corresponding relationship; Determine the first data blocks within the range from the first data block indicated by the storage location information included in the target corresponding relationship to the first data block indicated by the storage location information included in the next corresponding relationship in the SSTable file as the target first data blocks for aggregately storing the first key-value pair and the second key-value pair containing the target node identifier.

16. According to the method described in claim 10, when the subgraph composed of nodes and edges corresponding to the key-value pairs stored in the SSTable file is a sparse subgraph, the primary index in the SSTable file is the index block in the SSTable file; when the subgraph composed of nodes and edges corresponding to the key-value pairs stored in the SSTable file is a dense subgraph, the primary index in the SSTable file is a hash index.

17. An electronic device, comprising: a processor; a memory for storing processor-executable instructions; wherein, the processor realizes the method according to any one of claims 9 to 16 by running the executable instructions.

18. A computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the method according to any one of claims 9 to 16 is realized.

Images