Graph databases

Abstract

The present invention relates to a computer implemented method of natively storing a set of nodes and / or edges in a graph database, comprising generating a bitset representing a set of graph nodes or graph edges, and setting the value of a bit in the bitset in dependence on whether a corresponding node and / or edge is part of the set. The present invention also relates to a computer implemented method of querying data in a graph database, the graph database comprising graph nodes connected by graph edges, the method comprising: receiving a query, the query comprising a first query node, a first query edge, and a second query node arranged in a pattern; for the first query node, identifying a first set of graph nodes or graph edges associated with the graph nodes which satisfy criteria of the first query node; for the first query edge, identifying a second set of graph nodes or graph edges associated with the graph edges which satisfy criteria of the first query edge; for the second query node, identifying a third set of graph nodes or graph edges associated with the graph nodes which satisfy criteria of the second query node; and computing the intersection of the first, second and third sets of graph nodes or graph edges to obtain one or more first output sets of graph nodes or graph edges.

Description

[0001] Graph Databases

[0002] Field of Invention

[0003] The present invention relates to the field of graph databases. More specifically, the invention relates to computer implemented methods of natively storing a set of nodes and / or edges in a graph database, and querying data in a graph database.

[0004] Background

[0005] Graph databases store connected data in graph structures formed of nodes, edges and properties. Nodes (also known as vertices) represent individual entities or instances in the database. Edges (also known as relationships) connect the nodes together in the graph and represent the relationships between the nodes. In some cases, edges may have a direction; a unidirectional edge represents a one-way relationship between two nodes, and a bidirectional edge represents a two-way relationship between two nodes. The nodes and edges may have properties representing data associated with the nodes or edges.

[0006] Graph databases are typically used to store very large datasets of connected data. Exemplary uses of graph databases include social networks and large knowledge bases to serve internet search engines or large language models. For a graph database representing a social network, for example, the nodes in the graph may represent the individual users of the social network (where the properties of the nodes represent the data associated with the users) and the edges in the graph may represent the relationships between the users (where the properties of the edges may represent, for example, the type of relationship between the users).

[0007] The current state of the art algorithms for graph querying use row-expansion methods, which operate by picking a starting graph node and expanding outwards into the graph through edges and adjacent nodes that correspond to the query pattern. These algorithms often involve traversing a significant proportion of nodes and edges in the graph and exploring all matching paths in order to satisfy the query, particularly when the query demands specific sort orders or aggregations. Due to the connected nature of graphs, the compute cost of rowexpansion or path-at-a-time approaches scales exponentially with the length of the pattern being matched and the connectedness of the graph and entails repeated redundant tests of nodes and edges that have already been visited. Thread coordination overhead also makes this hard to parallelise efficiently. of the Invention

[0008] Aspects and embodiments of the present invention are set out in the appended claims. These and other aspects and embodiments of the present disclosure are also described herein.

[0009] In overview, the present disclosure seeks to address the non-linear performance characteristics when querying a graph by first deriving an intermediate representation via an efficient and parallelisable row agnostic process (described as “reduction” below). This intermediate representation, and the method of deriving it, avoids redundant checks and affords many optimisations and shortcuts not available to path exploration approaches for satisfying the query in its entirety.

[0010] According to at least one aspect described herein, there is provided a computer implemented method of natively storing a set of nodes and / or edges in a graph database, comprising generating a bitset representing a set of graph nodes or graph edges, and setting the value of a bit in the bitset in dependence on whether a corresponding node and / or edge is part of the set.

[0011] As used in this disclosure, the term "graph storage” and related terms are used to refer to the underlying structure of connected data persisted in either memory or on disk. When the storage is built specifically for graph data, it is known as “native” graph storage. Native graph storage is optimised for storing and querying highly connected data and is designed to handle complex relationships between data points efficiently. Accordingly, as used in this disclosure, the term “native graph storage”, or related terms such as “natively storing”, are used to refer to the underlying data structures optimised specifically for storing connected graph data, persisted in either memory or on disk.

[0012] Preferably, the nodes and / or edges in the graph database each have an associated unique identifier. The unique identifier may be an internal identifier, in the sense that it is applied internally within the graph database, rather than presented externally to a user of the graph database.

[0013] Preferably, the method comprises setting the value of a bit in the bitset in dependence on whether a node and / or edge having a unique identifier corresponding to that bit is part of the set.

[0014] Preferably, the method comprises setting the value of a bit in the bitset in dependence on whether a node and / or edge having a unique identifier corresponding to the position of that bit in the bitset is part of the set. Preferably, wherein the unique identifiers for the nodes and / or edges in the graph database are integers, preferably monotonically increasing integers.

[0015] Preferably, the method comprises assigning the unique identifiers to the nodes and / or edges in the graph database based on a property of the node and / or edge.

[0016] Preferably, the assigning the unique identifiers comprises selecting a unique identifier from one or more of a plurality of ranges of possible unique identifiers based on a property of the node or edge, preferably wherein each range in the plurality of ranges corresponds to a different property.

[0017] Preferably, for the nodes, the property comprises a label of the node.

[0018] Preferably, for the edges, the property comprises one or more of: a property of a source node for the edge, such as a label of the source node; a property of a target node for the edge, such as a label of the target node; or a label of the edge.

[0019] In this way, the unique identifiers assigned to nodes and edges form blocks with a shared property. Thus, queries involving retrieving data relating to nodes or a certain type, or edges of a certain term, can be satisfied trivially by retrieving the data from the relevant block(s).

[0020] Preferably, the method comprises, for a node in the graph database, generating a bitset representing the set of edges which point at that node as a target.

[0021] Preferably, the method comprises, for a node in the graph database, generating a bitset representing the set of edges which point from that node as a source.

[0022] Preferably, the method comprises, for an edge in the graph database, generating a bitset representing the source node(s) for that edge.

[0023] Preferably, the method comprises, for an edge in the graph database, generating a bitset representing the target node(s) for that edge.

[0024] Preferably, wherein the bitsets are formed of words having a length that is selected to suit hardware used to perform operations on the bitsets, preferably 64-bit or 128-bit words. The bitsets may be implemented as efficient bitsets.

[0025] The hardware used to perform any of the set operations described in the present disclosure maybe be one or more central processing units (CPUs), one or more graphics processing units (GPUs) or one or more neural processing units (NPUs) or any combination therefore arranged to perform set operations in parallel. According to another aspect disclosed herein, there is provided a computer implemented data structure for representing a set of nodes and / or edges in a graph database, the data structure comprising at least one bit array representing a set of graph nodes or graph edges, wherein a value of a bit in the array is set in dependence on whether a corresponding node and / or edge in the graph database is part of the set. The data structure can be used for retrieval of data from the graph database.

[0026] Preferably, the nodes and / or edges in the graph database each have an associated unique identifier.

[0027] Preferably, the value of a bit in the array is set in dependence on whether a node and / or edge having a unique identifier corresponding to that bit is part of the set.

[0028] Preferably, the value of a bit in the array is set in dependence on whether a node and / or edge having a unique identifier corresponding to the position of that bit in the array is part of the set.

[0029] Preferably, the unique identifiers for the nodes and / or edges in the graph database are integers, preferably monotonically increasing integers.

[0030] Preferably, the unique identifiers are assigned to the nodes and / or edges in the graph database based on a property of the node and / or edge.

[0031] Preferably, the unique identifier for a node or edge is assigned to the node or edge from one or more of a plurality of ranges of possible unique identifiers based on a property of the node or edge, preferably wherein each range in the plurality of ranges corresponds to a different property.

[0032] Preferably, the data structure includes metadata associated with each of the plurality of ranges of possible unique identifiers.

[0033] Preferably, the ranges are sub-divided into blocks of fixed size. Preferably, unique identifiers from within a range are assigned from a first of the blocks until all unique identifiers in first block are assigned, after which unique identifiers from a second of the blocks are assigned, and so on.

[0034] Preferably, the metadata associated with each of the plurality of ranges of possible unique identifiers comprises metadata associated with each block in the range.

[0035] Preferably, the fixed block size is a multiple of 64 bits. Preferably, the fixed block size is in the range 1 ,024 to 16,384 bits.

[0036] Preferably, the metadata includes the property of the node or edge. Preferably, for the nodes, the property comprises a label of the node.

[0037] Preferably, for the edges, the property comprises one or more of: a property of a source node for the edge, such as a label of the source node; a property of a target node for the edge, such as a label of the target node; or a label of the edge.

[0038] The ‘label’ may specify the type of the node or edge.

[0039] When the abovementioned data structure is used to represent nodes or edges in the graph database, querying the graph database may comprise identifying, from the metadata associated with each of the plurality of ranges of possible unique identifiers, a set of nodes or edges that satisfy at least a part of a query pattern.

[0040] Preferably, for a node in the graph database, the data structure comprises a bit array representing the set of edges which point at that node as a target.

[0041] Preferably, for a node in the graph database, the data structure comprises a bit array representing the set of edges which point from that node as a source.

[0042] Preferably, for an edge in the graph database, the data structure comprises a bit array representing the source node(s) for that edge.

[0043] Preferably, for an edge in the graph database, the data structure comprises a bit array representing the target node(s) for that edge.

[0044] Preferably, the at least one-bit array comprises a plurality of bit arrays, and wherein each different bit array in the plurality represents a set of nodes and / or edges with a different property.

[0045] Any of the above aspects of the data structure may be applied equally to the method aspects described above.

[0046] According to another aspect described herein, there is provided a computer implemented method of querying data in a graph database, the graph database comprising graph nodes connected by graph edges, the method comprising: receiving a query, the query comprising at least one query node and at least one query edge arranged in a pattern; performing at least one set operation between set data structures representing the at least one query node and the at least one query edge to obtain an intermediate representation of the graph nodes and graph edges matching at least part of the query pattern. The method may also comprise retrieving, from the graph database, data associated with the graph nodes and graph edges from the intermediate representation. According to another aspect described herein, there is provided a computer implemented method of querying data in a graph database, the graph database comprising graph nodes connected by graph edges, the method comprising: receiving a query, the query comprising a first query node, a first query edge, and a second query node arranged in a pattern; for the first query node, identifying a first set of graph nodes or graph edges associated with the graph nodes which satisfy criteria of the first query node; for the first query edge, identifying a second set of graph nodes or graph edges associated with the graph edges which satisfy criteria of the first query edge; for the second query node, identifying a third set of graph nodes or graph edges associated with the graph nodes which satisfy criteria of the second query node; and computing the intersection of the first, second and third sets of graph nodes or graph edges to obtain one or more first output sets of graph nodes or graph edges.

[0047] This first output (which is also referred to herein as an ‘intermediate representation’) is therefore derived via an efficient and parallelizable row agnostic process. This intermediate representation, and the method of deriving it, avoids redundant checks and affords many optimisations and shortcuts not available to path exploration approaches for satisfying the query in entirety. The method may comprise retrieving from the graph database data associated with the graph nodes and / or graph edges represented by the first output set.

[0048] Preferably, the query comprises a second query edge, and the method further comprises: for the second query edge, identifying a fourth set of graph nodes or graph edges associated with the graph edges which satisfy criteria of the second query edge; and computing the intersection of a set associated with the first output set and the fourth set of graph nodes or graph edges to obtain a second output set of graph nodes and / or graph edges. In this way, candidate nodes or edges from the first segment of the query are propagated forward through further segments of the query, reducing redundant checks.

[0049] Preferably, the query comprises a third query node, and the method further comprises: for the third query node, identifying a fifth set of graph nodes or graph edges associated with the graph nodes which satisfy criteria of the third query node; and computing the intersection of the set associated with the first output set the fourth and fifth sets of graph nodes or graph edges to obtain the second output set of graph nodes and / or graph edges.

[0050] Preferably, the method comprises removing at least one graph node or graph edge from the first output set of graph nodes or graph edges based on the second output set of graph nodes or graph edges. Preferably, the first query node and / or the first query edge are selected from among multiple query nodes and / or multiple query edges in the query based on statistics associated with the query nodes and / or query edges.

[0051] Preferably, the first and / or third set of graph nodes or graph edges comprises a set of graph edges which point to graph nodes which satisfy the criteria of the first and / or second query nodes respectively, and / or a set of graph edges which point from graph nodes which satisfy the criteria of the first and / or second query node respectively.

[0052] Preferably, the second set of graph nodes or graph edges comprises a set of graph nodes which are sources for graph edges which satisfy the criteria of the first query edge and / or a set of graph nodes which are targets for graph edges which satisfy the criteria of the first query edge.

[0053] Preferably, the method comprises traversing the query pattern from a first query segment (i.e., a chosen start query segment) to a last query segment. Each segment comprises either a single query node in isolation or a combination of at least one query node and at least one query edge (such as exactly two query nodes and exactly one query edge, preferably wherein the one query edge is between the two query nodes). For each segment, the method comprises: identifying sets of graph nodes or graph edges associated with the graph nodes or graph edges which satisfy the criteria of the query nodes or query edges in the segment; and computing the intersection of the sets to obtain an output set for the segment.

[0054] Preferably, the method comprises traversing the query pattern from the last query segment to the first query segment; and based on the output set obtained from one segment, removing at least one graph node or graph edge from the output sets obtained from at least one other segment. In this case, the output sets (after this removal stop, which is described in more detail below as ‘pruning’) are used as the intermediate representation.

[0055] Preferably, the method comprises obtaining, from the output set(s) and node-edge maps, at least one output sub-graph of the graph database.

[0056] Preferably, the sets of graph nodes or graph edges are represented by set data structures (such as bitsets or rangesets) wherein the value of a bit in the set depends on whether a corresponding graph node and / or graph edge is part of the set.

[0057] Preferably, computing the intersection comprises parallelising the intersection computation across multiple processors, preferably across multiple graphics processing units (GPUs). According to another aspect described herein, there is provided a computer implemented method of querying data in a graph database, the graph database comprising nodes and / or edges stored according to any of the aforementioned methods, the method of querying data comprising: retrieving or identifying a bitset representing a set of graph nodes or graph edges; and retrieving, from the graph database, data associated with the graph nodes or graph edges represented in the bitset.

[0058] The method may comprise retrieving or identifying multiple bitsets representing multiples set of graph nodes and / or graph edges, and performing at least one set operation (e.g., an intersection operation) between the bitsets to obtain an intermediate representation of graph nodes or graph edges which match query nodes and / or query edges.

[0059] The method may comprise deriving from the intermediate representation one or more paths in the graph database (i.e., paths of nodes and edges through the graph database) matching a query path comprising the query nodes and / or query edges.

[0060] The method may comprise retrieving, from the graph database, data associated with the graph nodes and / or graph edges in the one or more paths in the graph database derived from the intermediate representation.

[0061] For any of the query nodes, identifying the corresponding set of graph nodes which satisfy criteria of the query node may comprise checking each graph node in the database against the query node at most once. Similarly, for any of the query edges, identifying the corresponding set of graph edges which satisfy criteria of the query edge comprises checking each graph edge in the database against the query edge at most once. Thus, for each node or edge in the query pattern, the nodes and edges in the graph database are only checked at most once for a match against the query nodes and edges. Some graph nodes or edges may not be checked at all, for example if a group of nodes or edges can be discounted without needing to check each node or edge in the group. This reduces redundant checks and speeds up the query process.

[0062] Preferably, any of the methods described above comprise retrieving data from the graph database. The data may be retrieved based on the output of the method, such as an output bitset, a retrieved or identified bitset, and / or output set of nodes or edges, or an output subgraph (i.e., intermediate representation).

[0063] Preferably, the any of the methods described above may include representing the nodes and edges in the graph database using any of the data structures described above during the process of retrieving data from the graph database. According to another aspect described herein, there is provided a computer program product comprising software code adapted, when executed on one or more processors, to carry out any of the aforementioned methods.

[0064] According to another aspect described herein, there is provided a graph database comprising nodes and / or edges stored according to any of the aforementioned methods, or using any of the aforementioned data structures.

[0065] In contrast to the background art, the present disclosure does not traverse the graph or walk the paths, is easy to parallelise, and produces an intermediate graph data representation that allows some queries to be satisfied without needing a walk of the paths during projection.

[0066] The novel native graph storage data structure, the unconventional derivation of an intermediate representation, and the novel recursive algorithm for deriving the intermediate representation give rise to a number of technical advantages. In particular, the present disclosure enables the ability to query the data without doing any graph walking, keeping the query linear in the size of the query pattern and size and order of the graph; the ability to do highly uncoordinated parallel processing of the query; and the ability to do this on hardware that natively supports bitset operations (e.g., GPUs) for even faster processing of the query.

[0067] As used herein, the term “bitsets” refers to an array of binary bits. The “position” of a bit in the bitset refers to where the bit appears in sequence in the array; it does not refer to the significance of the bit in the array.

[0068] Any apparatus feature described herein may also be provided as a method feature, and vice versa. As used herein, means plus function features may be expressed alternatively in terms of their corresponding structure.

[0069] Any feature in one aspect of the disclosure may be applied to other aspects of the disclosure, in any appropriate combination. In particular, method aspects may be applied to apparatus aspects, and vice versa.

[0070] Furthermore, any, some and / or all features in one aspect can be applied to any, some and / or all features in any other aspect, in any appropriate combination. It should also be appreciated that particular combinations of the various features described and defined in any aspects of the present disclosure can be implemented and / or supplied and / or used independently. As used herein, means plus function features may be expressed alternatively in terms of their corresponding structure, such as a suitably programmed processor and associated memory.

[0071] The disclosure also provides a computer program or a computer program product for carrying out any of the methods described herein, and / or for embodying any of the apparatus features described herein. The disclosure also provides a non-transitory computer readable medium having stored thereon a program for carrying out any of the methods described herein and / or for embodying any of the apparatus features described herein.

[0072] The disclosure also provides a signal embodying a computer program or a computer program product for carrying out any of the methods described herein, and / or for embodying any of the apparatus features described herein, a method of transmitting such a signal, and a computer product having an operating system which supports a computer program for carrying out the methods described herein and / or for embodying any of the apparatus features described herein.

[0073] Furthermore, features implanted in hardware may generally be implemented in software, and vice versa. Any reference to software and hardware features herein should be construed accordingly.

[0074] One or more aspects will now be described, by way of example only and with reference to the accompanying drawings having I ike- reference numerals, in which:

[0075] Figure 1 is a diagram showing an exemplary simplified graph database structure;

[0076] Figure 2 is a diagram showing another exemplary simplified graph database structure;

[0077] Figure 3a is a flowchart showing the method for querying data stored in the graph database;

[0078] Figure 3b is a flowchart showing the method for querying, with a multi-segment query, data stored in the graph database; and

[0079] Figure 4 is a process flow diagram showing the method for querying data stored in the graph database.

[0080] Detailed description

[0081] The present disclosure provides a method for natively storing data in a graph database and a method for querying data in a graph database. A simplified exemplary graph database is described below with reference to Figures 1 and 2, and this exemplary graph database is used throughout this disclosure to illustrate the methods of storing and querying data in the database. The method for querying data in the database is described below with reference to Figure 3a and 3b, and an overall process flow is described below with reference to Figure 4. Graph databases

[0082] Figure 1 is a diagram showing an exemplary simplified graph database structure. The graph database in this example may be (part of) a knowledge graph and is used to store information about films. This exemplary graph database comprises three types of nodes (Person, Film, and Country) connected by three types of edges (starredln, directed, and producedin) representing the relationships between the nodes. A “starredln” edge may connect a person node to a film node to indicate that the person starred in that film. A “directed” edge may also connect a person node to a film node to indicate that the person directed that film. A “producedin” edge may connect a film node to a country node to indicate that the film was produced in that country.

[0083] In this example each of the edges is unidirectional because the three types of relationships represented by the edges are one-way relationships. It should also be noted that the directions of the edges have been chosen to fit the semantic definitions of the edges. For example, the “directed” edges point in this example from a person node to a film node to fit the semantic definition “Person X directed Film Y”. It would be equally possible to reverse the direction of this edge and change the semantic definition of the edge to “directedBy” to fit a different semantic definition such as “Film Y was directed by Person X”.

[0084] Figure 2 is a diagram showing another exemplary simplified graph database structure, which is a more specific example of the general graph database shown in Figure 1. In the graph database shown in Figure 2 the person, film, and country nodes have been specified along with their respective node identification (ID) numbers in brackets. The node ID numbers are the unique identifiers associated with each node and are used to store the nodes. In this example, the node ID numbers are simply monotonically increasing integers assigned to each node.

[0085] In Figure 2, the graph database includes five person nodes:

[0086] • Tom Hanks (node ID: 1)

[0087] • Matt Damon (node ID: 2)

[0088] • Steven Spielberg (node ID: 3)

[0089] • Robert Zemeckis (node ID: 4)

[0090] • Doug Liman (node ID: 5)

[0091] The graph database also includes three film nodes:

[0092] • Saving Private Ryan (node ID: 6)

[0093] • Forest Gump (node ID: 7) The Bourne Identity (node ID: 8)

[0094] Finally, the graph database includes one country node:

[0095] • United States of America (node ID: 9)

[0096] The edges of the graph database can also be assigned unique edge identification numbers to identify the edges. In this example the edge ID numbers are also monotonically increasing integers assigned to each edge in brackets in Figure 2. Each edge may additionally or alternatively be identified by the nodes it connects; for example, the edge can be identified by the node ID for its source (i.e., the node from which the edge originates) and the node ID for its target (i.e., the node at which the edge terminates).

[0097] In Figure 2, the graph database includes three “starredln” edges:

[0098] • Tom Hanks - starredln - Saving Private Ryan (edge ID: 1 ; source 1 , target 6)

[0099] • Tom Hanks - starredln - Forrest Gump (edge ID: 2; source 1 , target 7)

[0100] • Matt Damon - starredln - Bourne Identity (edge ID: 3; source 2, target 8)

[0101] The graph database also includes three “directed” edges:

[0102] • Saving Private Ryan <- directed - Steven Spielberg (edge ID: 4; source 3, target 6)

[0103] • Forrest Gump <- directed - Robert Zemeckis (edge ID: 5; source 4, target 7)

[0104] • Bourne Identity <- directed - Doug Liman (edge ID: 6; source 5, target 8)

[0105] Finally, the graph database includes three “producedin” edges:

[0106] • Saving Private Ryan - producedin - United States of America (edge ID: 7; source 6, target 9)

[0107] • Forrest Gump - producedin - United States of America (edge ID: 8; source 7, target 9)

[0108] • Bourne Identity - producedin - United States of America (edge ID: 9; source 8, target 9)

[0109] A simple query of a graph database may involve a user asking for a set of nodes that match some constraints. For example, a database query written in the Cypher / GQL language reading “MATCH (a:Person) RETURN a;” would return all nodes in the database of type “Person”. In the exemplary graph database shown in Figure 2, this query would return nodes 1 to 5. A slightly more complex query may involve a user asking for the sets of nodes and edges which fulfil certain relationship criteria. In this case, the query would span an edge and the user might input criteria for the edge as well as the nodes on either side of that edge. For example, in the exemplary graph database shown in Figure 2, the query: “MATCH p=(a:Person)-[e:starredln]->(b:Film) RETURN p;” would return all sets of nodes and edges which represent a person who starred in a film, that is:

[0110] • Tom Hanks - starredln - Saving Private Ryan

[0111] • Tom Hanks - starredln - Forest Gump

[0112] • Matt Damon - starredln - The Bourne Identity.

[0113] A similar query could be used for a graph database representing a social network. For example, the query “MATCH p=(a:Person)-[e:knows]->(b:Person {name:"Joe Bloggs"}) RETURN p;” would return all person nodes in the social network who know Joe Bloggs. It is important to note that in this example the first node in the query “(a:Person)” is not limited to any particular person, it is just limited to any node of the type “Person”. In contrast, the second node in the query “(b:Person {name:"Joe Bloggs"})” is limited to a particular person, Joe Bloggs, rather than all nodes of the type “Person”.

[0114] A yet more complex query can span multiple edges or search for branched relationships. An exemplary query spanning multiple edges may be “p=(a:Person)-[e1 :knows]->(b:Person)- [e2:bornln]->(c:Country) RETURN p;” which will return results where a first person knows a second person who was born in a certain country. An exemplary query for branched relationships may be “MATCH p=(a:Person)-[e1 :knows]->(b:Person)-[e2:bornln]- >(c:Country), (b)-[e3:hasPet]->(d:Animal)” which will return results where a first person knows a second person who was born in a certain country, and where the second person has a pet which is a certain animal; in this example, there are two relationships branching from the second person node “(b:Person)”.

[0115] An important aspect of querying a graph database is that the results of the query are essentially pathways through the graph database. These pathways represent sub-sections or ‘sub-graphs’ of the graph database which fulfil the criteria of the query. It should also be noted that the query itself resembles a pathway through the graph database in that it specifies nodes connected by edges. For queries spanning multiple edges, the query can be broken down into segments, with each segment of the query containing one edge. For example, the query “p=(a:Person)-[e1 :knows]->(b:Person)-[e2:bornln]->(c:Country) RETURN p;” can be broken down into the following two segments: “(a:Person)-[e1 :knows]->(b:Person)” and “(b:Person)- [e2:bornln]->(c:Country)” where the central node in the query “(b:Person)” is common to both segments.

[0116] The exemplary graph databases shown in Figures 1 and 2 are highly simplified for the purposes of clarity and brevity. In practice, graph databases are very large, typically containing potentially millions if not billions of nodes and edges.

[0117] Bitsets as native graph data storage

[0118] In the present disclosure, node adjacency lists in the graph database are natively stored using bitsets.

[0119] Bitsets (also known by other terms including bit arrays, bitmaps, and bit vectors) are an efficient and compact data structure for storing information in strings of binary bits (“0” or “1”). Modern computers typically process bitsets in the form of 32-bit or 64-bit words (i.e., strings of 32 or 64 bits). Various bitwise operations can be performed on bitsets, such as bitmask, intersection, and union operations, and such operations can be performed extremely quickly. Bitset compression algorithms, such as Roaring Bitmaps, can be used to compress bitsets for efficient storage; such compression algorithms are particularly effective for dense bitsets (i.e., containing primarily “1” bits) or sparse bitsets (i.e., containing primarily “0” bits).

[0120] In the present disclosure, bitsets are used to represent sets of nodes in the graph database. This is achieved by generating bitsets wherein the value of a bit in the bitset depends on whether a corresponding node is part of the set. For example, the value of bits in the bitset is set to “1” or “0” in dependence on whether a given node is or is not contained in a set. One way to achieve this is to assign unique identifiers to each node in the graph database, such as monotonically increasing integer numbers, and setting the value of a bit in a given position in the bitset to “1” or “0” depending on whether the node with the unique identifier corresponding to that position is or is not contained in the set.

[0121] The following description of this data structure refers to some examples using 8-bit words (i.e., bytes). This is for simplicity and brevity; in practice it is preferred to construct bitsets of (arrays of) a size that suits the native hardware word size to optimise for processing (specifically, to optimise for hardware that natively supports bitset operations such as graphics processing unit (GPU) Textures).

[0122] Suppose a graph database contains 8 nodes, each of which is assigned a monotonically increasing integer as its unique identifier. The 8 nodes would be represented by the following set of unique identifiers: 1 , 2, 3, 4, 5, 6, 7, 8. The set of the 8 nodes can then be represented by an 8-bit bitset where each bit in the bitset represents one of the unique identifiers: the first bit represents the node with identifier “1”, the second bit represents the node with identifier “2”, and so on. Therefore, to generate a bitset to identify a particular node, or a particular set of nodes, the value of the bit in the position corresponding to the particular node is set to “1” while the other bits remain “0”. For example, to generate a bitset containing the nodes with identifiers 3 and 6, the third and sixth bits in the bitset would be set to “1”, as follows:

[0123] Bitset: 0 0 1 0 0 1 0 0

[0124] Identifiers: 1 2 3 4 5 6 7 8

[0125] In other examples, the unique integer identifiers may start from “0” rather than “1”, in which case the first bit in the bitset represents the node with identifier “0”, the second bit represents the node with identifier “1”, and so on.

[0126] Given two bitsets, it is possible to carry out bitwise operations such as intersection (“n”) and union (“u") operations. An intersection of first and second bitsets returns a third bitset containing only the elements that are part of both the first and second bitsets. For example:

[0127] Bitset 1 : 0 0 0 1 0 0 1 0

[0128] Bitset 2: 0 1 0 1 0 0 0 0

[0129] Intersection: 0 0 0 1 0 0 0 0

[0130] A union of first and second bitsets returns a third bitset containing all elements that are part of either the first or second bitset. For example:

[0131] Bitset 1 : 0 0 0 1 0 0 1 0

[0132] Bitset 2: 0 1 0 1 0 0 0 0

[0133] Union: 0 1 0 1 0 0 1 0

[0134] This data structure can be extended beyond 8-bit bytes to much larger numbers by stacking bytes in an array. For example, 8 bytes stacked into a 64-bit array can be used to represent integers 1 to 64 inclusive. The intersection and union operations can be calculated by iterating over the bytes of the arrays (i.e., for an intersection operation, the first byte of a first array is intersected with the first byte of the second array, the second bye of the first array is intersected with the second byte of the second array, and so on). For example, the intersection

[0135] (“n”) of a first 64-bit bitset identifying the numbers 6, 33, and 57 and a second 64-bit bitset identifying the numbers 3, 6, 9, 19, 24, 37, 40, 41 , 42, 43, 49, 50, 57, 59, 63 can be represented in binary form as follows:

[0136] The bitset resulting from this intersection identifies only the numbers 6 and 57 which are the only numbers identified by both the first and second bitsets. Union operations can also be computed over such bitsets.

[0137] Extending this concept further, rather than using 8-bit words, as set out in the example above, 64-bit words can be used. If using 64-bit words, the above intersection can be computed with a single intersection operation rather than in 8 separate iterated intersection operations. Very large bitsets can be formed of stacked 64-bit words.

[0138] Storage of nodes - example

[0139] Applying this native graph storage method to the exemplary graph database shown in Figure 2, which contains 9 nodes, an empty bitset (using, for the sake of example, two 8-bit words) representing the 9 node identifiers would be:

[0140] Bitset: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

[0141] Identifiers: 1 2 3 4 5 6 7 8 9 -

[0142] For the graph database shown in Figure 2, the nodes of the type “Person” have unique identifiers 1 to 5, the nodes of type “Film” have unique identifiers 6 to 8, and the node of the type “Country” has the unique identifier 9. Therefore, bitsets for all Person, Film and Country nodes can be represented respectively as follows:

[0143] Person node bitset: 1111100000000000

[0144] Film node bitset: 0000011100000000

[0145] Country node bitset: 0000000010000000

[0146] Identifiers: 123456789 -

[0147] These bitsets can be maintained and updated whenever new nodes are added to the graph database. For example, if a new Film node was added to the graph database, with a unique identifier 10, the value of the 10thbit in the Film bitset would be changed from “0” to “1”, and the value of the 10thbit in the Person and Country node bitsets would remain “0”.

[0148] Native graph storage of edges

[0149] Similar bitset representations can be used to natively store sets of edges of the graph database. In the present disclosure, bitsets are used to represent sets of edges in the graph database. This is achieved by generating bitsets wherein the value of a bit in the bitset depends on whether a corresponding edge is part of the set. For example, the value of bits in the bitset is set to “1” or “0” in dependence on whether a given edge is contained in a set. One way to achieve this is to assign unique identifiers to each edge in the graph database, such as monotonically increasing integer numbers, and setting the value of a bit in a given position in the bitset to “1” or “0” depending on whether the edge with the unique identifier corresponding to that position is or is not contained in the set.

[0150] In particular, the node adjacency lists are stored as follows:

[0151] • For each node, the set of "in" edges which point at that node as a target. This set of graph edges can be represented as a bitset as described above.

[0152] • For each node, the set of "out" edges which point from that node as a source. This set of graph edges can also be represented as a bitset as described above.

[0153] In addition, the graph database may include a collection of key-value pairs to map the graph edges to their source and target nodes. In particular, the graph database may include the following maps:

[0154] For each edge, a map identifying its source node (that is, the node from which the edge starts) • For each edge, a map identifying its target node (that is, the node at which the edge ends)

[0155] Using these bitsets and maps, it is possible to convert between a bitset representing a set of nodes to a corresponding bitset representing a set of associated edges . For example, these bitsets and maps can be used to obtain, from a bitset representing a set of nodes, a bitset representing the set of edges which point out from that set out nodes.

[0156] The bitset of "in" edges for each node identifies the set of edges which point at that node as a target. For example, if a node is the target of edges with the unique identification numbers “1 ” and “2”, the bitset representing the “in” edges forthat node will have the first and second bits set to “1” and all other bits set to “0”.

[0157] The bitset of "out" edges for each node identifies the set of edges which point from that node as a source. For example, if a node is the source of edges with the unique identification numbers “3” and “4”, the bitset representing the “out” edges for that node will have the third and fourth bits set to “1” and all other bits set to “0”.

[0158] Native storage of edges - example

[0159] With reference to the exemplary graph database in Figure 2, the bitsets representing the “in” edges for each node are computed as:

[0160] Nodel: 000000000 / / No edges point to node 1

[0161] Node 2: 000000000 / / No edges point to node 2

[0162] Node 3: 000000000 / / No edges point to node 3

[0163] Node 4: 000000000 / / No edges point to node 4

[0164] Node 5: 000000000 / / No edges point to node 5

[0165] Node 6: 100100000 / / Edges 1 and 4 point to node 6

[0166] Node 7: 010010000 / / Edges 2 and 5 point to node 7

[0167] Node 8: 001001000 / / Edges 3 and 6 point to node 8

[0168] Node 9: 0 00000 111 / / Edges 7, 8 and 9 point to node 9

[0169] The bitsets representing the "out" edges for each node are:

[0170] Node 1: 110000000 / / Edges 1 and 2 point from node 1 Node 2: 001000000 / / Edge 3 points from node 2

[0171] Node 3: 000100000 / / Edge 4 points from node 3

[0172] Node 4: 000010000 / / Edge 5 points from node 4

[0173] Node 5: 000001000 / / Edge 6 points from node 5

[0174] Node 6: 000000100 / / Edge 7 points from node 6

[0175] Node 7: 000000010 / / Edge 8 points from node 7

[0176] Node 8: 000000001 / / Edge 9 points from node 8

[0177] Node 9: 000000000 / / No edges point from node 9

[0178] The maps identifying the source node for each edge are, as key-value pairs:

[0179] / / Node 1 is the source of edge 1 / / Node 1 is the source of edge 2 / / Node 2 is the source of edge 3 / / Node 3 is the source of edge 4 / / Node 4 is the source of edge 5 / / Node 5 is the source of edge 6 / / Node 6 is the source of edge 7 / / Node 7 is the source of edge 8

[0180] / / Node 8 is the source of edge 9

[0181] The maps identifying the target node for each edge are, as key-value pairs:

[0182] / / Node 6 is the target of edge 1

[0183] / / Node 7 is the target of edge 2

[0184] / / Node 8 is the target of edge 3

[0185] / / Node 6 is the target of edge 4

[0186] / / Node 7 is the target of edge 5

[0187] / / Node 8 is the target of edge 6

[0188] / / Node 9 is the target of edge 7

[0189] / / Node 9 is the target of edge 8 / / Node 9 is the target of edge 9

[0190] These bitsets and maps can be used during the process for querying data stored in the graph database as described in more detail below.

[0191] Querying data

[0192] When querying data from a graph database, the structure of the query (typically written using the Cypher or GQL languages) takes the form of a pattern of nodes and edges. Therefore, the result of the query will itself be a pattern of graph nodes and graph edges which represent sub-graphs of the graph database matching the query pattern. The query may include certain criteria associated with any of the nodes or edges in the query which limit the query to graph nodes and edges satisfying those criteria. For example, the query may specify nodes / edges of a certain type or nodes / edges having a certain property. The output of a graph database query is one or more sub-graphs of the graph database representing each and every set of nodes and edges which matches the structure of the query and satisfies the criteria of the query.

[0193] Figure 3 shows a flowchart representing a method 300 for querying a graph database according to the present disclosure. The method 300 will be described with reference to the exemplary query pattern “p=(a:Person)-[e1 :knows]->(b:Person)-[e2:bornln]->(c:Country) RETURN p” discussed in the graph databases section above. Throughout the description of the querying method, references to “query nodes” and “query edges” are used to mean the nodes and edges specified in the query pattern - for example, “(a:Person)” is a query node of the query pattern above, and “[e1 :knows]” is a query edge of the pattern above. These query nodes and query edges are distinct from the “graph nodes” and “graph edges” which refer to the actual nodes and edges in the graph database.

[0194] In the first step 302, the query is received. The query may be received from a user input. Typically, the query may be written by a user in the Cypher / GQL language. Alternatively, the user may interact with a user interface having fields in which the user can enter node and / or edge criteria, and a query in Cypher / GQL can then be automatically generated based on the information entered by a user via the user interface.

[0195] In the second step 304, the query is segmented. Each segment of the query includes a first query node and a query second node connected by a query edge between the first and second query nodes. Therefore, the exemplary query set out above can be broken down into two segments: “(a:Person)-[e1 :knows]->(b:Person)” and “(b:Person)-[e2:bornln]->(c:Country)”.

[0196] The first segment comprises a first query node “(a:Person)”, a second query node “(b:Person)”, and a first query edge “[e1 :knows]” between the first and second query nodes.

[0197] For the first segment each of the following bitsets is computed (or retrieved, if it has already been computed and stored):

[0198] 1 . A bitset representing graph nodes satisfying the criteria for the first query node (for the query set out above, in which the first query node is “(a:Person)” that would be a bitset representing all nodes of the type “Person”)

[0199] 2. A bitset representing graph edges satisfying the criteria for the query edge (for the query set out above, in which the first query node is “[e1 :knows]” that would be a bitset representing all edges of the type “knows”)

[0200] 3. A bitset representing graph nodes satisfying the criteria for the second query node (for the query set out above, in which the second query node is “(b:Person)” that would be a bitset representing all nodes of the type “Person”, which would be the same bitset as was computed or retrieved in point 1 above)

[0201] These three bitset computations or retrievals are labelled steps 306, 308 and 310 respectively in Figure 3, however in practice these computations / retrieval may be carried out in a different sequence or in parallel, rather than in the sequence shown in Figure 3.

[0202] In order to then execute bitwise operations between these three bitsets, it is necessary to convert the bitsets into like quantities (i.e., to convert the bitsets into sets which represent only edges, or only nodes). This is achieved by converting graph node bitsets to corresponding graph edge bitsets (step 312). This conversion is carried out using the bitsets of "in" and “out” edges for each node and / or the maps of “source” and “target” nodes for each edge as described above under the heading “Native graph storage of edges”. As used in this disclosure, references to “converting” a set of graph nodes to a set of graph edges are intended to mean determining a set of edges that are associated with a set of nodes (for example, obtaining the set of edges which point out from, or which point to, the set of nodes).

[0203] To convert a graph node bitset to a corresponding graph edge bitset, the adjacency lists for each node identified in the graph node bitset are retrieved. Then, the union of the computed / retrieved “in” and / or ’’out” graph edge bitsets is computed. The resultant bitset identifies the set of edges pointing into or out of, as the case may be, the nodes identified in the graph node bitset. In the example query set out above, the graph node bitset for “(a:Person)” may be converted to a bitset of edges pointing out from any Person nodes, because “(a:Person)” is the source query node in the query, and the graph node bitset for “(b:Person)” may be converted to a bitset of edges pointing into any Person nodes, because “(b:Person)” is the target query node in the query.

[0204] Finally, once the bitsets for the query nodes and edges have been computed and, where necessary, converted into like quantities, the intersection of the bitsets is taken (step 314). The resultant bitset identifies the full set of edges that satisfy the criteria of the segment of the query.

[0205] When the graph node bitsets are converted into graph edge bitsets, three bitsets will be obtained as follows:

[0206] 1 . A bitset of all graph edges which point out from graph nodes that fulfil the criteria of the first query node (in the example above, this will be a bitset of all graph edges which point out from graph nodes of the type “Person”) - for reference, bitset “Ei”

[0207] 2. A bitset of all graph edges which fulfil the criteria of the query edge (in the example above, this will be a bitset of all graph edges of type “knows”) - for reference, bitset »l— 11 t2

[0208] 3. A bitset of all graph edges which point into graph nodes that fulfil the criteria of the second query node (in the example above, this will be a bitset of all graph edges which point into nodes of the type “Person”) - for reference, bitset “E3”

[0209] To find the set of edges satisfying all criteria of the first segment of the query, the intersection of each of these three bitsets is computed (that is, Ei n E3n E3) to obtain an output bitset (E4) for the first segment of the query.

[0210] For a query consisting of only a single segment, the bitset (E4) resulting from step 314 is sufficient to determine the final results of the query by finding, using the node-edge maps, the set of nodes which are sources for the edges in E4and the set of nodes which are targets for the edges in E4; this then gives the intermediate representation, IR (i.e., sets of graph nodes and edges) which match the criteria of the query segment. The corresponding data may then be retrieved from the graph database to satisfy the query.

[0211] For a query comprising multiple segments, it is necessary to proceed further by repeating the steps 306 to 314 in Figure 3 for each additional segment of the query. The onward procedure for a multi-segment query is set out in Figure 4. Given that the second query node of the first segment becomes the first query node of the second segment (and so on), the search space can be reduced by carrying the bitset of graph nodes satisfying the criteria of the second query node of the first segment (as found when processing the first segment of the query) and using it as the graph node bitset for the first query node of the second segment (i.e., step 306 of processing the second segment). In this way, any graph nodes that were eliminated as candidates for the query node due to the constraints of the first segment will already be eliminated from consideration while processing the second segment, thereby reducing the search space. Otherwise, the steps 306 to 314 are repeated as described above for the second and subsequent segments of the query.

[0212] Accordingly, repeating the steps 306 to 314 for each segment of the query involves traversing the query pattern from a starting query node along adjacent query edges and query nodes in the pattern to a final query node in the pattern. Each time a segment of query nodes and / or query edges in the query pattern is visited during the traversal, the sets of graph nodes or graph edges associated with the graph nodes or graph edges which satisfy the criteria of the query nodes or query edges in the segment are identified, and the intersection of the sets is computed to obtain an output set for the segment.

[0213] It is possible that some of the nodes identified as satisfying the criteria of the first segment of the query do not satisfy the criteria of the second (or subsequent) segments of the query. For example, continuing with the exemplary query set out above (“p=(a:Person)-[e1 :knows]- >(b:Person)-[e2:bornln]->(c:Country) RETURN p;”), a graph node may satisfy the second query node of the first segment (“(b:Person)”) because it is a node of type “Person” and has an ”in” edge of type “knows” from another node of type “Person”. Therefore, that node will be identified in the bitset obtained while processing the first segment of the query. However, that node may not satisfy the criteria of the second segment of the query (for example, because it does not have an “out” edge of type “bornln”). To filter out such nodes or edges from the query results, once the query pattern has been fully traversed, descending from the starting query node to the final query node, the query pattern is then traversed again, but this time ascending from the final query node to the starting query node. When ascending the graph query, the output bitset resulting from processing the last segment of the query is propagated in step 320 back through the output bitsets obtained from processing earlier segments of the query to prune (i.e., remove) in step 322 any nodes and edges from the respective output bitsets which do not satisfy the criteria of subsequent segments of the query.

[0214] Referring again to the exemplary query above, this recursive traversal of the query pattern would result in the bitset representing graph nodes satisfying the query node “(b:Person)” being pruned to remove any nodes which do not satisfy criteria of the second segment of the query. Then, the bitset for the candidate graph edges for the query element “[e1 :knows]” is pruned to remove any edges which point only to nodes that were pruned from the “(b:Person)” bitset. Finally, the bitset for the candidate graph nodes for the query element “(a:Person)” is pruned to remove any nodes which had only “out” edges that were pruned from the “[e1 :knows]” bitset. The remaining nodes and edges then represent the result of the query, and the data corresponding to these remaining nodes and edges may then be retrieved from the graph database to satisfy the query. As used here, the term “pruning” it intended to mean removing graph edges or graph nodes from a set by setting to “0” the bits which correspond to those graph edges or graph nodes.

[0215] In the above example, the query pattern is traversed starting from the first segment appearing in the query (i.e., at the beginning of the query) and moving onwards to the last segment (i.e., at the end of the query). However, as an alternative, the querying method may instead begin by determining an optimal starting point for the query pattern traversal and the query pattern traversal may begin at that point. In one example, statistics about the graph database may be used to determine the optimal node at which to begin the query traversal. The optimal node may be the node that reduces the search space the most for the least (computational) cost. For example, for a query pattern which includes a query node which specifies a node with a particular property (such as a person with a specific name), it is likely that the number of graph nodes satisfying the criteria of that query node will be very small. Accordingly, it is most efficient to begin the query pattern traversal at this node and carry the reduced node / edge bitsets into subsequent segments of the query pattern. The statistics of the graph database may provide information of how many nodes / edges matching the criteria of a given element of a query pattern; for example, for query containing query nodes such as “(b:Person)” and “(c:Country)”, the statistics for the number of “Person” nodes compared to the number of “Country” nodes in the graph database may be used to determine which node would be a preferable starting point in the query pattern.

[0216] To avoid duplicating computations when recursively traversing the query pattern, the method may involve checking, when descending or ascending along the query pattern, whether a query edge has already been visited in that descent or ascent. If the query edge has not been visited already in the traversal, the relevant bitsets and bitset operations are computed forthat segment. If the query edge has already been visited in the query pattern traversal, the segment is skipped and the traversal moves to the next segment in the query pattern until all query nodes / edges in the query pattern have been visited. In this way, each query edge is only visited once during the descent of the query pattern, and once again during the ascent of the query pattern. Querying data - example

[0217] Applying this querying method to the exemplary graph database shown in Figure 2, a user may wish to find, for example, all films starring Tom Hanks and directed by Steven Spielberg. A corresponding Cypher query would reading “MATCH p=(actor:Person {name:"Tom Hanks"})-[s:starredln]->(f:Film)<-[d:directed]-(director:Person {name:"Steven Spielberg"}) RETURN p;” would be received (step 302).

[0218] This query can be segmented (step 304) into two segments as follows:

[0219] 1. (actor:Person {name:"Tom Hanks"})-[s:starredln]->(f:Film)

[0220] 2. (f:Film)<-[d:directed]-(director:Person {name:"Steven Spielberg"})

[0221] Starting from the first segment, the query pattern is traversed, descending from the first query node (actor:Person {name:"Tom Hanks"}) to the last query node (director:Person {name:"Steven Spielberg"}). the bitset representing graph nodes satisfying the criteria for the first query node of the first segment is obtained (step 306). In the exemplary graph database in Figure 2, the only graph node of type “Person” with the name property “Tom Hanks” is node 1. Accordingly, the bitset for the first query node of the first segment will have its first bit set to “1” with all other bits set to “0”. This bitset can be represented in shorthand as <1>.

[0222] Next, the bitset for edge nodes satisfying the criteria for the query edge of the first segment is obtained (step 308). In the exemplary graph database in Figure 2, the graph edges of type “starredln” are nodes 1 , 2, and 3. Accordingly, the bitset forthe query edge of the first segment will can be represented in shorthand as <1 , 2, 3> (i.e., a bitset with “1”s in the first, second, and third bits, with “0”s elsewhere).

[0223] Next, the bitset for graph nodes satisfying the criteria for the second query node of the first segment is obtained (step 310). In the exemplary graph database in Figure 2, the graph edges of type “Film” are nodes 6, 7, and 8. Accordingly, the bitset for the query edge of the first segment will can be represented in shorthand as <6, 7, 8> (i.e., a bitset with “1”s in the sixth, seventh, and eighth bits, with “0”s elsewhere).

[0224] The graph node bitsets must be converted to graph edge bitsets (step 312). In this example, the graph node bitsets (computed in steps 306 and 310) will be converted to graph edge bitsets; the graph edge bitset computed in step 308 does not need to be converted as it already represents a set of edges. Taking the bitset computed at step 306, the node with identifier 1 acts as the source node for edges with identifiers 1 and 2. Therefore, the corresponding graph edge bitset will be <1 , 2>. Taking the bitset computed at step 310, the nodes with identifiers 6, 7 and 8 act as target nodes for edges 1 , 2, 3, 4, 5 and 6. Therefore, the corresponding graph edge bitset will be <1 , 2, 3, 4, 5, 6>.

[0225] To summarise, the three edge bitsets for the first query segment are as follows:

[0226] Ei = <1 , 2>

[0227] E2= <1 , 2, 3>

[0228] E3= <1 , 2, 3, 4, 5, 6>

[0229] Taking the intersection (step 314) of these gives a first output bitset of <1 , 2> for the first query segment.

[0230] Accordingly, there are two edges (with identifiers 1 and 2) which satisfy the criteria of the first segment of the graph. Using the node-edge maps, we see that node 1 is the source of edges 1 and 2, and nodes 6 and 7 are the targets. Accordingly, after processing the first segment of the query, there are two sub-graphs which satisfy the criteria of the first segment of the query, and are candidates for satisfying the criteria of the whole query pattern:

[0231] • Tom Hanks - starredln - Saving Private Ryan

[0232] • Tom Hanks - starredln - Forrest Gump

[0233] As this is a multi-segment query, these steps are repeated by traversing the query onwards to the second segment. The graph node bitset for the second query node in the first segment (that is, <6, 7, 8>) is carried forward and used as the bitset for the first query node of the second segment (step 306).

[0234] Next, the bitset for edge nodes satisfying the criteria for the query edge of the second segment is obtained (step 308). In the exemplary graph database in Figure 2, the graph edges of type “directed” are nodes 4, 5, and 6. Accordingly, the bitset for the query edge of the first segment will can be represented in shorthand as <4, 5, 6>.

[0235] Finally, the bitset for graph nodes satisfying the criteria for the second query node of the second segment is obtained (step 310). In the exemplary graph database in Figure 2, the only graph node of type “Person” with the name property “Steven Spielberg” is graph node 3. Accordingly, the bitset for the second query node of the second segment will be <3>. Again, the graph node bitsets are converted to graph edge bitsets (step 312). Taking the first graph node bitset of the second segment, as set out above (E3for the first segment) is <1 , 2, 3, 4, 5, 6>. Taking the second graph node bitset of the second segment, the node 3 acts as a source node for edge 4 only. Therefore, the graph edge bitset will be <4>. To summarise, the three edge bitsets for the second segment are as follows:

[0236] Ei = <1 , 2, 3, 4, 5, 6>

[0237] E2= <4, 5, 6>

[0238] E3= <4>

[0239] Taking the intersection (step 316) of these gives a second output bitset <4> for the second query segment.

[0240] Accordingly, there is only a single edge (with identifier 4) which satisfies the criteria of the second segment of the graph. Using the node-edge maps, we see that node 3 is the source and node 6 is the target for edge 4. Accordingly, after processing the first segment of the query, there is only one sub-graph which satisfies the criteria of the second segment of the query:

[0241] • Saving Private Ryan <- directed - Steven Spielberg

[0242] Lastly, the result of processing the second segment of the query must be propagated back through the query pattern to prune the output from processing the first segment of the query. This is achieved by traversing the query again, this time ascending from the last query segment to the first query segment. In this case, as the query node “(f:Film)” is common to both the first and second segments of the query, the bitset of candidate nodes obtained for the “(f: Film)” query node in the first segment can be pruned to remove any graph nodes which are not targets of edge node 4, which is the only edge satisfying the criteria of the second segment of the query. This results in removing node 7 and edge 2 from the query results for the first segment. This leaves just one sub-graph which satisfies the global criteria of the query:

[0243] • Tom Hanks - starredln - Saving Private Ryan <- directed - Steven Spielberg

[0244] This single sub-graph is provided to the user as the result of the query. Assigning identifiers

[0245] In the description above, it should be noted that the integer unique identifiers assigned to each node and edge are not arbitrary; instead, nodes and edges of different types are assigned identifiers in blocks.

[0246] For example, it can be seen that the unique identifiers assigned to the nodes are arranged in blocks: all of the “People” nodes (Tom Hanks, Matt Damon, Steven Spielberg, Robert Zemeckis, and Doug Liman) are assigned identifiers in the range 1-5, while all of the “Film” nodes (Saving Private Ryan, Forest Gump, and The Bourne Identity) are assigned identifiers in the range 6-8, and the “Country” node (United States of America) is assigned identifier 9.

[0247] Similarly, it can be seen that the unique identifiers assigned to the edges are also arranged in blocks: all of the “starredln” edges are assigned identifiers in the range 1-3, while all of the “directed” edges are assigned identifiers in the range 4-6, and the “producedin” edges are assigned identifiers in the range 7-9.

[0248] Thus, the unique identifiers can be assigned in a way that groups together “like” types of nodes and edges, thereby incorporating additional information into the unique identifiers and enabling more efficient querying of the graph database.

[0249] For nodes, unique identifiers may be assigned in blocks of size N grouped by label (i.e., by the type of node, such as “Person” nodes or “Film” nodes, in the example above), such that any given block will only contain identifiers of nodes with the same label, and each block has an associated property - the “type” identifier, that identifies the label of nodes in that block. Exemplary sizes for N are multiples of 64 in the range 1 ,024 to 16,384 to balance space against computational efficiency. Each block of N identifiers has associated metadata identifying the property (e.g., the node label) associated with the nodes identified in the block. There may be multiple blocks of N for a given type of node if there are more than N nodes of that type in the graph database.

[0250] In this way, simple queries, such as a query for all “Person” nodes, can be satisfied simply by checking the metadata of each block, identifying the block(s) associated with the label “Person”, and retrieving the nodes identified by the identifiers in that block, ignoring any block that is not associated with the “Person” label. Therefore, it is not necessary to create and store indexes that include enough bits to potentially represent all of the nodes in the graph database (as would be necessary if the identifiers were not assigned in this way, for example if identifiers were simply assigned in the order in which the nodes are created). Instead, by assigning the identifier to the nodes in blocks, the same functionality can be derived from identifiers themselves.

[0251] Another advantage of this approach is that it enables combination queries to be performed more efficiently without additional memory allocation or significant compute cost. For example, to query for all “Person” OR “Film” nodes, it is necessary only to iterate through the blocks (i.e., check the metadata of each block), collect data from the blocks with a type property that matches either “Person” or “Film”, and ignore any blocks with a type property that doesn’t match “Person” or “Film". This contrasts with the multiple-bitsets approach, where it would be necessary to create a new bitset by generating the union of the “Person” and “Film” indexes described above, at the cost of additional memory and compute to perform the union operation.

[0252] For edges, the unique identifiers can again be assigned in blocks, but with different properties being associated with each block. Specifically, the properties associated with each block include the edge type and the types of the source and target nodes. For example, following the examples described above, a “starredln” edge will be stored in a block which has the associated properties of: “starredln” as the edge type; “Person” as the source node type; and “Film” as the target node type (since this type of edge will always point from a person to a film in which the person starred).

[0253] Again, assigning the identifiers to the edges in this way results in memory savings since it is again not necessary to maintain indexes of different edge types, each index having a size suitable to represent potentially every edge in the graph database (as the same functionality can be derived from the identifiers themselves). Moreover, using this approach, it is possible to perform certain queries more efficiently, as the identifiers allow for filtering edges not only by their own types, but also by the source and / or target node types. For example, given a pattern that does not specify the edge type, like “(p:Person)-[e]->(f:Film)”, it is possible to find the set of matching edges by filtering against the source and target types of each block of identifiers.

[0254] To assign identifiers to the nodes and edges in the way described in this section, the system may include an identifier generator responsible for managing the blocks of identifiers. When a new node or edge is stored in the graph database, the identifier generator may assign the next available identifier integer for a given label type (for example, if a new “Person” node is added, and the last added “Person” node received identifier “5”, the new node will be assigned “6”). If there are already N nodes or edges of a given type in the graph database, the identifier generator may generate a new block for that type of node or edge, and assign the new node or edge the first identifier from the new block. If a new node or edge is added to the graph database and there does not already exist any block associated with the type of node or edge that has been added, the identifier generator may generate a new block associated with that node or edge type (for example, if the graph database only includes “Person”, “Film” and “Country” nodes, but a new node of type “Producer” is added, the graph database will create a new block of “Producer” nodes and assign the first identifier from that block to the new node). The identifier generator may reserve ranges of possible unique identifiers for nodes and edges of different types; for example, the identifier generator may reserve the range 0 to 16,384 for nodes of type “Person”, and within this range may create up to 16 blocks of 1 ,024 identifiers which can be assigned to “Person” nodes each time such a node is added to the graph database.

[0255] This approach to assigning identifiers to nodes and edges gives rise to advantages when querying the database. Taking the exemplary query “MATCH (p:Person)-[e]->(f:Film)”, the process for satisfying this query in the case where identifiers are assigned arbitrarily to new nodes and edges added to the database would be as follows:

[0256] 1 . Obtaining the index representing the set of p:Person nodes, and converting to all of its outbound edges

[0257] 2. Obtaining the indexes for each edge type and intersecting them (or, an index representing edges of ALL types may have been pre-created, in which case this index can be used)

[0258] 3. Obtaining the index representing the set of f:Film nodes, and converting to all of its inbound edges

[0259] 4. Computing the intersection of the 3 bitsets obtained in steps 1 , 2 and 3 to obtain the final set “e”

[0260] 5. Finding the set of sources for those edges (which gives final set “p”)

[0261] 6. Finding the set of targets for those edges (which gives final set “f”)

[0262] In contrast, when the nodes and edges have been assigned identifiers in blocks as described in this section, the process for satisfying this query may involve just:

[0263] 1 . Finding “e” by checking the properties associated with of each block of edges (i.e., the edge type, source node type, and target node type), and identifying the block of edges having a source node type and target node type corresponding to the query (i.e., “Person” and “Film” respectively) 2. Find the set of sources for “e” (which gives final set “p”)

[0264] 3. Find the set of targets for “e” (which gives final set “f”)

[0265] It should be noted that the final two steps are the same in both cases, but in the latter case finding the set “e” involves significantly fewer operations and very little memory allocation, hence many queries can be dramatically accelerated by simultaneously filtering by the source, edge and target types and completely skipping the node to edge conversion and intersection steps.

[0266] Certain aggregation queries also benefit too, for example if the query were: “MATCH (p:Person)-[e]->(f:Film); RETURN count(e)” it would be necessary only to identify the appropriate block of edges (i.e., the same as step 1 above) and count how many bits are set to “1” in that block (indeed, if the block is full, this step can even be skipped, as the answer is simply “N”). This requires almost no memory allocation, and the filtered iteration and summation is extremely fast.

[0267] Process flow

[0268] Figure 5 is a diagram showing the overall process flow of the native graph data storage and querying methods set out in the present disclosure.

[0269] In the data storage phase, bitsets representing nodes and edges in a graph database are generated. These data structures may represent sets of nodes or edges of a particular type or sets of nodes or edges with a particular property. Generating these data structures in advance, and maintaining the data structures when new nodes and edges are added to the database, speeds up subsequent querying of the data because the relevant bitsets can simply be retrieved from memory rather than generated when required.

[0270] Querying data in the graph database involves a reduction phase followed by a row-production and projection phase. In the reduction phase the query pattern is recursively traversed (as set out in steps 306 to 322 above) and graph node and graph edge bitsets, which match the criteria of each element of the query, are generated or retrieved. Where necessary, the bitsets are converted into like quantities. In this example, the bitsets are converted to graph edge bitsets. The bitset intersection operations required to find the reduced bitsets which satisfy all elements of a query segment are carried out by a processor or multiple processors. Preferably, the processors are graphics processing units (GPUs) as these processors are optimised for handling bitset operations as the bitsets can be implemented as GPU Textures. However, even on standard central processing unit (CPU) the computational operations involved in the query method of the present disclosure are extremely fast. The bitset operations required during the reduction phase are “embarrassingly parallel”, meaning that little or no effort or coordination is needed to split the operations into a number of parallel tasks which can be executed by parallel processors. For example, a large intersection operation could be broken down into a series of smaller intersection operations that could be parallelised (e.g., by representing the smaller bitsets as bitmap textures and processing the intersection operations using parallel GPUs). Alternatively or additionally, each bitset intersection operation in the reduction phase could be computed on a different processor independently from one another.

[0271] If necessary, the results of the reduction phase for the final segment in the query pattern are propagated back through the query pattern to prune the results of the reduction phase for preceding segments in the query pattern (to remove any nodes or edges which do not meet the global criteria of the query) resulting in an intermediate representation that can now be used to efficiently satisfy the remaining parts of the query.

[0272] Then, in the row-production and projection phase, the full set of query results is obtained from the intermediate representation (in Figure 5, a reduced set of edges), using the node-edge maps to find the source and target nodes of each edge, thereby obtaining the final sub-graphs which satisfy the criteria of the query, and these sub-graphs are output.

[0273] It is important to note that although the above description refers to using binary bitsets to represent the sets of graph nodes and graph edges, and using bitwise intersection operations, this is just a particularly advantageous way of representing the sets of graph nodes and graph edges. The querying method of this disclosure could be implemented with the sets of graph nodes and graph edges represented in a different way, and with different set operations.

[0274] It should be noted that deriving the intermediate representation entails an up-front cost that existing row-expansion methods do not incur. The reduction process requires that for each pattern element (node or edge in the query pattern), every candidate graph node or edge must be assessed for inclusion in the intermediate representation. In any query that involves ordering or aggregating, the cost of deriving the intermediate representation is typically more than compensated by the optimisations it affords, since the algorithm ensures that (i) each graph node or edge will only be assessed at most once for any given pattern element (hence linear in the size of the query pattern and size of the graph), and (ii) many candidates are eliminated early by choosing the most selective segment to start from and carrying forward candidates from earlier pattern segments to reduce the search space, the cost of deriving the intermediate representation is typically small. This row agnostic approach contrasts with row-expansion methods where a node at the end of a pattern is likely to be repeatedly explored, to the extent that in a pattern comprised of 3 edges (and therefore 3 node-edge-node segments) where each graph node has an average of 10 adjacent matching edges, the exploration will visit the candidate graph nodes at the final pattern position on average 1 ,000 times each, hence the exponential growth factor of rowexpansion approaches as compared to the linear reduction to intermediate representation of the present disclosure. If the query entails additional predicates on that pattern node this can be particularly expensive, for example:

[0275] MATCH (a:Person {id:123})-[:knows]->(b:Person)-[:knows]->(c:Person)-[:knows]- >(d:Person)

[0276] WHERE d.name CONTAINS ‘ste’

[0277] RETURN a, b, c, d

[0278] Here, the expensive CONTAINS test will run approximately 1 ,000 times because there will be approximately 1 ,000 paths in the result set. Many paths can lead to the same node, which means many of the 1 ,000 tests will be due to repeated assessment of nodes that were already assessed during exploration of alternate paths. This is redundant work that the present approach avoids.

[0279] A particular advantage of the intermediate representation is that, once it has been derived, the total set of nodes and edges for each position in the query pattern is known. This allows many optimisations not available to row-expansion methods.

[0280] For example, depending on the query, it may be possible to entirely skip the row-production phase and project a result computed directly from the intermediate representation. An aggregation query such as “MATCH (a:Person {id:1234})-[:knows]->(:Person)-[:knows]- >(:Person)-[:knows]->(d:Person) RETURN count(distinct d);”, which finds the size of the 3- hop neighbourhood of the Person with id “1234”, can be satisfied directly from the intermediate representation - it is simply the size of the intermediate representation bitset for pattern element (d:Person). No path production is required since the result has already effectively been computed in deriving the IR. This is a good example of a query which is linear in the number of hops and size of the graph in our approach, and exponential in the number of hops and size of the graph for row-expansion engines, meaning that even for modest graphs (100,000 nodes, 1 ,000,000 edges), a 5-hop variant of this query can take many minutes to process by row-expansion, and a few hundreds of milliseconds for the present reductionbased approach. It should also be noted that the present disclosure affords the ability to produce the resulting sub-graphs I paths in order, by sorting the sets of IDs in the intermediate representation by the desired node or edge property, then generating the paths already in order by iterating those sorted sets, thus avoiding a huge amount of work that is unavoidable without the intermediate representation or expensive secondary indexes. Notably, each set of IDs is small compared with the number of paths, which means that sorting the sets is much faster than sorting the resulting sub-graph rows, and knowing all of the values before commencing row production means that no redundant sorting is performed, and dramatically cheaper sorting methods can be employed to further reduce the computational cost.

[0281] In particular when sorting by properties of multiple pattern elements, this is significantly more performant than the only option available to row-expansion engines, which must traverse all possible paths and insertion-sort or post-sort (if the query requires all matching paths be returned) or maintain a heap of the top N paths seen so far until all paths have been visited, and then sort the entries in the heap. Either way, all rows must be explored to find the top N, which can be exorbitantly expensive in large or densely connected graphs.

[0282] Further, when the query projection entails aggregation, all rows must be produced and bucketed according to the grouping criteria, before finally computing the aggregated result for each bucket. Row at a time approaches must retain all buckets until the last row is produced and manage them in a data structure such that they can be looked up by grouping key. There is a significant memory and associated compute cost to this. Contrastingly, the intermediate representation of the present disclosure allows rows to be cheaply produced in order of the grouping criteria, such that all rows for the same grouping key are produced consecutively, and when the grouping key changes the current bucket can be disposed of, meaning that only one bucket is resident at any given time. The reduced memory requirements and savings on memory management and bucket lookup can be very significant.

[0283] It should be noted that the advantages of computing the intermediate representation grow with the size and order of the graph and the size of the query pattern.

[0284] Each feature disclosed in the description, and (where appropriate) the claims and drawings may be provided independently or in any appropriate combination. Reference numerals appearing in the claims are by way of illustration only and shall have no limiting effect on the scope of the claims.

Claims

Claims1 . A computer implemented method of natively storing a set of nodes and / or edges in a graph database, comprising generating a bitset representing a set of graph nodes or graph edges, and setting the value of a bit in the bitset in dependence on whether a corresponding node and / or edge is part of the set.

2. A method according to Claim 1 , wherein the nodes and / or edges in the graph database each have an associated unique identifier.

3. A method according to Claim 2, comprising setting the value of a bit in the bitset in dependence on whether a node and / or edge having a unique identifier corresponding to that bit is part of the set.

4. A method according to Claim 2 or 3, comprising setting the value of a bit in the bitset in dependence on whether a node and / or edge having a unique identifier corresponding to the position of that bit in the bitset is part of the set.

5. A method according to any of Claims 2 to 4, wherein the unique identifiers for the nodes and / or edges in the graph database are integers, preferably monotonically increasing integers.

6. A method according to any of Claims 2 to 5, comprising assigning the unique identifiers to the nodes and / or edges in the graph database based on a property of the node and / or edge.

7. A method according to Claim 6, wherein assigning the unique identifiers comprises selecting a unique identifier from one or more of a plurality of ranges of possible unique identifiers based on a property of the node or edge, preferably wherein each range in the plurality of ranges corresponds to a different property.

8. A method according to Claim 6 or 7, wherein, for the nodes, the property comprises a label of the node.

9. A method according to any of Claims 6 to 8, wherein, for the edges, the property comprises one or more of: a property of a source node for the edge, such as a label ofthe source node; a property of a target node for the edge, such as a label of the target node; or a label of the edge.

10. A method according to any preceding claim, comprising, for a node in the graph database, generating a bitset representing the set of edges which point at that node as a target.

11. A method according to any preceding claim, comprising, for a node in the graph database, generating a bitset representing the set of edges which point from that node as a source.

12. A method according to any preceding claim, comprising, for an edge in the graph database, generating a bitset representing the source node(s) for that edge.

13. A method according to any preceding claim, comprising, for an edge in the graph database, generating a bitset representing the target node(s) for that edge.

14. A method according to any preceding claim, wherein the bitsets are formed of words having a length that is selected to suit hardware used to perform operations on the bitsets, preferably 64-bit or 128-bit words.

15. A computer implemented method of querying data in a graph database, the graph database comprising graph nodes connected by graph edges, the method comprising: receiving a query, the query comprising a first query node, a first query edge, and a second query node arranged in a pattern; for the first query node, identifying a first set of graph nodes or graph edges associated with the graph nodes which satisfy criteria of the first query node; for the first query edge, identifying a second set of graph nodes or graph edges associated with the graph edges which satisfy criteria of the first query edge; for the second query node, identifying a third set of graph nodes or graph edges associated with the graph nodes which satisfy criteria of the second query node; and computing the intersection of the first, second and third sets of graph nodes or graph edges to obtain one or more first output sets of graph nodes or graph edges.

16. A method according to Claim 15, wherein the query comprises a second query edge, and the method further comprises:for the second query edge, identifying a fourth set of graph nodes or graph edges associated with the graph edges which satisfy criteria of the second query edge; and computing the intersection of a set associated with the first output set and the fourth set of graph nodes or graph edges to obtain a second output set of graph nodes and / or graph edges.

17. A method according to Claim 16, wherein the query comprises a third query node, and the method further comprises: for the third query node, identifying a fifth set of graph nodes or graph edges associated with the graph nodes which satisfy criteria of the third query node; and computing the intersection of the set associated with the first output set the fourth and fifth sets of graph nodes or graph edges to obtain the second output set of graph nodes and / or graph edges.

18. A method according to Claim 16 or 17, comprising removing at least one graph node or graph edge from the first output set of graph nodes or graph edges based on the second output set of graph nodes or graph edges.

19. A method according to any of Claims 15 to 18, wherein the first query node and / or the first query edge are selected from among multiple query nodes and / or multiple query edges in the query based on statistics associated with the query nodes and / or query edges.

20. A method according to any of Claims 15 to 19, wherein the first and / or third set of graph nodes or graph edges comprises a set of graph edges which point to graph nodes which satisfy the criteria of the first and / or second query nodes respectively, and / or a set of graph edges which point from graph nodes which satisfy the criteria of the first and / or second query node respectively.21 . A method according to any of Claims 15 to 20, wherein the second set of graph nodes or graph edges comprises a set of graph nodes which are sources for graph edges which satisfy the criteria of the first query edge and / or a set of graph nodes which are targets for graph edges which satisfy the criteria of the first query edge.

22. A method according to any of Claims 15 to 21 , comprising:traversing the query pattern from a first query segment to a last query segment, each segment comprising at least one query node and at least one query edge; for each segment, identifying sets of graph nodes or graph edges associated with the graph nodes or graph edges which satisfy the criteria of the query nodes or query edges in the segment; and computing the intersection of the sets to obtain an output set for the segment.

23. A method according to Claim 22, comprising: traversing the query pattern from the last query segment to the first query segment; and based on the output set obtained from one segment, removing at least one graph node or graph edge from the output sets for obtained from at least one other segment.

24. A method according to any of Claims 15 to 23, comprising obtaining, from the output set(s) and node-edge maps, at least one output sub-graph of the graph database.

25. A method according to any of Claims 15 to 24, wherein the sets of graph nodes or graph edges are represented by bitsets wherein the value of a bit in the bitset depends on whether a corresponding graph node and / or graph edge is part of the set.

26. A method according to any of Claims 15 to 24, wherein computing the intersection comprises parallelising the intersection computation across multiple processors, preferably across multiple graphics processing units (GPUs).

27. A method according to any of Claims 15 to 26, comprising retrieving, from the graph database, data associated with the graph nodes or graph edges represented in the output set or output sub-graph.

28. A computer implemented method of querying data in a graph database, the graph database comprising nodes and / or edges natively stored according to the method of any of Claims 1 to 14, the method of querying data comprising: retrieving or identifying a bitset representing a set of graph nodes or graph edges; and retrieving, from the graph database, data associated with the graph nodes or graph edges represented in the bitset.

29. A computer implemented method according to Claim 28, comprising: retrieving or identifying a plurality of bitsets representing sets of graph nodes or graph edges; performing at least one set operation between two or more of the plurality of bitsets to obtain a resultant bitset; and retrieving, from the graph database, data associated with the graph nodes or graph edges represented in the resultant bitset.

30. A method according to Claim 28 or 29, comprising retrieving or identifying multiple bitsets representing multiple sets of graph nodes and / or graph edges, and performing at least one set operation between the bitsets to obtain an intermediate representation of graph nodes or graph edges which match query nodes and / or query edges.

31. A method according to Claim 30, comprising deriving from the intermediate representation one or more paths in the graph database matching a query path comprising the query nodes and / or query edges.

32. A method according to Claim 31 , comprising retrieving, from the graph database, data associated with the graph nodes and / or graph edges comprised in the one or more paths in the graph database derived from the intermediate representation.

33. A method according to any of Claims 15 to 32, wherein, for any of the query nodes, identifying the corresponding set of graph nodes which satisfy criteria of the query node comprises checking each graph node in the database against the query node at most once.

34. A method according to any of Claims 15 to 33, wherein, for any of the query edges, identifying the corresponding set of graph edges which satisfy criteria of the query edge comprises checking each graph edge in the database against the query edge at most once.

35. A computer program product comprising software code adapted, when executed on one or more processors, to carry out the method of any preceding claim.

36. A graph database comprising nodes and / or edges natively stored according to the method of any of Claims 1 to 14.

Images