Transactional multi-versioning of cached graph index enables updates

Abstract

A method can include, in response to a transaction operating on graph data, accessing a cache storing a graph index corresponding to the graph data. A cache miss triggered by a change to underlying graph data can be detected. In response to detecting the cache miss, the graph index can be updated at least by replaying or rewinding one or more other changes made by one or more other transactions to the graph data between a first time of the transaction and a second time of a current version of the graph index in the cache. The graph index can be updated to avoid a full rebuild of the graph index. The transaction can be run based on the updated graph index. Related systems and computer program products are also provided.

Description

Technical Field

[0001] The topics described in this article generally involve database processing, and more specifically, the updating of cached graph indexes. Background Technology

[0002] Databases can be configured to store data according to a database schema. For example, in a graph database, data can be represented and stored using a graph structure that includes, for example, vertices, directed edges, and undirected edges. It is worth noting that graph databases can explicitly store relationships between different data items. For example, the vertices of a graph can correspond to individual data items stored in the graph database, while the edges of the graph can define the relationships between these data items. Attributes associated with vertices and / or edges can provide additional properties of the relationships between data items and / or between different data items stored in the graph database. Conversely, relational databases can implicitly store relationships between different data items, for example, by organizing data items into one or more database tables. Relational databases can be configured to store graph data, for example, by storing the vertices of the graph in a vertex table and the edges of the graph in a separate edge table. Summary of the Invention

[0003] Systems, methods, and artifacts, including computer program products, are provided for updating cached graph indexes. In one aspect, a system is provided comprising at least one data processor and at least one memory. The at least one memory may store instructions that, when executed by the at least one data processor, cause an operation. The operation may include: accessing a cache storing graph indexes corresponding to the graph data in response to a transaction operating on graph data stored in a database; updating the graph indexes, at least by replaying or rewinding one or more changes made to the graph data by one or more other transactions between a first time of the transaction and a second time of the current version of the graph indexes in the cache, in response to detecting a cache miss; and running the transaction at least based on the updated graph indexes.

[0004] In some variations, one or more of the features disclosed herein, including the following characteristics, may optionally be included in any feasible combination. The execution of the transaction may include performing a graph processing algorithm based at least on the updated graph index, which includes one or more of the following: subgraph, complement graph, in-degree, out-degree, in-edge, out-edge, neighbor, reachability, shortest path, one to all shortest paths, k shortest paths, strongly connected components, depth-first search, or breadth-first search.

[0005] In some variations, graph indexing may include identifying a first vertex as an adjacency structure adjacent to a second vertex based at least on the first vertex being connected to the second vertex by one or more edges.

[0006] In some variations, one or more other transactions can modify the graph data at least by inserting vertices, deleting vertices, inserting edges, and / or deleting edges.

[0007] In some variations, cache misses can be triggered by modifications to graph data stored in a database.

[0008] In some variations, the operation may also include: performing multi-version concurrency control (MVCC) to track multiple transactions that modify graph data stored in the database.

[0009] In some variations, the operation may further include: maintaining a redo log that tracks multiple changes made to graph data stored in a database; and reading the redo log to replay or rewind one or more changes made to the graph data between a first time of the transaction and a second time of the current version of the graph index.

[0010] In some variations, the database may include a relational database that stores graph data in one or more vertex and edge tables.

[0011] In some variations, the operation may further include generating a graph index based on at least one or more vertex tables and edge tables.

[0012] In some variations, a database may include document storage.

[0013] In some variations, the graph index can be updated without rebuilding the graph index as a whole.

[0014] In some variations, updating a graph index may also include replacing the current version of the graph index in the cache with the updated graph index.

[0015] In another aspect, a method for updating a cached graph index is provided. This method may include: in response to a transaction operating on graph data stored in a database, accessing a cache storing graph indexes corresponding to the graph data; in response to detecting a cache miss, updating the graph indexes at least by replaying or rewinding one or more changes made to the graph data by one or more other transactions between a first time of the transaction and a second time of the current version of the graph indexes in the cache; and running the transaction at least based on the updated graph indexes.

[0016] In some variations, one or more of the features disclosed herein, including the following characteristics, may optionally be included in any feasible combination. The execution of a transaction may include performing a graph processing algorithm based at least on the updated graph index, which includes one or more of the following: subgraph, complement graph, in-degree, out-degree, in-edge, out-edge, neighbor, reachability, shortest path, a pair of all shortest paths, k shortest paths, strongly connected components, depth-first traversal, or breadth-first traversal.

[0017] In some variations, graph indexing may include identifying a first vertex as an adjacency structure adjacent to a second vertex based at least on the first vertex being connected to the second vertex by one or more edges.

[0018] In some variations, one or more other transactions can modify the graph data at least by inserting vertices, deleting vertices, inserting edges, and / or deleting edges.

[0019] In some variations, cache misses can be triggered by modifications to graph data stored in a database.

[0020] In some variations, the method may also include performing multi-version concurrency control (MVCC) to track multiple transactions that modify graph data stored in a database.

[0021] In some variations, the method may further include: maintaining a redo log that tracks multiple changes made to graph data stored in a database; and reading the redo log to replay or rewind one or more changes made to the graph data between a first time of the transaction and a second time of the current version of the graph index.

[0022] In another aspect, a computer program product comprising a non-transitory computer-readable medium including stored instructions is provided. The instructions enable at least one data processor to perform operations. Operations may include: accessing a cache storing a graph index corresponding to the graph data in response to a transaction operating on graph data stored in a database; updating the graph index, at least by replaying or rewinding one or more changes made to the graph data by one or more other transactions between a first time of the transaction and a second time of the current version of the graph index in the cache, in response to a detected cache miss; and performing the transaction, at least based on the updated graph index.

[0023] Implementations of the present subject matter may include, but are not limited to, methods consistent with those described herein, and articles comprising a tangibly embodied machine-readable medium operable to cause one or more machines (e.g., computers, etc.) to perform operations implementing one or more of the described features. Similarly, computer systems comprising one or more processors and one or more memories coupled to the one or more processors are also described. Memory that may include non-transitory computer-readable or machine-readable storage media may include one or more programs, encode one or more programs, store one or more programs, etc., causing one or more processors to perform one or more of the operations described herein. Computer-implemented methods consistent with one or more implementations of the present subject matter may be implemented by one or more data processors residing in a single computing system or multiple computing systems. Such multiple computing systems may be connected via one or more connections (including, for example, connections via networks (e.g., the Internet, wireless wide area networks, local area networks, wide area networks, wired networks, etc.), direct connections between one or more of the multiple computing systems, etc., and may exchange data and / or commands or other instructions, etc.

[0024] Details of one or more variations of the subject matter described herein are set forth in the accompanying drawings and the following description. Further features and advantages of the subject matter described herein will become apparent from the description, the drawings, and the claims. Although certain features of the currently disclosed subject matter are described in conjunction with the processing of the graphical data for illustrative purposes, it should be readily understood that these features are not intended to be limiting. The appended claims are intended to define the scope of the protected subject matter. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate certain aspects of the subject matter disclosed herein and, together with the specification, help explain some principles associated with the disclosed implementations. In the drawings,

[0026] Figure 1A A system diagram illustrating an example of a graph data processing system according to some example embodiments is provided.

[0027] Figure 1B A block diagram illustrating an example of a graph engine according to some example embodiments is depicted;

[0028] Figure 2 Examples of graph data according to some example embodiments are depicted;

[0029] Figure 3 A sequence diagram illustrating an example of a process for updating a cached graph index, according to some example embodiments;

[0030] Figure 4A flowchart illustrating an example of a process for graph processing using an updatable graph index, according to some example embodiments, is depicted; and

[0031] Figure 5 A block diagram illustrating an example computing system according to some example embodiments is depicted.

[0032] In practice, similar reference numerals in drawings indicate similar structures, features, or elements. Detailed Implementation

[0033] Relational databases storing graph data can support graph processing algorithms, including shortest path, risk propagation, minimum flow, and page rank. Efficient processing of graph data stored in relational databases may require the materialization of the graph index, such as enumerating the adjacency structure of each vertex's adjacent vertices and / or each vertex's adjacent edges (e.g., adjacency lists, adjacency matrices, etc.). In some cases, the efficiency of processing graph data stored in relational databases can be further maximized by ensuring that the graph index contains only graph data visible to the current transaction. When traversing graph data to run graph processing algorithms, excluding graph data not visible to the current transaction from the graph index avoids visibility checks.

[0034] In some example embodiments, graph indexes can be cached, allowing the same graph index to be reused by multiple transactions with the same transaction visibility properties. However, maintaining the currency of a graph index by rebuilding it for every modification to the underlying graph data (e.g., to include only graph data visible to the current transaction) can be resource-intensive, at least because a complete rebuild of the graph index is a computationally expensive operation. Thus, according to some example embodiments, the graph engine can be configured to update the cached graph index at least based on transaction version data to reflect modifications to the underlying graph data. For example, in response to a transaction at a first time t1, a cached graph index from a second time t2 (before or after the first time t1) can be updated by applying (or rewinding) changes made to the underlying graph data by one or more transactions between the first time t1 and the second time t2. Updating the cached graph index in this way maintains the currency of the cached graph index while avoiding a complete rebuild of the cached graph index. In doing so,

[0035] Figure 1A A system diagram illustrating an example of a graph data processing system 100 according to some example embodiments is depicted. References Figure 1AThe graph data processing system 100 may include a database 110 storing graph data 115, a database management system 120 including a graph engine 125, and a client device 130. The database 110, database management system 120, and client device 130 may be communicatively coupled via a network 150. It should be understood that the client device 130 may be a processor-based device, including, for example, a smartphone, tablet computer, wearable device, virtual assistant, Internet of Things (IoT) device, etc. The network 150 may be a wired network and / or a wireless network, including, for example, a Public Land Mobile Network (PLMN), a Wide Area Network (WAN), a Local Area Network (LAN), a Virtual Local Area Network (VLAN), the Internet, etc.

[0036] In some example embodiments, database 110 may be a relational database configured to store graph data 115, for example, in one or more vertex and edge tables. Database management system 120 may be configured to respond to requests for access to graph data 115 from one or more client devices, including, for example, client device 130. Figure 1A As shown, the client device 130 can send a request to the database management system 110 to run the graph processing algorithm 135, which derives a solution by manipulating the graph data 115 stored in the database 110.

[0037] To further illustrate, Figure 2 An example of graph data 115 stored in database 115 (e.g., in vertex and edge tables) is depicted. Figure 2 As shown, graph data 200 may include one or more vertices, including, for example, a first vertex A, a second vertex B, and a third vertex C. Furthermore, one or more vertices may be interconnected via one or more edges (including, for example, a first edge 210, a second edge 212, a third edge 214, a fourth edge 216, a fifth edge 218, and a sixth edge 220). Figure 2 In the example of graph data 115 shown, one or more edges are directed edges, but it should be understood that one or more edges can also be undirected edges that connect one or more vertices.

[0038] Graph processing algorithm 135 may include one or more graph processing functions, including, for example, subgraph, complement graph, in-degree, out-degree, incoming edge, outgoing edge, neighbor, reachability, shortest path, one to all shortest paths, k shortest paths, strongly connected components, depth-first traversal, breadth-first traversal, etc. To improve the efficiency of running graph processing algorithm 135 on graph data 115, graph engine 125 may materialize graph index 145. For example, graph engine 125 can run graph processing algorithm 135 by traversing graph data 115 based on graph index 145. An adjacency structure such as an adjacency list or adjacency matrix is ​​an example of graph index 145, which identifies a first vertex as adjacent to a second vertex based at least on the first vertex being connected to a second vertex by one or more edges. For example, in Figure 2 In the example of graph data 115 shown, the adjacency structure can identify the first vertex A as adjacent to the second vertex B, the third vertex C, and the first vertex A itself.

[0039] Graph index 145 can be cached, for example, in cache 140, allowing the same graph index 145 to be reused by multiple transactions. For example, in some cases, cache 140 can store a binary representation of graph index 145. Graph engine 125 can maintain the timeliness of graph index 145 to ensure that graph index 145 includes graph data visible to the current transaction and excludes graph data not visible to the current transaction. For example, for a transaction running at time t1, graph index 145 can include vertices and / or edges that were inserted by one or more other transactions before time t1 and were not deleted by the transaction until after time t1. Furthermore, graph index 145 can exclude vertices and / or edges that were deleted by one or more other transactions before time t1 and / or inserted after time t1.

[0040] As used herein, the term "transaction" can refer to database operations that read or write data via graph engine 125 (e.g., at database 110), run stored procedures, and / or run graph algorithms on graph data 115. For example, each transaction can have its own consistent view of the data stored at database 110, based on one or more atomicity, consistency, isolation, and durability (ACID) rules applied at database 110.

[0041] When traversing graph data 115 to run graph processing algorithm 135, excluding graph data not visible to the current transaction from graph index 145 can avoid visibility checks. However, maintaining the timeliness of graph index 145 in cache 140 by rebuilding graph index 145 entirely every time the underlying graph data 115 is modified can be resource-intensive, at least because a complete rebuild of graph index 145 is a computationally expensive operation. Thus, in some example embodiments, graph engine 125 can update cached graph index 145 at least based on transaction version data to reflect modifications to the underlying graph data. Therefore, in response to running a transaction at a first time t1, graph index 145 cached from a second time t2 (before or after the first time t1) can be updated by applying (or rewinding) changes associated with one or more other transactions that modified the underlying graph data 115 between the first time t1 and the second time t2 (e.g., by inserting and / or deleting one or more vertices and / or edges into and / or from graph data 115). Updating the cached graph index 145 in this way can maintain the timeliness of the cached graph index 145 while avoiding a complete reconstruction of the cached graph index 145.

[0042] Database management system 120 can implement multi-version concurrency control (MVCC) to support multiple concurrent transactions without imposing read-write locking. Alternatively, the database management system can, for example, track previous versions of graph data 115 by maintaining a redo log. In doing so, database management system 120 can maintain one or more logs of transactions operating on graph data 115, for example, by inserting and / or deleting one or more vertices and / or edges into and / or from graph data 115. Thus, in response to a transaction running at a first time t1, the cached graph index 145 from a second time t2 (before or after the first time t1) can be updated by applying (or rewinding) the recorded changes associated with one or more other transactions that modified the underlying graph data 115 between the first time t1 and the second time t2.

[0043] Updating the cached graph index 145 may include adjustments to cached entries, such as removing old cached entries for the graph index selected for the update and creating new cached entries with updated transaction visibility settings. For example, in response to a transaction running at a second time t2 after a first time t1, a cached version of graph index 145 from the first time t1 may be removed from the cache and replaced by an updated version of graph index 145. The updated version of graph index 145 may include graph data visible to the transaction running at the first time t1, including, for example, vertices and / or edges that were inserted by the transaction before the first time t1 and not deleted until after the first time t1 (e.g., data associated with transactions committed before the first time t1). Furthermore, the updated version of graph index 145 may exclude vertices and / or edges that were deleted by one or more other transactions after the first time t1 and before the second time t2 and / or inserted after the first time t1 and before the second time t2 (e.g., data associated with transactions committed between the first time t1 and the second time t2).

[0044] Figure 1B A block diagram illustrating an example of a graph engine 125 according to some example embodiments is depicted. References Figures 1A-1B The graph engine 125 may include a graph engine manager 152, which can respond to a request to load a graph by sending a corresponding request to the build manager 162 to create a graph index. Figure 1B In the example of graph engine 125 shown, graph engine 125 may include update manager 154, which can interact with multi-version concurrency control (MVCC) manager 156 and version pool 158 to update graph index 145 stored in cache 140, at least based on transactions that modify graph data 115 in database 110. Updating graph index 145 in this manner can provide several advantages. For example, reclaiming outdated versions of graph index 145 can minimize the memory footprint associated with cache 140, at least by limiting the number of copies of graph index 145 to the number of concurrent transactions with different visibility to graph data 115. Updating graph index 145 is also a more computationally efficient operation than a complete reconstruction of graph index 145. Thus, by avoiding a complete reconstruction of graph index 145 as much as possible, transaction runtime can be greatly reduced. Finally, updating graph index 145 can be achieved without a central graph index with dynamic transaction capabilities, which would require maintaining extremely complex data structures, additional transaction visibility checks for each transaction, and complex management to avoid race conditions during concurrent modifications to graph data 115.

[0045] Figure 3A sequence diagram illustrating an example of a process 300 for updating a cached graph index, according to some example embodiments. (Reference) Figures 1A-1B as well as Figure 3 Procedure 300 can be executed to update the graph index 145 stored in cache 140, based at least on a transaction that modifies the graph data 115 in database 110.

[0046] like Figure 3 As shown, at 302, graph engine 125 can generate graph index 145 based at least on graph data 115. In some cases, graph index 145 can be an adjacency structure (e.g., adjacency list, adjacency matrix, etc.) enumerating the adjacent vertices of each vertex and / or the adjacent edges of each edge in graph data 115. At 304, graph engine 125 can insert graph index 115 into cache 140. For example, in some example embodiments, cache 140 can store a binary representation of graph index 145.

[0047] At 306, graph engine 125 responds to the first graph script call by accessing cache 140. For example, the first graph script call may run graph processing algorithm 135 on graph data 115. If the transaction access of graph processing algorithm 135 exists in graph data at graph index 145 in cache 140, graph engine 125 may reuse graph index 145, for example, by traversing graph data 115 based on graph index 145, to run graph processing algorithm 135. Alternatively, at 308, graph engine 125 may respond to the second graph script call by accessing cache 140. However, in this case, another process 350 may have modified graph data 115 such that running graph processing algorithm 135 on graph data 115 can access graph data that does not exist in graph index 145 in cache 140. For example, at 310, another process 350 can modify graph data 115 by inserting and / or deleting one or more vertices and / or edges (e.g., "insertion key = 4711"). Alternatively, at 310, another process 350 can modify graph data 115 by updating the attributes of one or more vertices and / or one or more edges included in graph data 115. Thus, accessing graph index 145 by graph engine 125 after modification of the underlying graph data 115 can trigger a cache miss.

[0048] It should be understood that any modification to the underlying graph data 115 may trigger a cache miss in subsequent read transactions, regardless of the vertices or edges accessed (or attempted to access) by the read transaction. Entries in cache 140 (such as graph index 145) are created with transaction visibility information (which is active when the cache entry is created) and additional version information about the corresponding database table (which is also active when the cache entry is created). Thus, when the database table is modified (e.g., through a modification to the graph data 115), its version information is changed by database 110. At query time, the caching framework can use the information about transaction visibility and table state to determine whether a cache entry is appropriate for the current transaction's transaction context.

[0049] At 312, in response to a cache miss, graph engine 125 can determine the increment between a first state of graph data 115 associated with the version of graph index 145 in cache 140 and a second state of graph data 115 at database 110. Furthermore, at 314, graph engine 125 can update graph index 145 based on this increment. For example, a second graph script call may be received at a first time t1, while the cached graph index 145 may come from a second time t2 (before or after the first time t1). Therefore, graph engine 125 can update the version of graph index 145 in cache 140 by applying (or rewinding) changes associated with one or more other transactions between the first time t1 and the second time t2 that modify the underlying graph data 115 (e.g., by inserting and / or deleting one or more vertices and / or edges into and / or from graph data 115). At 316, graph engine 125 can replace the version of graph index 145 in cache 140 with the updated version of graph index 145.

[0050] Figure 4 A flowchart illustrating an example of a process 400 for graph processing using an updatable graph index, according to some example embodiments, is depicted. (Reference) Figures 1A-1B as well as Figures 3-4 In response to a request from client device 130 to run graph processing algorithm 135 on graph data 115 stored in database 110, process 400 can be executed by database management system 120 (e.g., graph engine 125).

[0051] At 402, graph engine 125 can receive transactions that operate on graph data. For example, graph engine 125 can receive a request from client device 130 to run a transaction that includes executing graph processing algorithm 135 on graph data 115 stored in database 110. Graph processing algorithm 135 may include one or more graph processing functions, including, for example, subgraph, complement graph, in-degree, out-degree, in-edge, out-edge, neighbor, is-reachable, shortest path, one to all shortest paths, k shortest paths, strongly connected components, depth-first traversal, breadth-first traversal, etc.

[0052] At a 404 error, graph engine 125 can respond to the transaction by accessing a cache to retrieve the graph index associated with the graph data. In some example embodiments, graph engine 125 can generate graph index 145 to improve the efficiency of running graph processing algorithm 135 on graph data 115. Graph index 145 can be, for example, an adjacency structure (e.g., adjacency list, adjacency matrix, etc.) that identifies a first vertex as adjacent to a second vertex, at least based on the first vertex being connected to a second vertex by one or more edges.

[0053] In 406, graph engine 125 can detect cache misses. In some cases, the version of graph index 145 stored in cache 145 may not be current, at least because one or more other transactions may have modified the underlying graph data 115 (e.g., by inserting and / or deleting one or more vertices and / or one or more edges into and / or from graph data 115).

[0054] At 408, graph engine 125 can respond to a cache miss by updating the graph index. In some example embodiments, graph engine 125 can maintain the timeliness of graph index 145 by updating graph index 145 instead of rebuilding graph index 145 as a whole. For example, in response to a cache miss triggered by a transaction running at a first time t1, graph engine 125 can update the cached graph index 145 from the second time t2 by applying (or rewinding) changes made to the underlying graph data 115 by one or more other transactions between the first time t1 and the second time t2 (before or after the first time t1).

[0055] At 410, graph engine 125 can perform the transaction based at least on the updated graph index. In some example embodiments, graph engine 125 can perform the transaction, for example, by traversing at least a portion of graph data 115 based on the updated graph index 145.

[0056] In view of the above-described implementation of the subject matter, this application discloses the following list of examples, wherein a feature of a single example or a combination of more than one feature of the example, and optionally, a combination with one or more features of one or more other examples, are other examples that also fall within the scope of this application:

[0057] Example 1: A system comprising: at least one data processor; and at least one memory storing instructions that, when executed by the at least one data processor, cause operations including: in response to a transaction operating on graph data stored in a database, accessing a cache storing a graph index corresponding to the graph data; in response to detecting a cache miss, updating the graph index at least by replaying or rewinding one or more changes made to the graph data by one or more other transactions between a first time of the transaction and a second time of the current version of the graph index in the cache; and running the transaction at least based on the updated graph index.

[0058] Example 2: The system of Example 1, wherein the execution of the transaction includes performing a graph processing algorithm based at least on the updated graph index, the graph processing algorithm including one or more of subgraph, complement graph, in-degree, out-degree, in-edge, out-edge, neighbor, reachability, shortest path, a pair of all shortest paths, k shortest paths, strongly connected components, depth-first traversal or breadth-first traversal.

[0059] Example 3: A system of any one of Examples 1 to 2, wherein the graph index includes an adjacency structure that identifies a first vertex as adjacent to a second vertex based at least on the first vertex being connected to a second vertex by one or more edges.

[0060] Example 4: A system of any one of Examples 1 to 3, wherein one or more other transactions modify the graph data at least by inserting vertices, deleting vertices, inserting edges, and / or deleting edges.

[0061] Example 5: A system of any of Examples 1 through 4, where a cache miss is triggered by a modification to graph data stored in a database.

[0062] Example 6: A system of any one of Examples 1 to 5, wherein the operation further includes: performing multi-version concurrency control (MVCC) to track multiple transactions that modify graph data stored in a database.

[0063] Example 7: A system of any one of Examples 1 to 6, wherein the operation further includes: maintaining a redo log that tracks multiple changes made to graph data stored in a database; and reading the redo log to replay or rewind one or more changes made to the graph data between a first time of the transaction and a second time of the current version of the graph index.

[0064] Example 8: A system of any one of Examples 1 to 7, wherein the database comprises a relational database that stores graph data in one or more vertex tables and edge tables.

[0065] Example 9: The system of Example 8, wherein the operation further includes: generating a graph index based on at least one or more vertex tables and edge tables.

[0066] Example 10: A system of any one of Examples 1 through 9, wherein the database includes document storage.

[0067] Example 11: A system of any of Examples 1 through 10, where the graph index is updated without being rebuilt as a whole.

[0068] Example 12: A system of any of Examples 1 through 11, wherein updating the graph index also includes replacing the current version of the graph index in the cache with the updated graph index.

[0069] Example 13: A computer-implemented method comprising: in response to a transaction operating on graph data stored in a database, accessing a cache storing a graph index corresponding to the graph data; in response to detecting a cache miss, updating the graph index at least by replaying or rewinding one or more changes made to the graph data by one or more other transactions between a first time of the transaction and a second time of the current version of the graph index in the cache; and running the transaction at least based on the updated graph index.

[0070] Example 14: The method of Example 13, wherein the execution of the transaction includes performing a graph processing algorithm based at least on the updated graph index, the graph processing algorithm including one or more of subgraph, complement graph, in-degree, out-degree, in-edge, out-edge, neighbor, reachability, shortest path, one to all shortest paths, k shortest paths, strongly connected components, depth-first traversal or breadth-first traversal.

[0071] Example 15: A method of any one of Examples 13 to 14, wherein the graph index includes identifying the first vertex as an adjacency structure adjacent to the second vertex based at least on the first vertex being connected to the second vertex by one or more edges.

[0072] Example 16: A method of any of Examples 13 to 15, wherein one or more other transactions have modified the graph data at least by inserting a vertex, deleting a vertex, inserting an edge, and / or deleting an edge.

[0073] Example 17: A method from any of Examples 13 through 16, where a cache miss is triggered by a modification to graph data stored in a database.

[0074] Example 18: The method of any of Examples 13 through 17 also includes: performing multi-version concurrency control (MVCC) to track multiple transactions that modify graph data stored in a database.

[0075] Example 19: The method of any of Examples 13 to 18 further includes: maintaining a redo log that tracks multiple changes made to graph data stored in a database; and reading the redo log to replay or rewind one or more changes made to the graph data between a first time of the transaction and a second time of the current version of the graph index.

[0076] Example 20: A non-transitory computer-readable medium storing instructions that, when executed by at least one data processor, cause operations including: in response to a transaction operating on graph data stored in a database, accessing a cache storing a graph index corresponding to the graph data; in response to detecting a cache miss, updating the graph index at least by replaying or rewinding one or more changes made to the graph data by one or more other transactions between a first time of the transaction and a second time of the current version of the graph index in the cache; and running the transaction at least based on the updated graph index.

[0077] Figure 5 A block diagram illustrating an example of a computing system 500 consistent with the implementation of the present topic is provided. (Reference) Figures 1A-1B and Figure 5 The computing system 500 can be used to implement the database management system 110 and / or any of its components.

[0078] like Figure 5 As shown, the computing system 500 may include a processor 510, a memory 520, a storage device 530, and an input / output device 540. The processor 510, memory 520, storage device 530, and input / output device 540 may be interconnected via a system bus 550. The processor 510 is capable of processing instructions that run within the computing system 500. Such running instructions may implement one or more components of, for example, a database management system 110. In some example embodiments, the processor 510 may be a single-threaded processor. Alternatively, the processor 510 may be a multi-threaded processor. The processor 510 is capable of processing instructions stored on the memory 520 and / or storage device 530 to display graphical information of a user interface provided via the input / output device 540.

[0079] Memory 520 is a computer-readable medium (such as volatile or non-volatile) that stores information within computing system 500. For example, memory 520 may store data structures representing a configuration object database. Storage device 530 provides persistent storage for computing system 500. Storage device 530 may be a solid-state drive, floppy disk drive, hard disk drive, optical disk drive, or magnetic tape drive, or other suitable persistent storage components. Input / output device 540 provides input / output operations for computing system 500. In some example embodiments, input / output device 540 includes a keyboard and / or a pointing device. In various implementations, input / output device 540 includes a display unit for displaying a graphical user interface.

[0080] According to some example embodiments, input / output device 540 can provide input / output operations for network devices. For example, input / output device 540 may include an Ethernet port or other network port for communicating with one or more wired and / or wireless networks (e.g., local area network (LAN), wide area network (WAN), Internet).

[0081] In some example embodiments, computing system 500 can be used to run various interactive computer software applications that can be used to organize, analyze, and / or store data in various formats. Alternatively, computing system 500 can be used to run any type of software application. These applications can be used to perform various functions, such as planning functions (e.g., generation, management, and editing of spreadsheet documents; word processing documents and / or any other objects, etc.), computing functions, communication functions, etc. Applications may include various plug-in functions or may be standalone computing products and / or functions. When activated within an application, functions can be used to generate a user interface provided via input / output device 540. The user interface can be generated by computing system 500 and presented to the user (e.g., on a computer screen monitor, etc.).

[0082] One or more aspects or features of the subject matter described herein can be implemented in digital electronic circuits, integrated circuits, specially designed ASICs, field-programmable gate arrays (FPGAs), computer hardware, firmware, software, and / or combinations thereof. These different aspects or features can include implementations in one or more computer programs executable and / or interpretable on a programmable system, which includes at least one programmable processor, which may be dedicated or general-purpose, coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to send data and instructions to the storage system, at least one input device, and at least one output device. The programmable system or computing system can include clients and servers. Clients and servers are generally geographically distant from each other and typically interact via a communication network. The client-server relationship arises from computer programs running on their respective computers, and they have a client-server relationship with each other.

[0083] These computer programs (which may also be referred to as programs, software, software applications, applications, components, or code) include machine instructions for programmable processors and can be implemented in high-level programming and / or object-oriented programming languages ​​and / or assembly / machine languages. As used herein, the term "machine-readable medium" means any computer program product, apparatus, and / or device, such as, for example, a disk, optical disk, memory, and programmable logic device (PLD), for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" means any signal used to provide machine instructions and / or data to a programmable processor. Machine-readable media may store such machine instructions non-transitory (e.g., like non-transitory solid-state memory or magnetic hard disk drives or any equivalent storage medium). Machine-readable media may alternatively or additionally store such machine instructions transiently (e.g., like processor caches or other random access memory associated with one or more physical processor cores).

[0084] To provide interaction with the user, one or more aspects or features of the subject matter described herein can be implemented on a computer having a display device, such as a cathode ray tube (CRT), liquid crystal display (LCD), or light-emitting diode (LED) monitor for displaying information to the user, and a keyboard and pointing device, such as a mouse or trackball, through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user. For example, feedback provided to the user can be any form of sensory feedback, such as visual, auditory, or tactile feedback; and input from the user can be received in any form, including sound, speech, or tactile input. Other possible input devices include touchscreens or other touch-sensitive devices, such as single-point or multi-point resistive or capacitive trackpads, speech recognition hardware and software, optical scanners, optical indicators, digital image capture devices, and associated interpretation software, etc.

[0085] In the foregoing description and claims, phrases such as “at least one of” or “one or more of” may appear after a combined list of elements or features. The term “and / or” may also appear in a list of two or more elements or features. Unless implicitly or explicitly contradicted by the context in which it is used, such phrases are intended to mean any element or feature listed separately, or any enumerated element or feature combined with any other enumerated element or feature. For example, the phrases “at least one of A and B”, “one or more of A and B”, and “A and / or B” are each intended to mean “A alone, B alone, or A and B together.” A similar interpretation applies to lists comprising three or more items. For example, the phrases “at least one of A, B, and C”, “one or more of A, B, and C”, and “A, B, and / or C” are each intended to mean “A alone, B alone, C alone, A and B together, A and C together, B and C together, or A and B and C together.” The term “based on” as used in the foregoing and claims is intended to mean “at least partially based on,” allowing for the inclusion of unstated features or elements.

[0086] Depending on the required configuration, the subject matter described herein can be embodied in systems, apparatus, methods, and / or articles. The implementations set forth in the foregoing description do not represent all implementations consistent with the subject matter described herein. Rather, they are merely examples of aspects consistent with the described subject matter. Although some variations have been described in detail above, other modifications or additions are possible. In particular, further features and / or variations may be provided in addition to the features and / or variations set forth herein. For example, the above implementations may be for various combinations and sub-combinations of the disclosed features, and / or combinations and sub-combinations of several other features disclosed above. Furthermore, the logical flows depicted in the figures and / or described herein do not necessarily require the specific order or sequence shown to achieve the desired results. Other implementations may be within the scope of the appended claims.

Claims

1. A data processing system, comprising: At least one data processor; as well as At least one memory stores instructions that, when executed by the at least one data processor, cause an operation including: A graph index is generated based on at least one or more vertex tables and edge tables in a database, wherein the database includes a relational database that stores graph data of the one or more vertex tables and edge tables; In response to a transaction that operates on graph data stored in the database, access the cache that stores the graph index corresponding to the graph data; In response to a detected cache miss, the graph index is updated by at least replaying or rewinding one or more changes made to the graph data by one or more other transactions between the first time of the transaction and the second time of the current version of the graph index in the cache. For the aforementioned transaction, the updated graph index includes vertices and / or edges that were inserted by the one or more other transactions before the first time and were not deleted by the transaction until after the first time. Specifically, the updated graph index excludes vertices and / or edges deleted before and / or inserted after the first time by the one or more other transactions, ensuring that the transactions have a consistent view of the data stored in the database according to the atomicity, consistency, isolation, and durability rules imposed by the database; and The transaction should be run based on at least the updated graph index.

2. The system according to claim 1, wherein, Running the transaction includes performing graph processing algorithms based at least on the updated graph index, the graph processing algorithms including subgraph, complement graph, in-degree, out-degree, in-edge, out-edge, neighbor, reachability, shortest path, one to all shortest paths, etc. One or more of the following: shortest path, strongly connected components, depth-first traversal, or breadth-first traversal.

3. The system according to claim 1, wherein, The graph index includes identifying a first vertex as an adjacency structure adjacent to a second vertex based at least on the first vertex being connected to a second vertex by one or more edges.

4. The system according to claim 1, wherein, The one or more other transactions modified the graph data at least by inserting vertices, deleting vertices, inserting edges, and / or deleting edges.

5. The system according to claim 1, wherein, The cache miss was triggered by modifications to graph data stored in the database.

6. The system according to claim 1, wherein, The operation also includes: Multi-version concurrency control (MVCC) is used to track multiple transactions that modify graph data stored in the database.

7. The system according to claim 1, wherein, The operation also includes: Maintain a redo log that tracks multiple changes made to the graph data stored in the database; and Read the redo log to replay or rewind one or more changes made to the graph data between the first time of the transaction and the second time of the current version of the graph index.

8. The system according to claim 1, wherein, The database includes document storage.

9. The system according to claim 1, wherein, The graph index is updated instead of being rebuilt entirely.

10. The system according to claim 1, wherein, The graph index update also includes replacing the current version of the graph index in the cache with the updated graph index.

11. A computer-implemented data processing method, comprising: A graph index is generated based on at least one or more vertex tables and edge tables in a database, wherein the database includes a relational database that stores graph data of the one or more vertex tables and edge tables; In response to a transaction that operates on graph data stored in the database, access the cache that stores the graph index corresponding to the graph data; In response to a detected cache miss, the graph index is updated by at least replaying or rewinding one or more changes made to the graph data by one or more other transactions between the first time of the transaction and the second time of the current version of the graph index in the cache. For the aforementioned transaction, the updated graph index includes vertices and / or edges that were inserted by the one or more other transactions before the first time and were not deleted by the transaction until after the first time. Specifically, the updated graph index excludes vertices and / or edges deleted before and / or inserted after the first time by the one or more other transactions, ensuring that the transactions have a consistent view of the data stored in the database according to the atomicity, consistency, isolation, and durability rules imposed by the database; and The transaction should be run based on at least the updated graph index.

12. The method according to claim 11, wherein, Running the transaction includes performing graph processing algorithms based at least on the updated graph index, the graph processing algorithms including subgraph, complement graph, in-degree, out-degree, in-edge, out-edge, neighbor, reachability, shortest path, one to all shortest paths, etc. One or more of the following: shortest path, strongly connected components, depth-first traversal, or breadth-first traversal.

13. The method according to claim 11, wherein, The graph index includes identifying a first vertex as an adjacency structure adjacent to a second vertex based at least on the first vertex being connected to a second vertex by one or more edges.

14. The method according to claim 11, wherein, The one or more other transactions modified the graph data at least by inserting vertices, deleting vertices, inserting edges, and / or deleting edges.

15. The method according to claim 11, wherein, The cache miss was triggered by modifications to graph data stored in the database.

16. The method of claim 11, further comprising: Multi-version concurrency control (MVCC) is used to track multiple transactions that modify graph data stored in the database.

17. The method of claim 11, further comprising: Maintain a redo log that tracks multiple changes made to the graph data stored in the database; as well as Read the redo log to replay or rewind one or more changes made to the graph data between the first time of the transaction and the second time of the current version of the graph index.

18. A non-transitory computer-readable medium storing instructions that, when executed by at least one data processor, cause operation of the method as described in any one of claims 11-17.

Images