Method and system for view materialization for subgraph matching queries

Abstract

This invention relates to subgraph matching queries and view materialization operations in the field of graph databases, and discloses a view materialization method and system for subgraph matching queries. This invention designs a materialized view data structure based on a tree structure of subgraphs, achieving a balance between materialization overhead and query optimization performance. This invention designs a tree partitioning algorithm that can efficiently calculate the minimum connected tree partition. This invention also designs a matching query algorithm for the materialized view data structure, enabling efficient query optimization using the materialized view. The view materialization method of this invention can complete the construction of materialized views with controllable time and space overhead, and can significantly accelerate query response, making it applicable to fields such as financial risk control, bioinformatics, social network analysis, and knowledge graph retrieval.

Description

Technical Field

[0001] This invention belongs to the field of information technology and relates to subgraph matching query and view materialization operations in the field of graph databases. Specifically, it is a view materialization method and system for subgraph matching query. Background Technology

[0002] Subgraph matching is a fundamental task in graph analysis systems. Specifically, given a data graph and a query graph, the system finds all subgraphs in the data graph that match the query graph. In practical applications, graph query engines typically need to handle a large number of subgraph matching queries, many of which share the same substructure. Materializing the results of these common subqueries enables computational reuse, significantly improving query efficiency. However, existing view materialization techniques either have high space overhead or limited speedup effects. The following section introduces the background techniques of subgraph matching and materialized views: 1) Subgraph Matching: The subgraph matching problem is defined as finding all subgraphs in the data graph that match the query graph, given a data graph and a query graph. Subgraph matching is an NP-hard problem. It has been extensively studied. Existing subgraph matching techniques often focus on optimizing single queries. However, in practical applications, graph analysis systems receive multiple queries over a period of time, and these queries often share common subqueries. In this case, subgraph matching techniques for single queries cannot fully utilize these common subqueries for optimization. Some subgraph matching works utilize common subqueries between queries for query optimization. However, these works also have problems: due to the large number of subgraph matching query results, calculating and storing common subqueries leads to significant time and space overhead. These works do not address this problem.

[0003] 2) Materialized Views: Materialized view technology pre-computes (materializes) frequently occurring queries or subqueries (called views) and uses the materialized results of these views to accelerate subsequent related queries. Materialized views are widely used in relational databases and have been proven to significantly reduce redundant computation, thereby optimizing query performance. In recent years, the increasing emphasis on graph data structures has sparked interest in materialized views on graphs. However, relational data and graph data differ significantly in storage, data access, and computation, thus requiring a redesign of views to adapt to graph structures. Specifically, for materialized views used for subgraph matching, on the one hand, some works materialize views by computing and storing all matching results of the subgraph. Since the number of subgraph matching query results is often huge, this method leads to significant space and computational overhead. On the other hand, some works materialize views by storing a candidate set of nodes or edges of the view. This method can complete view materialization with less time and space overhead. However, the result extraction efficiency of materialized views built based on this method is not as good as the former method. This is because the system still needs to search from the materialized view to obtain the matching results of the view in the data graph.

[0004] Existing view materialization techniques for subgraph matching either incur extremely high preprocessing time and space overhead or offer limited speedup. Furthermore, existing subgraph matching techniques either focus solely on single-query optimization or multi-query optimization, but computing and storing the matching results of subqueries requires significant time and space overhead, sharing the same drawbacks as existing view materialization techniques for subgraph matching. Summary of the Invention

[0005] This invention designs a view materialization method and system for subgraph matching queries. Existing view materialization methods fall into two categories: 1) Query-based materialized views (QM views): This method enumerates and stores all matching results of the view. Since the number of matching results for a view is often very large, its materialization overhead is extremely high. 2) Node-based materialized views (VM views): This method only stores the candidate set of view nodes. This method has low construction overhead and low space consumption, but limited query optimization. The view materialization method of this invention achieves a balance between the two, enabling materialized view construction with controllable time and space overhead while significantly accelerating query response.

[0006] The technical solution adopted in this invention is as follows: A view materialization method for subgraph matching queries includes the following steps: Receive the view and use a search-based tree partitioning algorithm to obtain the tree partitioning of the view; Construct a materialized view based on tree partitioning; Receive queries and accelerate query responses using materialized views.

[0007] Furthermore, the search-based tree partitioning algorithm includes the following steps: 1) View Perform a depth-first search to obtain its depth-first search order, denoted as . ; 2) Perform multiple iterations, the first... Wheel extension width less than or equal to The state, except for the first Round, the first The initial state of each wheel is an expansion of the width of the previous wheel. The state; the first The initial state of the wheel contains only one superpoint, and that superpoint contains only one node. Each iteration starts the search from the initial state; 3) From the state Start the search. The node representing the newly joined state , Represented as ,in express Each superpoint generates a new state through state transition. ; 4) Check all new states obtained from the state transition, and only retain valid new states. The validity check includes: a) the current partition is tree-shaped; b) all superverts are either connected or may be connected in the future; 5) If the width of the new state is greater than d, then the state will not be processed in the current iteration, but will be stored for processing in the next iteration; 6) If the new state already contains a view If all nodes in the search are empty, terminate the search and return immediately; 7) If the new state is not the terminating state, use the new state for recursive search; 8) If all non-terminating new states have completed the recursive search, then the process ends. Search processing.

[0008] Furthermore, the search-based tree partitioning algorithm is optimized using the following two methods: 1) Iterative wide-limit search: The partitioning process is broken down into multiple iterations, with the width limit gradually increasing in each iteration. In each iteration, states that exceed the current width limit are not expanded but are marked as reserved states and saved. If a complete partition that satisfies the current width limit is found, it is returned as the optimal solution. Otherwise, the width limit is increased and the next iteration starts from the previously stored reserved states, thus effectively avoiding redundant search. 2) Connectivity pruning: Introduce an index ,for Each pair of nodes in and each integer ,use Represents a node and Whether it passes The last one A path consisting of nodes is connected, for the state Each superpoint in the list is defined if it contains a pair of nodes that are not yet connected and cannot be connected by the remaining unpartitioned nodes. If so, then prune that state.

[0009] Furthermore, the materialized view is constructed based on tree partitioning, wherein the materialized view consists of two parts: 1) candidates for superpoints; 2) candidates for superedges; and two strategies are adopted to prevent the materialized space from being too large: 1) edge compression; 2) further segmentation of excessively large superpoints.

[0010] Furthermore, the method of accelerating query responses through materialized views includes: 1) Rewrite the query based on the materialized view to construct a rewritten query; 2) Enumeration is performed using an enumeration algorithm based on rewrite query.

[0011] Furthermore, the enumeration algorithm based on rewrite query includes the following steps: 1) Received parameters: Current partial match Rewrite query Search order The currently enumerated superpoint labels And the candidate set for each superpoint ; 2) Update the answer: Determine the current partial match. Does it constitute a complete match? If so, match that part. Add to the results list and return; otherwise, expand the partial match. 3) Obtaining the local candidate set: based on partial matching Enumerate the next superpoint The possible values ​​of the local candidate set are called local candidates. : 4) Extended partial matching: Enumeration All possible values, for a given value Construct new partial matches Then recursively proceed to the search of the next superpoint; 5) Backtracking: After all local candidate values ​​have been enumerated, the process ends and returns.

[0012] A view materialization system for subgraph matching queries, comprising: The tree partitioning module receives the view and uses a search-based tree partitioning algorithm to obtain the tree partitioning of the view. The materialized view building module is used to construct materialized views based on tree partitioning; The query module receives queries and accelerates query responses through materialized views.

[0013] The key point of this invention is: 1) This invention designs a materialized view data structure based on a subgraph tree structure, which can achieve a balance between materialization overhead and query optimization effect.

[0014] 2) This invention designs a tree partitioning algorithm that can efficiently calculate the minimum connected tree partition.

[0015] 3) This invention also designs a matching query algorithm for the materialized view data structure, which can efficiently utilize the materialized view for query optimization.

[0016] The beneficial effects of this invention are as follows: This invention addresses the drawbacks of QM views (excessive time and space overhead for materialization) and VM views (limited query optimization). The invention was tested on a real dataset. Regarding view materialization, the time and space cost of the materialization in this invention is more than two orders of magnitude lower than that of QM views and similar techniques. Regarding query optimization, the view of this invention provides significantly better speedup than VM views, with a difference of up to two orders of magnitude. Attached Figure Description

[0017] Figure 1 Examples include query-based materialized views, node-based materialized views, and the materialized views of this invention.

[0018] Figure 2 This is a flowchart of the method of the present invention.

[0019] Figure 3 This is an example of a tree partitioning algorithm.

[0020] Figure 4 This is an example of edge compression.

[0021] Figure 5 This is an example of rewriting a query.

[0022] Figure 6 It is the spatial overhead (MB) of the materialized view in the experiment.

[0023] Figure 7 It is the time cost (ms) of the physical and chemical process in the experiment.

[0024] Figure 8 This is the subgraph matching query processing time (ms) in the experiment. Detailed Implementation

[0025] The present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0026] This invention presents a view materialization method for subgraph matching. Current view materialization methods for subgraph matching either incur significant time and space overhead or suffer from low efficiency in extracting materialization results. This invention, however, can construct a materialized view with relatively low time and space overhead, and such a materialized view provides high efficiency in extracting materialization results. The relevant definitions are given below: 1) Graph: A graph is a data structure consisting of nodes and edges. An edge can connect two nodes. In this invention, we focus on graphs where nodes have labels. Therefore, given a graph... It can be used Represented by triplets, among which express The set of nodes, express The set of edges, It is a mapping from a node to a label. Represents a node The tag.

[0027] 2) Subgraph matching: Subgraph matching is defined as a given data graph and query graph Find All of them The matching subgraph. The commonly used matching semantics here are homomorphism and isomorphism. exist A subgraph homomorphism can be obtained through a mapping This mapping needs to satisfy: 1) For any node The node it matches The tag should match the node. The tags are the same, that is ;2) For any edge in , There must be a matching edge in the middle, that is Subgraph isomorphism requires that the mapping be injective. In this invention, homomorphic semantics are considered. Since subgraph isomorphic matching is a subset of subgraph homomorphic matching, this invention can also be used for query optimization of subgraph isomorphism.

[0028] 3) Views: A view is a query obtained through specific rules. It is then materialized for subsequent query optimization. (This will be used in the following text.) Represents a view.

[0029] 4) Materialized View: A materialized view is a view... Corresponding data structure . Includes All matching results are shown in the data graph. When a new query contains... At that time, the graph analysis system can utilize pre-calculated... Speed ​​up queries.

[0030] Currently, materialized views for subgraph matching can be divided into two categories: query-based materialized views (QM views) and node-based materialized views (VM views). On the one hand, QM views directly store... All matches. This materialized view method can effectively optimize queries. However, because the number of matching results for the view is very large... This materialization method requires significant time overhead to construct the materialized view, and storing the view consumes a large amount of memory. On the other hand, the VM view stores a candidate set of nodes or edges for the view. The method for constructing this materialized view is as follows: first, obtain the candidate set for each node using its label. Then, the algorithm removes invalid candidate nodes using filtering rules. These filtering rules are defined as follows: data nodes... It is a query node Candidate nodes need to satisfy the following conditions: any neighbor There should also be corresponding neighbors. yes The candidate set. The VM view can be constructed in polynomial time and has a space footprint of... Compared to QM views, VM views are more efficient to build and have a smaller footprint. However, they cannot directly extract matching results from materialized views; instead, they must perform a search based on the materialized view to obtain matching results. Furthermore, VM views may contain invalid candidate nodes—nodes that meet the construction rules but do not appear in any match. These candidate nodes increase search costs. Moreover, building a materialized view that does not contain invalid candidate nodes has been proven to be NP-hard, making it difficult to quickly eliminate these candidate nodes. Therefore, VM views are less efficient at query optimization than QM views. Figure 1(d) and (e) illustrate the QM view and VM view, respectively. The QM view has 10,000 rows, and its size is even larger than the data graph. The VM view, while smaller in size, cannot directly provide matching results and contains invalid nodes. and .

[0031] In summary, current materialized view techniques for subgraph matching either have excessive space overhead (based on queries) or limited optimization effects (based on nodes).

[0032] To address the drawbacks of the two aforementioned approaches and integrate their advantages, this invention employs a subgraph-based view materialization method. First, the invention decomposes the view into several connected subgraphs, referred to as superverts. Superverts are connected by superedges. An edge exists between two superverts if and only if the two endpoints of an edge in the view are located in both superverts. Superverts are disjoint, and their union is the set of nodes in the view. The new graph formed by the superverts and superedges is called graph partitioning. This invention calculates a candidate set of superverts (i.e., all matching superverts) and then filters out invalid candidates using filtering rules similar to those in node-based materialization schemes, thereby completing the construction of the materialized view. Figure 1 (f) shows the view graph partitioning (The tree partitioning shown in the diagram will be explained in detail later.) The same rounded rectangle represents a superpoint. . Figure 1 (g) demonstrates based on Materialized view constructed The rounded rectangle This represents the matching results for the corresponding superpoints. These matches are stored in a table, with each column representing a match. The edges between matching results represent candidate superedges. The subgraph-based materialized view scheme is a compromise between the query-based materialized view scheme and the node-based materialized view scheme. On the one hand, it only needs to enumerate the candidate set for each superpoint; therefore, its construction cost and space usage are... ,in express The number of superpoints is much smaller than that of query-based materialization schemes. On the other hand, since the number of superpoints is less than the number of view nodes, its matching extraction efficiency is also higher than that of node-based materialization schemes.

[0033] Subgraph-based materialization schemes rely on graph partitioning. On one hand, the size of the supervertron determines the construction cost and space usage of the materialized view; the smaller the largest supervertron, the lower the materialized view overhead. On the other hand, when the graph is partitioned into a tree, the filtering rules can guarantee no invalid candidate matches. The proof is as follows: Consider a VM view where all nodes satisfying the filtering rules can construct a match containing that node; subgraph-based materialized views have similar properties. Therefore, given a view, we want to construct a graph partition that: 1) is tree-shaped; 2) has the largest supervertron as small as possible. This problem is defined as minimum-width connected tree partitioning. This invention designs an efficient search algorithm to quickly solve this problem. In addition, this invention also includes algorithms for query optimization using materialized views. These will be introduced later. Figure 2 This is a flowchart of the present invention. The process is divided into two parts: offline and online. The offline part receives views and constructs materialized views based on subgraphs for them. The online part receives queries, and when the query includes views, it rewrites the query. The rewritten query represents query optimization using materialized views. Finally, the final matching result is obtained through the rewritten query.

[0034] Next, the technical details of the present invention will be introduced from three aspects: tree partitioning, materialization details, and query response.

[0035] 1. Tree partitioning Given a view, we want to obtain a graph partition that satisfies the following conditions: 1) Each supervertex is internally connected. 2) Supervertexes are disjoint, and the union of the supervertexes is all nodes in the view. 3) This graph partition is tree-like. 4) Under the above conditions, the size of the largest supervertex is as small as possible.

[0036] The above problem is defined as the minimum connected tree partitioning problem, or simply tree partitioning. The size of the largest supernode is called the width of the tree partition. This invention designs a search-based tree partitioning algorithm. Starting from an empty partition, it proceeds sequentially... Add views iteratively The nodes in the [information]. In the implementation of this invention, It is a depth-first search (DFS) order starting from any root node, and any connectedness order (i.e., each node is connected to at least one previously added node) is valid. Each search state... Representing a subset A division on, in which This represents the number of nodes in the current state. (For exploration...) The successor state of the next node Add to In this process, the node is either inserted into an existing superpoint or a new superpoint is created. To ensure correctness, two constraints are imposed at each step: the partition must remain a tree, and each superpoint must be connected or potentially connected in future states (discussed in detail below). Invalid states are pruned accordingly.

[0037] Specifically, the search-based tree partitioning algorithm of the present invention includes the following steps: 1) View Perform a depth-first search to obtain its depth-first search order, denoted as . .

[0038] 2) Perform the search: The algorithm will perform multiple iterations, the first... Wheel extension width less than or equal to The state. Except for the first... Round, the first The initial state of each wheel is an expansion of the width of the previous wheel. The state. The... The initial state of the wheel contains only one superpoint, and that superpoint contains only one node. In each iteration, the algorithm starts its search from these initial states.

[0039] 3) State transition: The algorithm transitions from state... Start the search. The node representing the newly joined state , Represented as ,in express There are two superpoints. The algorithm generates new states through the following two state transition methods. .

[0040] a) Create a separate superpoint for the new node: .

[0041] b) Add the new node to the existing superpoint: The algorithm iterates through all superpoints, for the first superpoint... Each superpoint, the algorithm will... join in To construct a new state, that is .

[0042] 4) State validity check: The algorithm checks all new states obtained from state transitions and only retains valid new states. The validity check includes: a) The current partition is tree-like; b) All superpoints are either connected or may be connected in the future (discussed in detail below).

[0043] 5) State preservation: If the width of the new state is greater than d, the algorithm will not process this state in the current iteration, but will store it for processing in the next iteration.

[0044] 6) Answer Update: If the new state already contains... Having identified all nodes in the search (referred to as the terminating state), the algorithm has found the answer through iterative widening optimization (discussed in detail later). Therefore, the algorithm terminates the search and returns directly.

[0045] 7) Recursive search: If the new state is not the terminal state, the algorithm will use the new state for recursive search: take the new state as input and repeat steps 3)-7).

[0046] 8) The algorithm terminates when all non-terminating new states have completed the recursive search. Search processing.

[0047] Figure 3 An example of a search-based tree partitioning algorithm is shown, where each state is enclosed in a rounded rectangle and state transitions are indicated by arrows. Nodes with the same label are grouped together as supernodes. Although there are two possible solutions, ... and However, the algorithm stopped when it found the first answer.

[0048] Exhaustive search can be implemented based on state transitions. However, indiscriminately searching all states is inefficient. Therefore, this invention employs various optimization methods to optimize the search-based tree partitioning algorithm. The following are two significant optimization methods: (1) Iterative Wide-Constraint Search: This pruning strategy is inspired by Iterative Deeper Search (IDS). When V is sparse, the minimum width is often small, so searching for large-width states is unnecessary. To take advantage of this, this invention breaks the partitioning process into multiple iterations, with the width constraint gradually increasing in each iteration (steps 2-8 in the algorithm above). In each iteration, states exceeding the current width constraint are not expanded. Instead, they are marked as reserved states and saved for later use (step 5 in the algorithm above). If a complete partition that satisfies the current constraint is found, it is returned as the optimal solution. Otherwise, the constraint is increased, and the next iteration begins from the previously stored reserved states (step 2 in the algorithm above), thus effectively avoiding redundant searches.

[0049] (2) Connectivity pruning: generating new states Then, the algorithm checks each hypervertices in that state. If a disconnected hypervertices are found, it cannot be directly accessed. Perform pruning. Because these disconnected supernodes might become connected after new nodes are added in subsequent state transitions. If pruning is performed without further checks... Pruning might cause valid substates to be missed, thus leading to a missed correct answer. Therefore, this invention introduces an index. .for Each pair of nodes in and each integer ,use To represent nodes and Is it possible to pass? The last one The paths consisting of nodes are connected. This data structure can be preprocessed using a variant of the Floyd-Warshall algorithm. For the state... Each superpoint in the list is defined if it contains a pair of nodes that are not yet connected and cannot be connected by the remaining unpartitioned nodes. (Right now If the condition is false, pruning can be performed on that state.

[0050] exist Figure 3 The text shows an example of using these two optimization strategies: due to connectivity pruning, the state... Skipped because and Unable to pass through the remaining nodes Connected. State , and They were pruned because their width was 3, while the results had already been found in the stage with a width of 2.

[0051] 2. View materialization Once the tree partition of the view is obtained, a materialized view can be constructed based on the tree partition. It consists of two parts: 1) Candidates for superpoints: For a superpoint Store its candidates in a table. Candidate It needs to meet the following conditions: a) It is a match of the superpoint in the data graph; b) It meets the filtering conditions: for any neighbor ,exist Neighbors .

[0052] 2) Candidates for superedges: For each superedge Record all its candidates.

[0053] The views of this invention are complete and compact. Wherein, completeness means that for any... Matches can all be found from Extraction is performed within this process. Compactness represents the matching for any superpoint. It must belong to a certain Matching.

[0054] The above is the basic version of the materialization process. In addition, this invention incorporates two strategies to prevent the materialization space from becoming too large: 1) Edge compression: In the basic version, for each superedge... Store its candidate edges as super points and The connections between candidate nodes. The time and memory cost of materializing candidate edges is... ,in and They represent and The candidate set. Such materialization costs can be very high, especially for large supernodes that may have a large number of candidate nodes. Therefore, edge redundancy can be reduced through edge compression. The definition of a port node is given, which is a key concept in edge compression: given a superedge... , yes The port node in the middle, if and only if There exists at least one edge connecting to The nodes in. The definition of port nodes in [the code] is similar. Using [the code]... express Regarding hyperedge The port node. According to the definition of a port node, two... Candidate Matching and When the values ​​on the port nodes are the same, they are... The neighbors on the same node will also be identical. Therefore, candidate matches can be grouped according to the values ​​on the port nodes, and then hyperedge connections can be made in groups to compress the hyperedge candidate set. Figure 4 An example of edge compression is shown, for hyperedges. Without edge compression, 400 edges are needed. With edge compression, only 106 edges are required.

[0055] By using hyper-edge compression technology, The size of the candidate edges is reduced to However, in some cases, the size of the candidate edges is still very large. In such cases, a threshold is set. When materializing a candidate edge for a hyperedge, the candidate edges are grouped according to their port nodes, and then materialized group by group. If the number of materialized edges exceeds... If so, then the materialization of candidate edges is terminated, and the unmaterialized groups are marked as unmaterialized. This marker will be used later when performing query optimization using materialized views.

[0056] 2) Further Segmentation of Excessively Large Superpoints: Although the minimum width tree partition is calculated, the number of candidate matches for superpoints may still be very large, consuming significant space. Therefore, this invention designs a strategy for further segmentation of excessively large superpoints to address this issue. First, the number of subgraph matches for each superpoint is estimated. Specifically, for a superpoint, its spanning tree is obtained first, and then the number of matches in the spanning tree is obtained through dynamic programming as an estimate of the number of matches for this superpoint. This invention sets a threshold. This serves as a limit on the number of matches exceeding the limit. When the estimated number of matches exceeding the limit exceeds... The superpoint is then further partitioned. Since the estimation method used in this invention overestimates, excessively large superpoints will not be missed. After further partitioning the superpoint, the graph partition will no longer be tree-like, thus compatibility cannot be guaranteed. However, this is a necessary trade-off—sacrificing compatibility to ensure the controllability of the materialized space size. To minimize the impact, the algorithm requires that the number of edges minus the number of vertices in the graph partition be as small as possible (when the number of edges minus the number of vertices = 1, it is a tree partition), that is, to make the graph partition as close to a tree as possible, thereby reducing the cost of introducing cycles.

[0057] These are two strategies employed during the materialization process to ensure controllable spacetime and temporal costs. In practical applications, these strategies can be adjusted based on available space. and This achieves a balance between materialization overhead and materialized view optimization.

[0058] 3. Query and Answer The following describes how to use materialized views to speed up query response. Query response consists of two steps: 1) rewriting the query based on the view to construct a rewritten query; 2) enumerating based on the rewritten query.

[0059] The first step is to construct the rewritten query. A rewritten query indicates how views can be used for query optimization. A rewritten query... It consists of two parts: 1) A series of isomorphic mappings Each isomorphic mapping Both indicate that a subgraph in a query graph is isomorphic to a view. Furthermore, each query node can appear in at most one isomorphic mapping.

[0060] 2) A graph partition Each of these superpoints Either it contains one that does not belong to any nodes It either contains a set of nodes These nodes through In Mapped to the same view mode ,and belong The same superpoint For the second case, it is called... To materialize the superpoint, and in and Establish a mapping between them Furthermore, if two materialized superpoints and Mapped to the same view mode And the edges between them can be Matching. Then the hyperedges between them are also materialized and mapped to... hyperedge in .

[0061] Figure 5 This demonstrates an example of rewriting a query. Query via view and Rewritten The superscript lines enclose materialized hyperpoints, which are mapped to their corresponding view hyperpoints. The dashed lines enclose ordinary hyperpoints, which are mapped to null. Materialized hyperedges are represented by thick lines.

[0062] Intuitively, it's desirable for the query graph to be covered by views as much as possible. This allows for the use of more materialized view information to answer the query, rather than calculating from scratch. However, selecting views that cover as many nodes in the query graph as possible is NP-hard because it can be reduced to an exact-cover problem, a well-known NP-complete problem. Therefore, to prevent excessive computation time spent on rewriting, this invention employs a greedy approach: each time, select the view that covers the most nodes in the query graph, ensuring the covered area is not previously covered, repeating this step until no more views can be selected. After selecting suitable views, it's only necessary to construct them according to the definition. That's all.

[0063] The second step is to use The query-response method of this invention is similar to the query-response framework of existing subgraph matching algorithms. The difference is that the query-response method of this invention is based on superpoint granularity, and the candidate set of materialized superpoints is provided by the materialized view.

[0064] A general subgraph matching framework comprises three key components: filtering, sorting, and enumeration. Given a query graph, the framework first constructs a virtual machine (VM) view online, filtering candidate nodes for each query node based on node labels and adjacency connectivity. While specific algorithms may adjust the VM view differently, the core principles remain consistent. Next, the search order is determined by reordering the query nodes. Then, the enumeration phase explores all possible matches in this order, typically using a depth-first search (DFS) strategy. A partial matching is maintained. And expand it by progressively adding mappings of query nodes according to the search order. For each query node A local candidate set is derived based on the current partial matching and the initial candidate set. Each candidate node... By Add to This forms a new DFS branch. Then, the search continues along this new branch. The algorithm maps the data. A branch ends when a complete match is found or the local candidate set is empty. In both cases, the algorithm backtracks by removing the last mapping and explores other branches.

[0065] The enumeration step of this invention employs a similar framework, which also includes these three parts. However, the algorithm of this invention is based on rewriting the query execution matching process, which introduces some differences: 1) Filtering: The goal of the filtering phase is to prepare for rewriting queries. For each superpoint, a candidate set is calculated. Some superpoints in the query graph have already been mapped to superpoints in the view schema, and superpoint candidates in the materialized view can be used as an initial candidate set. However, these candidates are only filtered based on the topology of the subqueries mapped by the view, not the entire query graph. Therefore, further filtering of these candidates is needed. The algorithm of this invention filters the entire query graph. Create a VM view. Include any mappings that are not present in this VM view. Super Point All initial candidates need to be excluded. Furthermore, in view mode... compared to, Superpoints in It may contain additional internal edges. In this case, the algorithm of this invention will exclude candidates that cannot match these edges. For unmaterialized superpoints that are not mapped to any view mode. Their candidate sets are directly set as query nodes in the VM view. The candidate set. Furthermore, MAVIS builds an additional VM view for each view pattern during the offline phase. The candidate set from these VM views can be used to initialize the VM view of Q in this step, thus accelerating the build process. Since VM views are lightweight, materializing them does not incur significant memory overhead.

[0066] 2) Sorting: Since the enumeration in this invention is performed at the superpoint level, this step requires generating a sort of superpoints, not a sort of query nodes. Any existing subgraph matching sorting algorithm can be used. Furthermore, it only needs to be applied to the rewritten query graph composed of superpoints and superedges. That's all.

[0067] 3) Enumeration: Enumeration is typically the most computationally intensive stage in subgraph matching. This step generates search states that are exponentially related to the size of the query graph. The algorithm of this invention clusters multiple query nodes into superpoints and enumerates the matching results at the superpoint level. Therefore, the search space is significantly reduced, thereby improving search efficiency.

[0068] The enumeration process based on rewriting queries is very similar to that of general subgraph matching frameworks. This invention iteratively expands the partial matching using a depth-first search (DFS) approach. This allows for the enumeration of matching results, following the search order. Add the match for each supernode to In the middle, when Output when matching all superpoints Enumerating supernodes When matching, it is necessary to calculate Local candidate sets, which are based on the current partial match And the candidates selected from the candidate set generated during the filtering phase. Specifically, the algorithm has two methods for calculating local candidates. Assuming in... The ones that have already been matched The neighbors are set as For each Record the matching result as If the connection is established... and Candidate hyperedges are materialized in the view, and can be used to retrieve [objects / objects] related to [other objects]. Connected The candidates are denoted as If all All can be retrieved through candidate superedges. This invention will... The local candidate calculation is as follows However, some hyperedges are not materialized due to space constraints (see the View Materialization section) or because they are not mapped to any hyperedges in the view schema. In this case, a second approach is used, which computes local candidates at the query node level through the intersection of sets. This approach first... Each port node within Determine the local candidate set . Local candidates can be obtained by connecting all The neighborhood is used to calculate, where yes The matched neighbor nodes. Then, the candidate set generated during the filtering phase is... With each Select the connection Local candidates.

[0069] Specifically, the enumeration algorithm based on rewrite query of the present invention includes the following steps: 1) The enumeration based on rewritten queries is implemented recursively. This process receives the following parameter: the current partial match. Rewrite query Search order The currently enumerated superpoint labels And the candidate set for each superpoint The following steps constitute the recursive process.

[0070] 2) Update the answer: This process first checks the current partial match. Does it constitute a complete match? If so, match that part. Add to the results list and return. Otherwise, proceed with the next steps to expand the partial match.

[0071] 3) Obtaining the local candidate set: This process is based on Enumerate the next superpoint Possible values ​​( These values ​​are called local candidates. Calculate the local candidate set. There are two ways: a) If , They've all been objectified. .

[0072] b) Otherwise, the process will enumerate All port nodes are identified, and local candidate sets for these port nodes are calculated. Finally, the process is validated based on the values ​​of the local candidate sets of the port nodes and... Sure .

[0073] 4) Extended partial matching: This process enumerates... All possible values. For a given value Construct new partial matches If the matching uses isomorphic semantics, it needs to be checked. Does it contain duplicate values? Next, recursively proceed to the search for the next superpoint. Specifically, it will... Pass in step 1), and Repeat steps 2)-4).

[0074] 5) Backtracking: The process ends and returns after all local candidate values ​​have been enumerated.

[0075] This invention was tested on four real-world data graphs: Human, YouTube, WordNet, and Twitter. To comprehensively evaluate the performance of this invention, the experiments generated a large number of query graphs of varying sizes and densities. Query graphs with an average degree of 3 or higher were classified as dense, while the rest were classified as sparse. This experiment used the size of the query graph and the first letter of its density classification to represent each query type. For example, `16d` indicates that the query graph contains exactly 16 nodes and is classified as a dense query type. For view pattern selection, the experiment used a frequent subgraph mining algorithm to identify frequent subgraphs as candidate views. The importance of each view was then estimated by its size and the number of queries that could benefit from it. For each query type, the experiment selected the 10 most important view patterns.

[0076] For the evaluation method, during the materialization phase, this experiment configured a memory limit of 100GB and a time limit of one hour for each query type. Execution time and memory consumption were recorded as efficiency metrics for each materialization algorithm. In the query answering experiment, since previous subgraph matching work was based on isomorphic semantics, this experiment also configured all view-based methods to generate isomorphic matches accordingly. To maintain consistency with previous work, for each query, the query processing algorithm terminated when one of the following conditions was met: 1) all matches were found; 2) the number of matches reached [a certain threshold]. 3) The algorithm execution time reaches 300 seconds. Execution time is recorded as an efficiency metric.

[0077] Materialization Experiments: This experiment implements VM, QM, Factorization-based View (FDB), Answer Graph (AG), Tree Decomposition-based View (TD), and MQO-based View (MQO). Factorization uses a trie-like f-representation to compress the matching results of view patterns. d-representation is an improved version of f-representation, further merging common subtrees and organizing the matching results in a more compact directed acyclic graph (DAG). This experiment uses d-representation for comparison. AG is a method for constructing VM views. Compared to traditional VM view construction, AG uses triangulation to divide the view into several triangles and filters out invalid candidate points / edges by maintaining the matching of the triangles. Tree decomposition is an algorithm similar to tree partitioning: given a graph... Construct a tree ,in Each node is A subset (referred to as a package), satisfying the following conditions: 1) All The union of is ;2) For any edge There exists a simultaneous containing and 3) For any node ,Include The overlapping packets form a connected subtree. The biggest difference between tree decomposition and tree partitioning is that the packets in tree decomposition intersect, while the superpoints in tree partitioning are disjoint. This property makes tree partitioning more suitable for materializing views, which will be demonstrated in the experiment. In this experiment, the superpoints of tree partitioning are replaced with packets in tree decomposition. There are edges between superpoints if and only if two superpoints are adjacent packets. The tree decomposition algorithm is implemented using a state-of-the-art algorithm. MQO is a subgraph matching multi-query optimization algorithm that uses superpoint partitioning similar to this invention to store view matches. However, there are several key differences between it and this invention: 1) Different partitioning algorithms: MQO uses a minimal tree partitioning algorithm, not the minimum tree partitioning algorithm; 2) Its superpoints must be internally connected; 3) MQO first enumerates the matching results of the view and then projects them onto the superpoint partitioning, while this invention calculates the matching of each superpoint and then performs connection filtering. This experiment follows the settings in the original MQO paper: the size of each superpoint is limited to no more than 3, and any excessively large superpoint will be randomly partitioned into superpoints not exceeding this limit.

[0078] Figure 6 and Figure 7The total spatial overhead of the materialized view and the total time overhead of the materialization process are shown. Cases where materialization fails due to exceeding spatial or time constraints are marked with a "·". Methods that fail due to memory constraints (e.g., QM and TD on WordNet) do not have a bar chart in the time cost graph because they cannot be executed, and vice versa. This invention is denoted as MAVIS in the graph. VM completes materialization with less spatial and time overhead. However, its speedup in subsequent experiments is generally mediocre (see below). QM exceeds the space constraint in all cases. FDB times out on WordNet and Twitter due to its exponential time complexity. AG times out on WordNet and Twitter because maintaining the matching relationships of triangles incurs excessive time costs. However, the materialized view constructed by AG is smaller than that of VM. TD exceeds the space constraint on WordNet and Twitter due to the excessive size of its superpoints (bags), resulting in too many candidate matches. Furthermore, experiments show that the width of TD is generally larger than the book partition width of this invention. MQO only passed the Human-16s and Human-16d tests; it timed out in all other tests due to the excessive number of view matches. Experiments attempted to extend the MQO time limit to 12 hours to materialize a view in the Human-24s test, but this also timed out. Therefore, MQO is not suitable for view materialization in cases of complex views or large datasets. Finally, this invention successfully materialized all cases under the given constraints.

[0079] Next, the experiments evaluated the query-response performance of the materialized view algorithms. For a comprehensive comparison, the following subgraph matching algorithms were selected as benchmarks: RapidMatch (RM), VEQ, BICE, and BSX. These algorithms were implemented using their respective open-source code. For materialized views, for AG and VM, the experiments leveraged their materialized views to accelerate the filtering step of subgraph matching; other steps followed the implementation of RM. For FDB materialized views, the experiments extracted the matching results from the views and expanded them into complete query matching results. For TD materialized views, the experiments constructed a rewritten query for them using a method similar to that of this invention. Specifically, a superedge exists between two superpoints if they are adjacent packets in the tree decomposition or if a query edge exists between them. Other steps followed the same process as MAVIS. For MQO materialized views, the experiments used its original query-response method.

[0080] Figure 8Experimental results are shown. Failures in materialization are either not shown in the figure or marked with "·". In some scenarios (such as YouTube), both VM and AG show significant optimization effects. In addition, AG is better than VM in terms of optimization effect because it can filter out more invalid candidates. However, since they mainly optimize the filtering step, their impact is limited in datasets where enumeration dominates the workload (such as WordNet). FDB's materialized view is inefficient for querying YouTube. This is because the matching results found in the view are not necessarily the matching results of the query, and FDB lacks a mechanism to filter such cases. TD's enumeration efficiency is also not ideal for two reasons: (1) The superpoints in the TD materialized view are large, resulting in many matching results, which forces the filtering stage to scan too many candidate objects, thus generating high overhead. (2) The connections between superpoints contain repeated node connections and edge connections, which distorts the rewritten query and thus hinders the algorithm from generating the optimal search order. MQO achieves comparable efficiency to the present invention, but its materialization cost is significantly higher. Finally, this invention consistently achieved significant query acceleration in all evaluation scenarios.

[0081] This invention can be used in fields such as financial risk control, bioinformatics, social network analysis, and knowledge graph retrieval.

[0082] For example, in the scenario of community structure discovery in social networks, the method of this invention can be used to process the graph data composed of user nodes and attention relationship edges. The processing mainly includes: first, defining common tight community patterns (such as "triangular closure" and "k-kernel") as a set of basic query subgraphs; then, using the method of this invention, pre-compiling and persistently storing all matching instances of these subgraph patterns in the global network; finally, a materialized view library is obtained, which systematically organizes all user subgroups that conform to the predetermined community structure pattern, thereby providing pre-computed basic community structure data for upper-layer applications.

[0083] For example, in multi-hop relationship query scenarios on knowledge graphs, the method of this invention can be used to process the graph data containing entities (such as people, places, and organizations) and relationships (such as birthplace and employment). The processing mainly includes: First, analyzing the query logs and abstracting frequently accessed multi-hop relationship paths (such as "person A -> employment -> company B -> location -> city C") into query templates; then, using the method of this invention, selectively materializing these high-frequency templates and pre-calculating the entity pairs at both ends of the path and their complete relationship chains; finally, a set of materialized views of high-frequency complex relationships is obtained, thereby transforming the multi-step graph traversal operation required in the original query into a direct lookup or join operation on the materialized results.

[0084] Another embodiment of the present invention provides a view materialization system for subgraph matching queries, comprising: The tree partitioning module receives the view and uses a search-based tree partitioning algorithm to obtain the tree partitioning of the view. The materialized view building module is used to construct materialized views based on tree partitioning; The query module receives queries and accelerates query responses through materialized views.

[0085] The above division of modules is merely illustrative. In practical applications, the functions described above can be assigned to different functional modules as needed to complete all or part of the functions described in the aforementioned method. The specific working process of each module can be found in the corresponding processes in the aforementioned method embodiments.

[0086] Another embodiment of the present invention provides a computer device (computer, server, smartphone, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing steps of the method of the present invention.

[0087] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk) that stores a computer program, which, when executed by a computer, implements the steps of the method of the present invention.

[0088] Another embodiment of the present invention provides a computer program product, the computer program product including a computer program, which, when executed by a computer, implements the steps of the method of the present invention.

[0089] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and to implement it accordingly. Those skilled in the art will understand that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the content disclosed in the embodiments of this specification; the scope of protection of the present invention is defined by the claims.

Claims

1. A view materialization method for subgraph matching queries, characterized in that, Includes the following steps: Receive the view and use a search-based tree partitioning algorithm to obtain the tree partitioning of the view; Construct a materialized view based on tree partitioning; Receive queries and accelerate query responses using materialized views; The materialized view is constructed based on the tree partitioning, and the materialized view consists of two parts: 11) Candidates for a superpoint: for a superpoint store its candidates in a table , candidate need to satisfy: a) is a match of the superpoint in the data graph; b) satisfies the filter condition: for any neighbor of , there exists a neighbor of ;​​ 12) Candidates for superedges: For each superedge Record all its candidates; Furthermore, the following two strategies are employed to prevent the materialization space from becoming too large: 21) Edge Compression: Group candidate matches according to the values ​​on the port nodes, and then perform hyperedge connections on a group-by-group basis to achieve compression of the hyperedge candidate set; set a threshold. When materializing a candidate edge for a hyperedge, the candidate edges are grouped according to the port nodes, and then the edges are materialized group by group. If the number of materialized edges exceeds [a certain threshold], [the process continues]. If the materialization of candidate edges is terminated, the unmaterialized group will be marked as unmaterialized. 22) Further segmentation of excessively large super-vertexes: First, estimate the number of subgraph matches for each super-vertex and set a threshold. As a limit on the number of matches exceeding the limit, when the estimated number of matches exceeding the limit... The superpoint is then segmented. The method of accelerating query responses through materialized views includes: 31) Rewrite the query based on the materialized view to construct a rewritten query, which consists of two parts: a) A series of isomorphic mappings Each isomorphic mapping Both indicate that a subgraph in a query graph is isomorphic to a view, and each query node can appear in at most one isomorphic mapping; b) A graph partition Each of these superpoints Either it contains one that does not belong to any nodes It either contains a set of nodes These nodes through In Mapped to the same view mode ,and belong The same superpoint ; 32) Enumeration is performed using an enumeration algorithm based on rewrite query.

2. The method according to claim 1, characterized in that, The search-based tree partitioning algorithm includes the following steps: 41) View Perform a depth-first search to obtain its depth-first search order, denoted as . ; 42) Perform multiple iterations, the first... Wheel extension width less than or equal to The state, except for the first Round, the first The initial state of each wheel is an expansion of the width of the previous wheel. The state; the first The initial state of the wheel contains only one superpoint, and that superpoint contains only one node. Each iteration starts the search from the initial state; 43) From the state Start the search. The node representing the newly joined state , Represented as ,in express Each superpoint generates a new state through state transition. ; 44) Check all new states obtained from state transitions, and only retain valid new states. Validity checks include: a) the current partition is tree-like; b) all superverts are either connected or may be connected in the future. 45) If the width of the new state is greater than d, then the state will not be processed in the current iteration, but will be stored for processing in the next iteration. 46) If the new state already contains a view If all nodes in the search are empty, terminate the search and return immediately; 47) If the new state is not the terminating state, use the new state for recursive search; 48) The process ends if all non-terminating new states have completed the recursive search. Search processing.

3. The method according to claim 2, characterized in that, Step 43) Generate a new state using the following two state transition methods. : a) Create a separate superpoint for the new node: ; b) Add the new node to an existing superpoint: Iterate through all superpoints, for the first superpoint... One super point, through Add this superpoint to construct a new state.

4. The method according to claim 2, characterized in that, The search-based tree partitioning algorithm is optimized using the following two methods: 51) Iterative wide-limit search: The partitioning process is broken down into multiple iterations, with the width limit gradually increasing in each iteration. In each iteration, states that exceed the current width limit are not expanded but are marked as reserved states and saved. If a complete partition that satisfies the current width limit is found, it is returned as the optimal solution. Otherwise, the width limit is increased and the next iteration starts from the previously stored reserved states, thus effectively avoiding redundant search. 52) Connectivity pruning: Introduce an index ,for Each pair of nodes in and each integer ,use Represents a node and Whether it passes The last one A path consisting of nodes is connected, for the state Each superpoint in the list is defined if it contains a pair of nodes that are not yet connected and cannot be connected by the remaining unpartitioned nodes. If so, then prune that state.

5. The method according to claim 1, characterized in that, The enumeration algorithm based on rewriting queries includes the following steps: 61) Received parameters: Current partial match Rewrite query Search order The currently enumerated superpoint labels And the candidate set for each superpoint ; 62) Update the answer: Check the current partial match. Does it constitute a complete match? If so, match that part. Add to the results list and return; otherwise, expand the partial match. 63) Obtaining a local candidate set: based on partial matching Enumerate the next superpoint The possible values ​​of the local candidate set are called local candidates. : 64) Extended Partial Matching: Enumeration All possible values, for a given value Construct new partial matches Then recursively proceed to the search of the next superpoint; 65) Backtracking: After all local candidate values ​​have been enumerated, the process ends and returns.

6. A view materialization system for subgraph matching query employing the method described in any one of claims 1 to 5, characterized in that, include: The tree partitioning module receives the view and uses a search-based tree partitioning algorithm to obtain the tree partitioning of the view. The materialized view building module is used to construct materialized views based on tree partitioning; The query module receives queries and accelerates query responses through materialized views.

7. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 5.

Images