View materialization method and system for sub-graph matching query

Abstract

The invention relates to subgraph matching query and view materialization operation in the field of graph databases, and discloses a view materialization method and system for subgraph matching query. The invention designs a sub-graph tree structure-based materialized view data structure, and the materialized overhead and the query optimization effect can be balanced. According to the method, a tree division algorithm is designed, and the minimum connected tree division can be efficiently calculated. The invention also designs a matched query algorithm for the data structure of the materialized view, and the materialized view can be efficiently utilized to perform query optimization. According to the view materialization method, materialization view construction can be completed at controllable space-time overhead, query answering can be greatly accelerated, and the method can be applied to the fields of financial risk control, bioinformatics, social network analysis, knowledge graph retrieval and the like.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of information technology, and relates to a subgraph matching query and view materialization operation in the field of graph database, in particular to a view materialization method and system for subgraph matching query. BACKGROUND

[0002] Subgraph matching is a basic task in graph analysis systems. Specifically, given a data graph and a query graph, the system will find all subgraphs in the data graph that match the query graph. In practical applications, graph query engines often need to process a large number of subgraph matching queries, many of which share the same substructure. Materializing the results of these common subqueries can achieve computational reuse, thereby significantly improving query efficiency. However, existing view materialization techniques either have high space overhead or limited acceleration effect. Next, the background technology is introduced from subgraph matching and materialized view: 1) Subgraph matching: The subgraph matching problem is defined as given a data graph and a query graph, finding all subgraphs in the data graph that match the query graph. The subgraph matching problem is NP-hard. This problem has been extensively studied. Existing subgraph matching techniques are often optimized for a single query. However, in practical applications, graph analysis systems will receive multiple queries over a period of time, and these queries often have common parts. In this case, subgraph matching techniques for single queries cannot fully utilize these common subqueries for optimization. Some works on subgraph matching utilize common queries between queries for query optimization. However, these works also have problems: due to the large number of subgraph matching query results, computing and storing common subqueries will result in huge time and space overhead. These works do not solve this problem.

[0003] 2) Materialized view: Materialized view technique is a technique that pre-computes frequently occurring queries or sub-queries (called views) and uses the materialized results of these views to speed up the subsequent related queries. Materialized views are widely used in relational databases and have been proven to significantly reduce redundant computation and thus optimize query performance. In recent years, the growing interest in graph data structures has sparked interest in materialized views on graphs. However, there are significant differences between relational data and graph data in terms of storage, data access, and computation, so the view needs to be redesigned to adapt to the graph structure. Specifically for materialized views for subgraph matching, on the one hand, some works achieve view materialization by computing and storing all matching results of subgraph matching. Due to the large number of subgraph matching query results, this method will cause great space and computation overhead. On the other hand, some works achieve view materialization by storing the candidate set of nodes or edges of the view. This method can complete view materialization with small time and space cost. However, the result extraction efficiency of the materialized view constructed based on this method is not as good as the former method. This is because the system still needs to search from the materialized view to obtain the matching results of the view in the data graph.

[0004] The existing view materialization techniques for subgraph matching either have great preprocessing time and space overhead or have limited acceleration effect. The existing subgraph matching techniques either only focus on single query optimization or focus on multi-query optimization, but computing and storing the matching results of sub-queries requires great time and space overhead, and has the same defects as the existing view materialization techniques for subgraph matching. SUMMARY

[0005] The present application designs a view materialization method and system for subgraph matching queries. The existing view materialization methods have two categories: 1) Query-based materialized view (QM view): This method enumerates and stores all matching results of the view. Since the matching results of the view are often very large, its materialization overhead is extremely high. 2) Node-based materialized view (VM view): This method only stores the candidate set of view nodes. This method has small construction overhead and low space occupation, but the query optimization is limited. The view materialization method of the present application can achieve a balance between the two, both completing the construction of the materialized view with controllable time and space overhead and greatly accelerating the query answer.

[0006] The technical scheme adopted by the present application is as follows: A view materialization method for subgraph matching queries, comprising the following steps: Receiving a view, obtaining a tree partition of the view using a search-based tree partition solving algorithm; Constructing a materialized view according to the tree partition; Receiving a query, and accelerating the query answer through the materialized view.

[0007] Further, the search-based tree partitioning algorithm comprises the following steps: 1) Perform a depth-first search on the view , and obtain its depth-first search order, denoted as ; 2) Perform multiple rounds of iteration, the th round only expands states with width less than or equal to , except for the th round, the initial state of the th round is a state with width expanded from the previous round; the initial state of the th round only contains one supernode, and the supernode only contains one node , each round of iteration starts from the initial state to perform a search; 3) Start searching from state , and denotes the newly added node of the state, denotes , where denotes supernodes, and new states are generated by state transition; ; 4) Check all new states obtained by state transition, and only keep the legal new states, the legality check includes: a) the current partition is tree-shaped; b) all supernodes are either connected or may be connected in the future; 5) If the width of the new state is greater than d, do not perform subsequent processing on the state in the current round of iteration, but store it for processing in the next round; 6) If the new state already contains all nodes in the view , terminate the search and return directly; 7) If the new state is not a termination state, use the new state to perform recursive search; 8) If all non-termination new states have completed recursive search, end the search processing of .

[0008] Further, the search-based tree partitioning algorithm is optimized in the following two ways: 1) Iterative expansion of width limit search: the partitioning process is divided into multiple rounds of iteration, and the width limit of each round of iteration is gradually increased; in each iteration, states exceeding the current width limit are not expanded, but are marked as reserved states and saved; if a complete partition satisfying the current width limit is found, it is returned as the optimal solution, otherwise the width limit is increased, and the next iteration starts from the previously stored reserved states, thereby effectively avoiding redundant search; 2) Connectivity pruning: introduce an index , for each pair of nodes in and each integer , let denote whether nodes and are connected by a path consisting of the last nodes in , for each hypernode in state , prune it if it contains a pair of nodes that are not connected and cannot be connected by the remaining unpartitioned nodes.

[0009] Further, the materialized view is constructed according to the tree partitioning, wherein the materialized view consists of two parts: 1) candidates of hypernodes; 2) candidates of hyperedges; and two strategies are adopted to prevent the materialized space from being too large: 1) edge compression; 2) further partitioning of oversized hypernodes.

[0010] Further, the query answering is accelerated by the materialized view, comprising: 1) rewriting the query according to the materialized view to construct a rewritten query; 2) performing enumeration based on the enumeration algorithm based on the rewritten query.

[0011] Further, the enumeration algorithm based on the rewritten query comprises the following steps: 1) receiving parameters: current partial match , rewritten query , search order , current enumerated hypernode label , and candidate set of each hypernode ; 2) updating the answer: judging whether the current partial match constitutes a complete match, if yes, adding the partial match to the result list and returning, otherwise expanding the partial match; 3) obtaining a local candidate set: enumerating possible values of the next hypernode according to the partial match , the values are called local candidates, and the local candidate set is calculated: 4) expanding the partial match: enumerating all values of , for a value , constructing a new partial match , and then recursively entering the search of the next hypernode; 5) backtracking: when all local candidate values are enumerated, ending and returning.

[0012] A view materialization system for subgraph matching query, comprising: a tree partition module for receiving a view, obtaining a tree partition of the view by using a search-based tree partition solving algorithm; a materialized view construction module for constructing a materialized view according to the tree partition; a query module for receiving a query and accelerating query answering by using the materialized view.

[0013] The key points of the present application are: 1) The present application designs a materialized view data structure based on a tree structure of subgraphs, which can balance the materialized overhead and query optimization effect.

[0014] 2) The present application designs a tree partition algorithm, which can efficiently calculate the minimum connected tree partition.

[0015] 3) The present application also designs a matching query algorithm for the materialized view data structure, which can efficiently use the materialized view for query optimization.

[0016] The beneficial effects of the present application are as follows: The present application can solve the drawbacks of QM view (excessive space-time overhead of materialization) and VM view (limited query optimization effect). The present application has been tested on real data sets. In terms of view materialization, the space-time cost of the view materialization of the present application is 2 orders of magnitude lower than that of QM view and similar technologies. In terms of query optimization, the view of the present application can achieve better acceleration effect than VM view, with a difference of up to two orders of magnitude. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is an example of query-based materialized view, node-based materialized view and the materialized view of the present application.

[0018] Figure 2 is a flowchart of the method of the present application.

[0019] Figure 3 is an example of tree partition algorithm.

[0020] Figure 4 is an example of edge compression.

[0021] Figure 5 is an example of rewritten query.

[0022] Figure 6 is the space overhead (MB) of the materialized view in the experiment.

[0023] Figure 7 is the time overhead (ms) of the materialization process in the experiment.

[0024] Figure 8 This is the subgraph matching query processing time (ms) in the experiment. Detailed Implementation

[0025] The present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0026] This invention presents a view materialization method for subgraph matching. Current view materialization methods for subgraph matching either incur significant time and space overhead or suffer from low efficiency in extracting materialization results. This invention, however, can construct a materialized view with relatively low time and space overhead, and such a materialized view provides high efficiency in extracting materialization results. The relevant definitions are given below: 1) Graph: A graph is a data structure consisting of nodes and edges. An edge can connect two nodes. In this invention, we focus on graphs where nodes have labels. Therefore, given a graph... It can be used Represented by triplets, among which express The set of nodes, express The set of edges, It is a mapping from a node to a label. Represents a node The tag.

[0027] 2) Subgraph matching: Subgraph matching is defined as a given data graph and query graph Find All of them The matching subgraph. The commonly used matching semantics here are homomorphism and isomorphism. exist A subgraph homomorphism can be obtained through a mapping This mapping needs to satisfy: 1) For any node The node it matches The tags should match the nodes. The tags are the same, that is ;2) For any edge in , There must be a matching edge in the middle, that is Subgraph isomorphism requires that the mapping be injective. In this invention, homomorphic semantics are considered. Since subgraph isomorphic matching is a subset of subgraph homomorphic matching, this invention can also be used for query optimization of subgraph isomorphism.

[0028] 3) Views: A view is a query obtained through specific rules. It is then materialized for subsequent query optimization. (This will be used in the following text.) Represents a view.

[0029] 4) Materialized View: A materialized view is a view... Corresponding data structure . Includes All matching results are shown in the data graph. When a new query contains... At that time, the graph analysis system can utilize pre-calculated... Speed ​​up queries.

[0030] Currently, materialized views for subgraph matching can be divided into two categories: query-based materialized views (QM views) and node-based materialized views (VM views). On the one hand, QM views directly store... All matches. This materialized view method can effectively optimize queries. However, because the number of matching results for the view is very large... This materialization method requires significant time overhead to construct the materialized view, and storing the view consumes a large amount of memory. On the other hand, the VM view stores a candidate set of nodes or edges for the view. The method for constructing this materialized view is as follows: first, obtain the candidate set for each node using its label. Then, the algorithm removes invalid candidate nodes using filtering rules. These filtering rules are defined as follows: data nodes... It is a query node Candidate nodes need to satisfy the following conditions: any neighbor There should also be corresponding neighbors. yes The candidate set. The VM view can be constructed in polynomial time and has a space footprint of... Compared to QM views, VM views are more efficient to build and have a smaller footprint. However, they cannot directly extract matching results from materialized views; instead, they must perform a search based on the materialized view to obtain matching results. Furthermore, VM views may contain invalid candidate nodes—nodes that meet the construction rules but do not appear in any match. These candidate nodes increase search costs. Moreover, building a materialized view that does not contain invalid candidate nodes has been proven to be NP-hard, making it difficult to quickly eliminate these candidate nodes. Therefore, VM views are less efficient at query optimization than QM views. Figure 1(d) and (e) illustrate the QM view and VM view, respectively. The QM view has 10,000 rows, and its size is even larger than the data graph. The VM view, while smaller in size, cannot directly provide matching results and contains invalid nodes. and .

[0031] In summary, current materialized view techniques for subgraph matching either have excessive space overhead (based on queries) or limited optimization effects (based on nodes).

[0032] To address the drawbacks of the two aforementioned approaches and integrate their advantages, this invention employs a subgraph-based view materialization method. First, the invention decomposes the view into several connected subgraphs, referred to as superverts. Superverts are connected by superedges. An edge exists between two superverts if and only if the two endpoints of an edge in the view are located in both superverts. Superverts are disjoint, and their union is the set of nodes in the view. The new graph formed by the superverts and superedges is called graph partitioning. This invention calculates a candidate set of superverts (i.e., all matching superverts) and then filters out invalid candidates using filtering rules similar to those in node-based materialization schemes, thereby completing the construction of the materialized view. Figure 1 (f) shows the view graph partitioning (The tree partitioning shown in the diagram will be explained in detail later.) The same rounded rectangle represents a superpoint. . Figure 1 (g) demonstrates based on Materialized view constructed The rounded rectangle This represents the matching results for the corresponding superpoints. These matches are stored in a table, with each column representing a match. The edges between matching results represent candidate superedges. The subgraph-based materialized view scheme is a compromise between the query-based materialized view scheme and the node-based materialized view scheme. On the one hand, it only needs to enumerate the candidate set for each superpoint; therefore, its construction cost and space usage are... ,in express The number of superpoints is much smaller than that of query-based materialization schemes. On the other hand, since the number of superpoints is less than the number of view nodes, its matching extraction efficiency is also higher than that of node-based materialization schemes.

[0033] Subgraph-based materialization schemes rely on graph partitioning. On one hand, the size of the supervertron determines the construction cost and space usage of the materialized view; the smaller the largest supervertron, the lower the materialized view overhead. On the other hand, when the graph is partitioned into a tree, the filtering rules can guarantee no invalid candidate matches. The proof is as follows: Consider a VM view where all nodes satisfying the filtering rules can construct a match containing that node; subgraph-based materialized views have similar properties. Therefore, given a view, we want to construct a graph partition that: 1) is tree-shaped; 2) has the largest supervertron as small as possible. This problem is defined as minimum-width connected tree partitioning. This invention designs an efficient search algorithm to quickly solve this problem. In addition, this invention also includes algorithms for query optimization using materialized views. These will be introduced later. Figure 2 This is a flowchart of the present invention. The process is divided into two parts: offline and online. The offline part receives views and constructs materialized views based on subgraphs for them. The online part receives queries, and when the query includes views, it rewrites the query. The rewritten query represents query optimization using materialized views. Finally, the final matching result is obtained through the rewritten query.

[0034] Next, the technical details of the present invention will be introduced from three aspects: tree partitioning, materialization details, and query response.

[0035] 1. Tree partitioning Given a view, we want to obtain a graph partition that satisfies the following conditions: 1) Each supervertex is internally connected. 2) Supervertexes are disjoint, and the union of the supervertexes is all nodes in the view. 3) This graph partition is tree-like. 4) Under the above conditions, the size of the largest supervertex is as small as possible.

[0036] The above problem is defined as the minimum connected tree partitioning problem, or simply tree partitioning. The size of the largest supernode is called the width of the tree partition. This invention designs a search-based tree partitioning algorithm. Starting from an empty partition, it proceeds sequentially... Add views iteratively The nodes in the invention. In the implementation of this invention, It is a depth-first search (DFS) order starting from any root node, and any connectedness order (i.e., each node is connected to at least one previously added node) is valid. Each search state... Representing a subset A division on, in which This represents the number of nodes in the current state. (For exploration...) The successor state of the next node Add to At each step, two constraints are imposed to ensure correctness: the partition must remain a tree, and each supernode must be connected or potentially connected in the future state (discussed in detail below). Accordingly, invalid states are pruned.

[0037] In particular, the search-based tree partitioning algorithm of the present invention comprises the following steps: 1) Depth-first search of the view , obtaining its depth-first search order, denoted as .

[0038] 2) Search: the algorithm performs multiple iterations, the th iteration only expands states with width less than or equal to . Except for the th iteration, the initial states of the th iteration are states with width expanded from the previous iteration. The initial states of the th iteration only contain one supernode, and the supernode only contains one node . The algorithm starts searching from these initial states in each iteration.

[0039] 3) State transition: the algorithm starts searching from state . denotes the newly added node , denotes , where denotes supernodes. The algorithm generates new states by the following two state transition methods. .

[0040] a) Form a new supernode with the new node: .

[0041] b) Add the new node to an existing supernode: the algorithm traverses all supernodes, and for the th supernode, the algorithm constructs a new state by adding to , i.e. .

[0042] 4) State legality check: the algorithm checks all new states generated by state transition, and only keeps the legal new states. The legality check includes: a) the current partition is a tree; b) all supernodes are either connected or potentially connected in the future state (discussed in detail below).

[0043] 5) State preservation: If the width of the new state is greater than d, the algorithm will not process this state in the current iteration, but will store it for processing in the next iteration.

[0044] 6) Answer Update: If the new state already contains... Having identified all nodes in the search (referred to as the terminating state), the algorithm has found the answer through iterative widening optimization (discussed in detail later). Therefore, the algorithm terminates the search and returns directly.

[0045] 7) Recursive search: If the new state is not the terminal state, the algorithm will use the new state for recursive search: take the new state as input and repeat steps 3)-7).

[0046] 8) The algorithm terminates when all non-terminating new states have completed the recursive search. Search processing.

[0047] Figure 3 An example of a search-based tree partitioning algorithm is shown, where each state is enclosed in a rounded rectangle and state transitions are indicated by arrows. Nodes with the same label are grouped together as supernodes. Although there are two possible solutions, ... and However, the algorithm stopped when it found the first answer.

[0048] Exhaustive search can be implemented based on state transitions. However, indiscriminately searching all states is inefficient. Therefore, this invention employs various optimization methods to optimize the search-based tree partitioning algorithm. The following are two significant optimization methods: (1) Iterative Wide-Constraint Search: This pruning strategy is inspired by Iterative Deeper Search (IDS). When V is sparse, the minimum width is often small, so searching for large-width states is unnecessary. To take advantage of this, this invention breaks the partitioning process into multiple iterations, with the width constraint gradually increasing in each iteration (steps 2-8 in the algorithm above). In each iteration, states exceeding the current width constraint are not expanded. Instead, they are marked as reserved states and saved for later use (step 5 in the algorithm above). If a complete partition that satisfies the current constraint is found, it is returned as the optimal solution. Otherwise, the constraint is increased, and the next iteration begins from the previously stored reserved states (step 2 in the algorithm above), thus effectively avoiding redundant searches.

[0049] (2) Connectivity pruning: generating new states Then, the algorithm checks each hypervertices in that state. If a disconnected hypervertices are found, it cannot be directly accessed. Perform pruning. Because these disconnected supernodes might become connected after new nodes are added in subsequent state transitions. If pruning is performed without further checks... Pruning might cause valid substates to be missed, thus leading to a missed correct answer. Therefore, this invention introduces an index. .for Each pair of nodes in and each integer ,use To represent nodes and Is it possible to pass? The last one The paths consisting of nodes are connected. This data structure can be preprocessed using a variant of the Floyd-Warshall algorithm. For the state... Each superpoint in the list is defined if it contains a pair of nodes that are not yet connected and cannot be connected by the remaining unpartitioned nodes. (Right now If the condition is false, pruning can be performed on that state.

[0050] exist Figure 3 The text shows an example of using these two optimization strategies: due to connectivity pruning, the state... Skipped because and Unable to pass through the remaining nodes Connected. State , and They were pruned because their width was 3, while the results had already been found in the stage with a width of 2.

[0051] 2. View materialization Once the tree partition of the view is obtained, a materialized view can be constructed based on the tree partition. It consists of two parts: 1) Candidates for superpoints: For a superpoint Store its candidates in a table. Candidate It needs to meet the following conditions: a) It is a match of the superpoint in the data graph; b) It meets the filtering conditions: for any neighbor ,exist Neighbors .

[0052] 2) Candidates for superedges: For each superedge Record all its candidates.

[0053] The views of this invention are complete and compact. Wherein, completeness means that for any... of the match can be extracted from . The compactness indicates that for any match of a hyperedge , it must belong to some of the match.

[0054] The above is the basic version of the materialization process. In addition to this, the present invention also adds two strategies to prevent the materialization space from being too large: 1) Edge compression: in the basic version, for each hyperedge , its candidate edges are stored as connections between the candidate nodes of hypernode and . The time and memory cost of materializing the candidate edges is , where and represent the candidate sets of and respectively. Such materialization cost can be very high, especially for large hypernodes that can have a large number of candidate nodes. Therefore, the redundancy of edges can be reduced by edge compression. The definition of port node is given, which is a key concept in edge compression: given a hyperedge , is a port node in if and only if there is at least one edge connecting to a node in . The definition of port node in is the same. Let denote the port nodes of hyperedge . According to the definition of port node, two candidate matches and of will have the same neighbors in if they have the same value on the port nodes. Therefore, the candidate matches can be grouped according to the value on the port nodes, and then the hyperedge connections are performed in groups, so as to realize the compression of the candidate set of the hyperedge. Figure 4 An example of edge compression is shown. For hyperedge , if edge compression is not used, 400 edges are needed. However, using edge compression, only 106 edges are needed.

[0055] By the technique of hyperedge compression, the size of the candidate edges of is reduced to . However, in some cases, the size of the candidate edges is still very large. In such cases, a threshold is set. When materializing the candidate edges of a hyperedge, the candidate edges are grouped according to the port nodes, and then the edge materialization is performed in groups. If the number of edges that have been materialized so far exceeds Then the materialization of the candidate edge is terminated. And the group is marked as unmaterialized. This mark will be used later when the query optimization is performed using the materialized view.

[0056] 2) Further splitting of large supernodes: Although the tree decomposition with the minimum width is computed, the number of candidate matches of a supernode can still be very large, which will consume a lot of space. Therefore, the further splitting strategy of large supernodes is designed to solve this problem. First, the number of subgraph matches of each supernode is estimated. Specifically, for a supernode, its spanning tree is first obtained, and then the number of matches of the spanning tree is obtained by dynamic programming method as the estimated value of the number of matches of the supernode. The present application sets a threshold as the limit of the number of matches of the supernode. When the estimated value of the number of matches of the supernode exceeds , the supernode is further split. Since the estimation method used by the present application is overestimation, large supernodes will not be missed. After the further splitting of the supernodes, the graph decomposition will no longer be tree-shaped, so that the compactness cannot be guaranteed. However, this is a necessary trade-off - sacrificing compactness to ensure the controllable size of the materialization space. In order to reduce the impact as much as possible, the algorithm requires the edge-node ratio of the graph decomposition to be as small as possible (edge-node ratio = 1 is tree decomposition), that is, the graph decomposition is as close to tree-shaped as possible, so as to reduce the cost of introducing loops.

[0057] The above are two strategies in the materialization process to ensure the controllable time and space overhead. In actual use, the and can be adjusted according to the available space, so as to balance the materialization overhead and the optimization effect of the materialized view.

[0058] 3. Query answering Next, how to use the materialized view to speed up query answering is introduced. Query answering is divided into two steps: 1) rewriting the query according to the view to construct a rewritten query; 2) enumeration based on the rewritten query.

[0059] The first step is to construct a rewritten query. The rewritten query indicates how to use the view to optimize the query. A rewritten query is composed of two parts: 1) A series of isomorphic mappings : Each isomorphic mapping indicates that a subgraph in the query graph is isomorphic to a view. Moreover, each query node can appear in at most one isomorphic mapping.

[0060] 2) A graph decomposition : Each supernode in it contains a subgraph that does not belong to any nodes of , either contains a set of nodes , which are mapped to the same view pattern by , and belong to the same hypernode in . For the second case, we call a materialized hypernode, and construct a mapping between and . Furthermore, if two materialized hypernodes and are mapped to the same view pattern , and all the edges between them can be matched by , then the hyperedge between them is also materialized, and mapped to the hyperedge in .

[0061] Figure 5 An example of rewriting a query is shown. Query is rewritten to by view and . The materialized hypernodes are boxed by solid lines, and they are mapped to the corresponding view hypernodes. The normal hypernodes are boxed by dashed lines, and they are mapped to null. The materialized hyperedges are drawn by thick lines.

[0062] Intuitively, we want the query graph to be covered by the view as much as possible. In this way, we can utilize more information of the materialized view to answer the query, instead of computing from scratch. However, it is NP-hard to select views to cover as many nodes of the query graph as possible, because it can be reduced to the exact cover problem, which is a famous NP-complete problem. Therefore, to prevent the rewriting from taking too much computation time, we use a greedy approach: select the view that covers the most nodes of the query graph, and the coverage is not covered yet, and repeat the step until there is no view left to choose. After the appropriate views are selected, we just need to construct according to the definition.

[0063] The second step is to use to answer the query. The query answering method of the present invention is similar to the query answering framework of existing subgraph matching algorithms. The difference is that the query answering of the present invention is based on hypernode granularity, and the candidate set of materialized hypernodes is provided by the materialized view.

[0064] ​​A general subgraph matching framework consists of three key components: filtering, ordering, and enumeration. Given a query graph, the framework first constructs a VM view online, filtering candidate nodes for each query node according to node labels and adjacency connectivity. While specific algorithms can adjust the VM view in different ways, the core principle remains consistent. Next, the search order is determined by reordering the query nodes. Then, the enumeration phase explores all possible matches in this order, typically employing a depth-first search (DFS) strategy. A partial match is maintained and extended by adding the mapping of query nodes to it in search order. For each query node , a local candidate set is derived from the current partial match and the initial candidate set. Each candidate node forms a new DFS branch by adding to . The mapping of is then further searched along this new branch. When a complete match is found or the local candidate set is empty, a branch ends. In both cases, the algorithm backtracks by removing the last mapping and explores other branches.

[0065] The enumeration step of the present invention employs a similar framework, which also consists of these three parts. However, the algorithm of the present invention performs the matching process based on the rewritten query, which gives rise to some differences: 1) Filtering: The goal of the filtering phase is to compute a candidate set for each super-node in the rewritten query . Some super-nodes in have been mapped to super-nodes in the view schema, and the super-node candidates in the materialized view can be used as the initial candidate set. However, these candidates are only filtered according to the topology of the sub-query that the view maps, not the entire query graph. Therefore, further filtering of these candidates is needed. The algorithm of the present invention constructs a VM view for the entire query graph . Any initial candidate of a super-node that contains a mapping not present in this VM view is excluded. Moreover, compared to in the view schema, in the rewritten query may contain additional internal edges. In this case, the algorithm of the present invention excludes those candidates that cannot match these edges. For un-materialized super-nodes that are not mapped to any view schema, their candidate set is directly set to the query nodes in the VM viewIn addition, MAVIS builds an extra VM view for each view mode in the offline phase. The candidate sets in these VM views can be used to initialize the VM view of Q in this step, thus speeding up the construction process. Since VM views are lightweight, materializing them does not incur much memory overhead.

[0066] 2) Ordering: Since the enumeration of the present invention is in superpoint granularity, in this step, we need to generate the ordering of superpoints, instead of the ordering of query nodes. Any existing subgraph matching ordering algorithm can be used. And, it only needs to be applied to the rewritten query graph consisting of superpoints and superedges only.

[0067] 3) Enumeration: Enumeration is usually the most computationally expensive phase in subgraph matching. This step generates search states in exponential relation to the size of the query graph. The algorithm of the present invention groups multiple query nodes into superpoints, and enumerates matching results in superpoint granularity. Therefore, the search space is significantly reduced, thus improving the search efficiency.

[0068] The enumeration process based on the rewritten query is very similar to the general subgraph matching framework. The present invention iteratively extends the partial match in a depth-first search (DFS) manner, thus enumerating matching results. According to the search order , the matching of each super node is added to , and is output when it contains the matching of all superpoints . In enumerating the matching of a super node , the local candidate set of needs to be calculated, which is selected from the candidate set generated in the filtering phase according to the current partial match . Specifically, there are two methods to calculate the local candidate in the algorithm. Suppose the neighbor set of that has been matched in is . For each , its matching result is denoted as . If the candidate superedge connecting and is materialized in the view, the candidates of connected to can be retrieved through these candidate superedges, denoted as . If all can be retrieved through candidate superedges, the local candidate of is calculated as However, due to space limitations (see the view materialization chapter) or because they are not mapped to any hyperedge in the view schema, some hyperedges are not materialized. In this case, a second approach is used that computes local candidates at the query node level by set intersection. This approach first determines a local candidate set for each port node in . . . The local candidates for can be computed by joining all 's neighbors, where is the set of matched neighbor nodes of . Then, the local candidates for are selected by joining the candidate set generated by the filter phase with each .

[0069] In particular, the enumeration algorithm of the present invention based on rewritten queries comprises the following steps: 1) The enumeration based on rewritten queries is implemented recursively, the procedure receives as parameters: the current partial match , the rewritten query , the search order , the current enumerated hypernode label , and the candidate set for each hypernode . The following steps are the steps of this recursive procedure.

[0070] 2) Update answer: the procedure first checks if the current partial match constitutes a complete match, if so, it adds the partial match to the result list and returns. Otherwise, it proceeds to the next step to expand the partial match.

[0071] 3) Get local candidates: the procedure enumerates the possible values for the next hypernode ( ) based on . These values are called local candidates. There are two ways to compute the local candidate set : a) If , are both materialized, .

[0072] b) Otherwise, the procedure enumerates all port nodes of and computes the local candidate set for these port nodes. Finally, the procedure determines by taking a value from the local candidate set of the port nodes and .

[0073] 4) Expanding partial matches: the process enumerates all values of . For a value , a new partial match is constructed. If the matches adopt isomorphic semantics, it is necessary to check if there are duplicate values in . Next, the process recursively enters the search of the next superpoint. Specifically, it passes into step 1) and , repeating steps 2)-4).

[0074] 5) Backtracking: when all local candidate values are enumerated, the process ends and returns.

[0075] The invention was experimented on four real-world data graphs: Human, YouTube, WordNet, and Twitter. To comprehensively evaluate the performance of the invention, the experiments generated a large number of query graphs with varying sizes and densities. Query graphs with an average degree of 3 or higher were classified as dense, and the rest were classified as sparse. The experiments used the size of the query graph and the first letter of its density classification to represent each query type. For example, `16d' indicates that the query graph contains exactly 16 nodes and is classified as a dense query type. For view pattern selection, the experiments identified frequent subgraphs as candidate views based on a frequent subgraph mining algorithm. Thereafter, the importance of each view was estimated by its size and the number of queries that could benefit from it. For each query type, the experiments selected the top 10 most important view patterns.

[0076] For the evaluation approach, in the materialization phase, the experiments configured 100 GB of memory limit and one hour of time limit for each query type. The execution time and memory consumption were recorded as efficiency indicators for each materialization algorithm. In the query answering experiment, since the previous subgraph matching work was based on isomorphic semantics, the experiments also configured all view-based methods to generate isomorphic matches accordingly. To be consistent with previous work, for each query, the query processing algorithm terminated when one of the following conditions was met: 1) all matches were found; 2) the number of matches reached ; 3) the algorithm execution time reached 300 seconds. The execution time was recorded as an efficiency indicator.

[0077] Materialization Experiments: This experiment implements VM, QM, Factorization-based View (FDB), Answer Graph (AG), Tree Decomposition-based View (TD), and MQO-based View (MQO). Factorization uses a trie-like f-representation to compress the matching results of view patterns. d-representation is an improved version of f-representation, further merging common subtrees and organizing the matching results in a more compact directed acyclic graph (DAG). This experiment uses d-representation for comparison. AG is a method for constructing VM views. Compared to traditional VM view construction, AG uses triangulation to divide the view into several triangles and filters out invalid candidate points / edges by maintaining the matching of the triangles. Tree decomposition is an algorithm similar to tree partitioning: given a graph... Construct a tree ,in Each node is A subset (referred to as a package), satisfying the following conditions: 1) All The union of is ;2) For any edge There exists a simultaneous containing and 3) For any node ,Include The overlapping packets form a connected subtree. The biggest difference between tree decomposition and tree partitioning is that the packets in tree decomposition intersect, while the superpoints in tree partitioning are disjoint. This property makes tree partitioning more suitable for materializing views, which will be demonstrated in the experiment. In this experiment, the superpoints of tree partitioning are replaced with packets in tree decomposition. There are edges between superpoints if and only if two superpoints are adjacent packets. The tree decomposition algorithm is implemented using a state-of-the-art algorithm. MQO is a subgraph matching multi-query optimization algorithm that uses superpoint partitioning similar to this invention to store view matches. However, there are several key differences between it and this invention: 1) Different partitioning algorithms: MQO uses a minimal tree partitioning algorithm, not the minimum tree partitioning algorithm; 2) Its superpoints must be internally connected; 3) MQO first enumerates the matching results of the view and then projects them onto the superpoint partitioning, while this invention calculates the matching of each superpoint and then performs connection filtering. This experiment follows the settings in the original MQO paper: the size of each superpoint is limited to no more than 3, and any excessively large superpoint will be randomly partitioned into superpoints not exceeding this limit.

[0078] Figure 6 and Figure 7The total space overhead of materialized views and the total time overhead of materialization process are shown in the figures. The cases where the space or time limit is exceeded, resulting in a failed materialization, are marked with a "•". The methods that fail due to memory limit (e.g., QM and TD on WordNet) have no bar in the time cost figure because they cannot be executed, and vice versa. The present invention is denoted as MAVIS in the figures. VM completes the materialization with less space and time overhead. However, its speedup effect in the subsequent experiments is generally modest (see below). QM exceeds the space limit in all cases. FDB times out on WordNet and Twitter due to its exponential time complexity. AG times out on WordNet and Twitter because maintaining the matching relationship of triangles results in excessive time cost. However, the materialized view constructed by AG is smaller than that of VM. TD exceeds the space limit on WordNet and Twitter because the size of its hypernode (bag) is too large, resulting in too many candidate matches. Also, the experiments observe that the width of TD is generally larger than the book partition width of the present invention. MQO only passes Human-16s and Human-16d; it times out in all other tests because of too many view matches. The experiments try to extend the time limit of MQO to 12 hours to materialize one view in Human-24s, but it still times out. Therefore, MQO is not suitable for view materialization in the case of complex views or large datasets. Finally, the present invention successfully materializes all cases under the given constraints.

[0079] Next, the experiments evaluate the query answering effect of the materialized view algorithms. To make a comprehensive comparison, the following subgraph matching algorithms are chosen as the baselines: RapidMatch (RM), VEQ, BICE, and BSX. The experiments implement these algorithms using their respective open-source codes. For the materialized views, for AG and VM, the experiments speed up the filter step of subgraph matching using their materialized views, and the other steps follow the implementation of RM. For the materialized view of FDB, the experiments extract the matching results of the view from it and extend them to the complete matching results of the query. For the materialized view of TD, the experiments construct a rewritten query for it following a similar method of the present invention. Specifically, if two hypernodes are adjacent bags in the tree decomposition or there is a query edge between them, then there is a hyperedge between them. The other steps follow the same procedure as MAVIS. For the materialized view of MQO, the experiments use its original query answering method.

[0080] Figure 8The experimental results are shown. The cases of materialization failure are either not shown in the figures or marked with a "·". In some scenarios (e.g., YouTube), both VM and AG show significant optimization effects. In addition, AG is superior to VM in terms of optimization effect because it can filter out more invalid candidates. However, since they mainly optimize the filtering step, their impact is limited in datasets where enumeration dominates the workload (e.g., WordNet). The materialized view of FDB is inefficient for queries on YouTube. This is because the matching results found in the view are not necessarily the matching results of the query, and FDB lacks a mechanism to filter such cases. The enumeration efficiency of TD is also not ideal for two reasons: (1) the scale of the hypernode in the TD materialized view is large, resulting in a large number of matching results, forcing the filtering stage to scan too many candidate objects, thus generating high overhead. (2) The connection between hypernodes contains repeated node connections and edge connections, which distorts the rewritten query, thus hindering the algorithm from generating the optimal search order. MQO achieves comparable efficiency to the present application, but its materialization cost is significantly higher. Finally, the present application always achieves significant query acceleration in all evaluation scenarios.

[0081] The present application can be used in the fields of financial risk control, bioinformatics, social network analysis, knowledge graph retrieval, etc.

[0082] For example, for the community structure discovery scenario in a social network, the method of the present application can be used to process the graph data composed of user nodes and follow relationship edges in the social network. The processing process mainly includes: first, define common close community patterns (such as "triangle closure", "k-core") as a set of basic query subgraphs; then, precompute and persistently store all matching instances of these subgraph patterns in the global network by the method of the present application; finally, obtain a materialized view library which systematically organizes all user subgroups that meet the predetermined community structure patterns, thereby providing the upper-layer application with pre-computed community structure basic data.

[0083] For example, for the multi-hop relationship query scenario on a knowledge graph, the method of the present application can be used to process the graph data containing entities (such as persons, places, organizations) and relationships (such as place of birth, employed by) in the knowledge graph. The processing process mainly includes: first, analyze the query log and abstract the frequently accessed multi-hop relationship paths (such as "person A -> employed by -> company B -> located at -> city C") as query templates; then, selectively materialize these high-frequency templates by the method of the present application, precompute the entity pairs at both ends of the path and their complete relationship chain; finally, obtain a set of materialized views about high-frequency complex relationships, thereby converting the multi-step graph traversal operation in the original query into direct lookup or join operation on the materialized results.

[0084] Another embodiment of the present application provides a view materialization system for subgraph matching query, comprising: a tree partition module configured to receive a view and obtain a tree partition of the view using a search-based tree partition solving algorithm; a materialized view construction module configured to construct a materialized view according to the tree partition; a query module configured to receive a query and accelerate query answering through the materialized view.

[0085] The above-mentioned modules are only illustrative, and in actual applications, the above-mentioned functions can be distributed by different functional modules to complete all or part of the functions described in the foregoing method. The specific working process of each module can refer to the corresponding process in the foregoing method embodiment.

[0086] Another embodiment of the present application provides a computer device (computer, server, smart phone, etc.), comprising a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the steps of the method of the present application.

[0087] Another embodiment of the present application provides a computer readable storage medium (such as ROM / RAM, magnetic disk, optical disk), which stores a computer program, and the computer program is executed by a computer to realize the steps of the method of the present application.

[0088] Another embodiment of the present application provides a computer program product, which comprises a computer program, and the computer program is executed by a computer to realize the steps of the method of the present application.

[0089] The specific embodiments of the present application disclosed above are intended to help understand the content of the present application and to implement the same, and those skilled in the art can understand that various substitutions, changes and modifications are possible without departing from the spirit and scope of the present application. The present application should not be limited to the content disclosed by the embodiments of the present application, and the protection scope of the present application is defined by the scope of the claims.

Claims

1. A view materialization method for subgraph matching queries, characterized in that, Includes the following steps: Receive the view and use a search-based tree partitioning algorithm to obtain the tree partitioning of the view; Construct a materialized view based on tree partitioning; Receive queries and accelerate query responses using materialized views.

2. The method of claim 1, wherein, The search-based tree partitioning algorithm includes the following steps: 1) On the view A depth-first search is performed to obtain its depth-first search order, denoted as ; 2) Perform multiple iterations, the first... Wheel extension width less than or equal to The state, except for the first Round, the first The initial state of each wheel is an expansion of the width of the previous wheel. The state; the first The initial state of the wheel contains only one superpoint, and that superpoint contains only one node. Each iteration starts the search from the initial state; 3) from state Start searching, a node representing a new state , representing , where represents a hyperpoint, generating a new state by state transition ; 4) Check all new states obtained from the state transition, and only retain valid new states. The validity check includes: a) the current partition is tree-shaped; b) all superverts are either connected or may be connected in the future; 5) If the width of the new state is greater than d, then the state will not be processed in the current iteration, but will be stored for processing in the next iteration; 6) If the new state already contains all the nodes in the view terminate the search and return directly; 7) If the new state is not the terminating state, use the new state for recursive search; 8) If all non-terminal new states have completed recursive search, then end the search process.

3. The method according to claim 2, characterized in that, Step 3) Generate new state by two state transition ways as follows : a) Forming a new node into a supernode alone: ; b) Adding a new node to an existing hypernode: traverse all hypernodes, for each hypernode, construct a new state by adding to to , i.e. .

4. The method of claim 2, wherein, The search-based tree partitioning algorithm is optimized using the following two methods: 1) Iterative wide-limit search: The partitioning process is broken down into multiple iterations, with the width limit gradually increasing in each iteration. In each iteration, states that exceed the current width limit are not expanded but are marked as reserved states and saved. If a complete partition that satisfies the current width limit is found, it is returned as the optimal solution. Otherwise, the width limit is increased and the next iteration starts from the previously stored reserved states, thus effectively avoiding redundant search. 2) Connectivity pruning: Introduce an index For each pair of nodes in and each integer , let denote whether nodes and are connected by a path consisting of the last nodes in . For each supernode in state , prune it if it contains a pair of nodes that are not connected and cannot be connected by the remaining unpartitioned nodes.

5. The method of claim 1, wherein, The materialized view is constructed based on the tree partitioning, and the materialized view consists of two parts: 1) Candidates for a superpoint: for a superpoint , store its candidates in a table , candidate need to satisfy: a) is a match of the superpoint in the data graph; b) satisfy the filter condition: for any neighbor of , there exists a neighbor of ; 2) Candidates for hyper-edges: for each hyper-edge record all its candidates; Furthermore, the following two strategies are employed to prevent the materialization space from becoming too large: 1) Edge compression: group the candidate matches according to the value on the port node, and then perform super-edge connection in groups, so as to realize the compression of the super-edge candidate set; set a threshold When materializing a candidate edge of a super-edge, group the candidate edges according to the port node, and then materialize the edges in groups. If the number of edges that have been materialized exceeds , terminate the materialization of the candidate edge, and mark the unmaterialized group as unmaterialized. 2) Further splitting of oversized super-nodes: First, estimate the number of sub-graph matches for each super-node, set a threshold As a limit on the number of matches for a super-node, when the estimate of the number of matches for a super-node exceeds Split the super-node.

6. The method of claim 1, wherein, The method of accelerating query responses through materialized views includes: 1) Rewrite the query based on the materialized view to construct a rewritten query, which consists of two parts: a) A series of isomorphic mappings Each isomorphic mapping Both indicate that a subgraph in a query graph is isomorphic to a view, and each query node can appear in at most one isomorphic mapping; b) A graph partition Each of these superpoints Either it contains one that does not belong to any nodes It either contains a set of nodes These nodes through In Mapped to the same view mode ,and belong The same superpoint ; 2) Enumeration is performed using an enumeration algorithm based on rewrite query.

7. The method according to claim 6, characterized in that, The enumeration algorithm based on rewriting queries includes the following steps: 1) Received parameters: Current partial match Rewrite query Search order The currently enumerated superpoint labels And the candidate set for each superpoint ; 2) Update the answer: Determine the current partial match. Does it constitute a complete match? If so, match that part. Add to the results list and return; otherwise, expand the partial match. 3) Obtaining the local candidate set: based on partial matching Enumerate the next superpoint The possible values ​​of the local candidate set are called local candidates. : 4) Extended partial matching: Enumeration All possible values, for a given value Construct new partial matches Then recursively proceed to the search of the next superpoint; 5) Backtracking: After all local candidate values ​​have been enumerated, the process ends and returns.

8. A view materialization system for subgraph matching queries, characterized in that, include: The tree partitioning module receives the view and uses a search-based tree partitioning algorithm to obtain the tree partitioning of the view. The materialized view building module is used to construct materialized views based on tree partitioning; The query module receives queries and accelerates query responses through materialized views.

9. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 7.

Images