An efficient subgraph matching query method across database systems

Abstract

The application discloses a kind of efficient subgraph matching query methods across database system, utilize pattern graph cardinality constraint information to reduce the influence of vertex number and edge number increase on subgraph cardinality estimation precision;Utilize sparse sampling graph technology to improve the accuracy of query subgraph matching cardinality estimation with attribute filtering;Based on query subgraph cardinality estimation value calculation execution overhead, from bottom to top search optimal execution plan tree;On the basis of optimal execution plan tree, for different database system, the optimization query statement that can guide execution plan is generated, realizes efficient subgraph matching query across database system.The application solves the problem that existing method is difficult to realize the unified management and optimization across database, cannot give full play to the advantage of each database system, effectively improves the query efficiency when existing database system executes complex graph query, reduces query time.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of database query performance optimization, and particularly relates to a high-efficiency subgraph matching query method across database systems. BACKGROUND

[0002] Graph database systems are widely used in various industries to store and manage large amounts of graph-structured data. However, with the rapid growth of data volume and the diversification of business needs, the subgraph matching performance of graph databases is gradually becoming the main bottleneck in the query process. Graph database systems based on different types of query engines (such as AQL based on the PostgreSQL relational engine, and Neo4j based on the native graph engine) have different query optimization strategies and syntax structures. Traditional graph databases often only support a single query language, and their query optimization methods are often only suitable for a single system, and cannot meet complex cross-database or multi-database application scenarios. Therefore, how to implement efficient and unified query optimization for multiple graph databases has become an important technical problem. At the same time, the existing built-in subgraph cardinality estimation method of graph databases is too simple, and the estimation result of the cardinality of an attribute graph with high filtering attributes or low filtering attributes has a large deviation, resulting in an unreasonable execution plan generated by the database. How to optimize the accuracy of cardinality estimation and generate a better subgraph matching query plan is also a major difficulty.

[0003] Existing subgraph matching query optimization methods can be mainly divided into two categories: subgraph matching algorithm optimization and parallel subgraph matching query optimization methods. Subgraph matching algorithm optimization is usually implemented based on a backtracking framework, and various heuristic filtering strategies, index structures, and join types are used to improve the computational efficiency of subgraph matching. Parallel subgraph matching query optimization methods decompose the subgraph matching computation process into multiple independent sub-computation tasks, distribute the graph data to multiple computing nodes, and reduce the query time through multi-core multi-thread parallel or multi-machine multi-process parallel execution.

[0004] Subgraph matching query optimization methods estimate the cardinality of intermediate matching results and select a high-quality matching order to generate a low-computational-overhead query plan. A high-precision matching result cardinality estimation method is the basis for generating a high-quality subgraph matching query plan. Existing subgraph matching cardinality estimation methods can be divided into three categories: traditional methods, machine learning-based methods, and cardinality boundary-based methods. Traditional cardinality estimation methods support all mainstream database systems, and they rely on sparse graphs, different value counts, and the most common value list to optimize the precision of cardinality estimation. Machine learning-based methods use machine learning models to help determine the query cardinality of subgraph matching; cardinality boundary methods estimate the upper or lower bound of the query graph cardinality based on the degree sequence.

[0005] However, the existing performance optimization method for improving subgraph matching query only optimizes the performance of a single graph database, and needs to write specific query optimization logic for each database; when multiple graph database systems are involved, it is necessary to simultaneously understand and master the query optimization rules of multiple graph database systems, which has high development cost and is difficult to maintain. In addition, the query syntax and index mechanism of different graph databases differ greatly, resulting in complex compatibility problems in the query optimization process. The traditional subgraph matching performance optimization method cannot reuse common subgraph matching query optimization logic, implement unified management and optimization across graph databases, and cannot fully utilize the advantages of each graph database system. SUMMARY

[0006] The present application provides a high-efficiency subgraph matching query method across database systems, which supports generating an optimal logical execution plan for an input query graph, and generating corresponding graph query statements for multiple database systems according to the optimal logical execution plan, thereby supporting reducing the computational overhead of complex subgraph matching queries in multiple graph database systems and reducing the query difficulty across database systems.

[0007] TECHNICAL SOLUTION The high-efficiency subgraph matching query method across database systems comprises the following steps:

[0008] (1) An information query system is preset, the information query system receives an attribute graph data set G to be queried, processes the edge labels of the attribute graph data set G, so that each edge label is unique in the attribute graph data set, and imports the processed attribute graph data set into a target database;

[0009] (2) A data pattern graph S is extracted from the attribute graph data set G, the data pattern graph S stores the meta information of the attribute graph data set, and contains the vertex label set, the edge label set and the cardinality constraint relationship between the vertices and edges in the attribute graph data set; the cardinality constraint of each edge label is extracted, the cardinality constraint of the edge label is divided into one-to-many, many-to-one, one-to-one and many-to-many, the edge labels with one-to-many and many-to-one cardinality constraints are added to the set SelectedEdgeLabels, and the binary tuple composed of the edge label and the label of the injective vertex in the cardinality constraint is stored in the set ManyToOneExpansionSet;

[0010] (3) The attribute graph data set G is sparsely sampled, and the edge set of each edge label is sampled, and all the sampled edges and associated vertices constitute a sparse graph data set SampleG;

[0011] (4) Calculate the 1st to 3rd order Motif statistic features of the property graph dataset G, where the kth order Motif statistic feature refers to the number of occurrences of a Motif pattern subgraph containing k vertices in the original property graph dataset G;

[0012] (5) Store the Motif statistic features calculated in step (4) into a key-value storage structure HigherOrderStatisticMap, where the key is the Motif pattern graph and the value is the Motif statistic feature value;

[0013] (6) For a user input query graph Q, enumerate the query subgraphs of the query graph Q, and arrange the query subgraphs in an ordered list AllSubgraphs according to the number of vertices from small to large, and if the number of vertices is the same, arrange them according to the number of edges from small to large;

[0014] (7) Traverse each query subgraph p in the list AllSubgraphs, if p does not contain attribute constraints, use a matching result cardinality estimation method based on topological structure; if p contains attribute constraints, use a matching cardinality estimation method based on the combination of sparse graph query and formula estimation, estimate the matching result cardinality of the query subgraph p in the property graph dataset G, and store the cardinality estimation value in HigherOrderStatisticMap;

[0015] (8) Model the query graph Q subgraph matching query plan as a query tree, where each node in the query tree is a subgraph of Q, the root node of the query tree is Q itself, and the leaf nodes of the query tree are query subgraphs consisting of single vertices; each node of the query tree is generated by the child nodes according to the EXPAND-OUT, EXPAND-INTO and JOIN three query operators, where if a query subgraph p can be obtained by extending an edge e from a query subgraph p', and one of the endpoints of e is not in p', then it is considered that the tree node corresponding to p can be generated by the tree node corresponding to p' according to the EXPAND-OUT query operator; if a query subgraph p can be obtained by extending an edge e from a query subgraph p', and both endpoints of e are in p', then it is considered that the tree node corresponding to p can be generated by the tree node corresponding to p' according to the EXPAND-INTO query operator; if a query subgraph p can be obtained by merging query subgraph p1 and query subgraph p2, and p1 and p2 have common vertices, then it is considered that the tree node corresponding to p can be generated by the tree nodes corresponding to p1 and p2 through the JOIN query operator;

[0016] (9) According to the cardinality estimation results of all query subgraphs in the list AllSubgraphs, a bottom-up search strategy is adopted to generate a query tree T; each query subgraph p in AllSubgraphs is traversed to enumerate all possible generation modes of the tree node corresponding to p in the query tree T; according to the cardinality estimation values of the obtained tree node and its child nodes corresponding to the query subgraphs, the execution overhead of each generation mode is evaluated, the generation mode with the lowest overhead is selected, and the tree node corresponding to p, the child nodes and the query operator information in the generation mode are added to the query tree T; the query tree T obtained with the query graph Q as the root node is the optimal subgraph matching query plan;

[0017] (10) Different query statement generators are configured for different database systems, the query statement generator can convert the EXPAND-OUT, EXPAND-INTO and JOIN query operators in the query tree into specific database query statements, and use the query plan hint function provided by the database system to specify the execution order of each subquery in the query statement, the vertex in the graph at which binary connection is to be performed, and the specific connection mode used when performing binary connection, so that the execution plan generated by the query statement conforms to the subgraph matching order in the query tree to the greatest extent;

[0018] (11) The database query statement generated in step (10) is executed in the corresponding database system backend, and the query result generated by the database is returned to the user, realizing efficient subgraph matching query.

[0019] Further, the cardinality constraint of each edge label in step (2) refers to the mapping multiplicity relationship allowed by the edge label between the two vertices connected by the edge, including the cardinality constraint of the source vertex and the cardinality constraint of the target vertex, and the cardinality constraint is one of "multiple" or "one".

[0020] Further, the implementation process of extracting the cardinality constraint of each edge label in step (2) is as follows:

[0021] For each edge label Label e , all edges with the label are stored in a two-dimensional table T with two attributes src and dst, each edge corresponds to a record in the table, and the source vertex and target vertex numbers of the edge are src and dst attributes of the record; group all records in the two-dimensional table T according to the dst attribute, and calculate the number of records C in each group, calculate the maximum value of all group record numbers C, if the maximum value is greater than 1, the edge label Label ethe cardinality constraint of the source vertex of the edge is "many", otherwise it is "one"; group all records in the two-dimensional table T according to the src attribute, and calculate the number of records C in each group, calculate the maximum value of the number of records C of all groups, if the maximum value is greater than 1, the edge label Label e the cardinality constraint of the target vertex of the edge is "many", otherwise it is "one".

[0022] Further, the label of the single-shot vertex in step (2) is one of the types of the cardinality constraint of the edge label, that is, many-to-one or one-to-many, and the label of the vertex whose cardinality constraint is "many" in the source vertex or the target vertex of the edge label indicates that the vertex of the label can be connected to at most one edge of the label.

[0023] Further, the step (3) is implemented as follows:

[0024] each edge set is randomly sampled edges, where α is the sampling ratio, indicates the frequency of the edge with the label in the attribute graph data set G, M is the total number of edges in the attribute graph data set, and l is the total number of different edge labels; calculate the sparsification rate of the edge label:

[0025] Further, the step (7) is implemented as follows:

[0026] For a query subgraph p, if the key of the query subgraph is stored in HigherOrderStatisticMap, the cardinality estimation value of the query subgraph is directly obtained from HigherOrderStatisticMap; if there is a pattern subgraph p' which is only one edge and one vertex less than p, and neither the edge nor the vertex contains an attribute filtering condition, and the two-tuple composed of the label of the edge and the label of the other vertex on the edge exists in the ManyToOneExpansionSet generated in step (2), it indicates that the matching result of p can be obtained by one-to-one expansion of the matching result of p', and the cardinality estimation result of p is the cardinality estimation result of p' already stored in HigherOrderStatisticMap; otherwise, any existing subgraph number estimation method is used for estimation, and all two subgraphs p1, p2 which can constitute p are enumerated, and the matching cardinality of p is estimated as where F(x) is the cardinality estimation result of the subgraph x.

[0027] Further, the matching cardinality estimation method based on the combination of sparse graph query and formula estimation in step (7) is:

[0028] If the number of vertices contained in query subgraph p is less than or equal to 3, the number of matching results n of p in sparse graph dataset SampleG is calculated, and if n is not 0, the estimated cardinality is where k is the number of edges in p, and p i where k is the number of edges in p, and p

[0029] Further, the process of evaluating the execution overhead of each generation mode in step (9) is as follows:

[0030] According to different target database systems, a corresponding execution overhead calculation formula is defined for each query operator. If query subgraph p can be generated by query subgraph p' through the EXPAND-OUT query operator, the execution overhead of this generation mode is Cost(Plan'(p'))+ExpandCost(p'->p)+F(p), where Cost(Plan'(p')) refers to the execution overhead of the query sub-tree corresponding to query graph p', ExpandCost(p'->p) refers to vertex expansion execution overhead, and different parameters are set according to different target database systems. If query subgraph p can be generated by query subgraph p' through the EXPAND-INTO query operator, the execution overhead of this generation mode is Cost(Plan'(p'))+ExpandCost(p'->p)+F(p)+ExpandIntoFactorxF(p'), where ExpandIntoFactor is a factor parameter of the EXPAND-INTO operation. If query subgraph p can be generated by query subgraphs p1, p2 through the JOIN operation, the execution overhead of this generation mode is:

[0031] Cost(Plan'(p1))+Cost(Plan'(p2))+F(p)+(F(p1)+F(p2))xIntersectionNumxJoinFactor, where JoinFactor is an execution overhead parameter of the JOIN operation, and IntersectionNum is the number of different vertices intersected by p1 and p2.

[0032] Further, the query tree in step (9) is a binary tree, for each node p in the query tree, if the execution overhead of generating p from a child node p' through an EXPAND-OUT or EXPAND-INTO query operator is the lowest, the tree node p' is set as the left child of node p, the right child of p is set as null, and the query operator of p is updated to EXPAND-OUT or EXPAND-INTO; if the execution overhead of generating p from two child nodes p1 and p2 through a JOIN operation query operator is the lowest, the left and right child nodes of p are set as p1 and p2 respectively, and the query operator of p is updated to JOIN.

[0033] Further, the execution order of each subquery in the query statement is specified by using the query plan hint function provided by the database system in step (10):

[0034] For a node in the query tree and its corresponding query subgraph p, if p is a path structure, the starting vertex and starting table are specified by using the query plan hint function of SQL or Cypher language; if p is a tree structure or contains a ring structure, the subquery mechanism provided by SQL or Cypher language, or the way of splitting one query statement into multiple subquery statements is used to control the execution plan of the query statement.

[0035] Advantages: Compared with the prior art, the advantages of the present application are: the present application uses the information provided by the extracted data pattern graph S and the sparse sample graph SampleG in the attribute graph dataset G, reduces the influence of the increase of the number of vertices and edges on the estimation accuracy of the subgraph cardinality, and improves the accuracy of the query subgraph matching cardinality estimation with attribute filtering; the present application can generate a high-quality query tree for a query graph with attribute constraints, and generate an efficient query statement for different target database systems, improve the query efficiency and reduce the query time when the existing database systems (such as Neo4j and PostgreSQL) execute complex graph queries; the present application can generate an optimal execution plan for multiple database systems according to the input query pattern graph, and configure an optimal query statement generator for each database system, which can generate a corresponding query statement according to the abstract execution plan tree, control the database execution plan, and solve the problem that the existing method cannot realize unified management and optimization across databases and cannot fully exert the advantages of each database system. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 is a flowchart of the present application;

[0037] Figure 2 is a schematic diagram of an attribute graph dataset;

[0038] Figure 3This is a schematic diagram illustrating the calculation process of the cardinality constraint between the source and target vertices of the edge label;

[0039] Figure 4 yes Figure 2 A schematic diagram of the cardinality constraint information of the pattern graph corresponding to the attribute graph dataset shown;

[0040] Figure 5 This is a schematic diagram of the storage structure for first- to third-order statistics;

[0041] Figure 6 This is a schematic diagram of a cardinality estimation method for matching results based on topology.

[0042] Figure 7 This is a schematic diagram of a matching cardinality estimation method based on the combination of sparse graph query and formula estimation;

[0043] Figure 8 This is a schematic diagram of an abstract execution plan tree;

[0044] Figure 9 This is a comparison chart of the query times of the execution plans of this invention and Neo4j's native execution plan under LDBC standard query conditions;

[0045] Figure 10 This is a comparison chart of the query time of the execution plan of this invention and Neo4j's native execution plan under high-low filtering query conditions;

[0046] Figure 11 This is a comparison chart of the query time of the present invention and the native PostgreSQL execution plan under LDBC standard query conditions;

[0047] Figure 12 This is a comparison chart of the query time of the present invention and the native PostgreSQL execution plan under high-low filtering query conditions. Detailed Implementation

[0048] The present invention will now be described in further detail with reference to the accompanying drawings.

[0049] like Figure 1 As shown, the present invention provides an efficient subgraph matching query method across database systems, the specific implementation process of which is as follows:

[0050] Step 1: Run an information query system on a computing device. The information query system receives the attribute graph dataset G to be queried, processes the edge labels of the attribute graph dataset G to make each edge label unique in the attribute graph dataset, and imports the processed attribute graph dataset into the target database.

[0051] The target database system can be a relational database (such as PostgreSQL, MySQL, etc.) or a graph database (such as Neo4j, etc.).

[0052] Step 2: Extract the data schema S from the property graph dataset G, the data schema S stores the meta information of the property graph dataset, including the vertex label set, the edge label set and the cardinality constraint relationship between vertices and edges in the property graph dataset; for each edge label extract the cardinality constraint of the edge label, the cardinality constraint of the edge label describes the number limit of edges between the vertex labels connected by the edge label, the cardinality constraint of the edge label is divided into one-to-many, many-to-one, one-to-one and many-to-many four categories, add all edge labels with one-to-many and many-to-one cardinality constraints to the set SelectedEdgeLabels, and store the binary tuple composed of the edge label and the label of the injective vertex in the set ManyToOneExpansionSet. The label of the injective vertex refers to the label of the vertex whose cardinality constraint type is one of many-to-one or one-to-many, and the cardinality constraint of the source vertex or the target vertex of the edge label is “many”, which means that the vertex of the label can be connected to at most one edge of the label.

[0053] The cardinality constraint of each edge label refers to the mapping multiplicity relationship allowed by the edge label between the two vertices connected by the edge, including the source vertex cardinality constraint and the target vertex cardinality constraint, and the cardinality constraint can be one of “many” or “one”.

[0054] The cardinality constraint of each edge label in the property graph dataset is calculated using the relational algebra operation as shown in Figure 3 for each edge label Label e , store all edges with the label in a two-dimensional table T with two attributes src and dst, each edge corresponds to a record in the table, and the numbers of the source vertex and the target vertex of the edge are src and dst attributes of the record respectively; group all records according to the dst attribute in the two-dimensional table T, and calculate the number C of records in each group, calculate the maximum value of the number C of all groups, if the maximum value is greater than 1, the source vertex cardinality constraint of the edge label Label e is “many”, otherwise it is “one”; group all records according to the src attribute in the two-dimensional table T, and calculate the number C of records in each group, calculate the maximum value of the number C of all groups, if the maximum value is greater than 1, the target vertex cardinality constraint of the edge label Label e is “many”, otherwise it is “one”.

[0055] Step 3: Sparse sampling of the property graph dataset G, according to the sampling ratio parameter a set by the administrator, sample the edge set of each edge label, and randomly sample edges from each edge set, where indicates the label the frequency of the edge in the attributed graph dataset G, M is the total number of edges in the attributed graph dataset, and l is the total number of different edge labels; and calculating the sparseness rate of the edge label All the sampled edges and their associated vertices constitute the sparse graph dataset SampleG.

[0056] Step 4: calculating the 1st to 3rd order Motif statistical features of the original attributed graph dataset G, the kth order Motif statistical feature refers to the number of occurrences of a Motif pattern subgraph containing k vertices in the original attributed graph dataset G, the Motif pattern subgraph only considers the constraint conditions of vertex labels and edge labels, and k is referred to as a high-order level. All 1st to 3rd order Motif pattern subgraphs without any attribute constraints can be calculated by using a relational database.

[0057] Step 5: storing the Motif statistical features calculated in step 4 into a key-value storage structure HigherOrderStatisticMap, with the key being a Motif pattern graph and the value being a Motif statistical feature value.

[0058] Step 6: for a query graph Q input by a user, enumerating all query subgraphs of the query graph Q, arranging the query subgraphs in ascending order of the number of vertices, and if the number of vertices is the same, arranging the query subgraphs in ascending order of the number of edges, and storing the arrangement result into an ordered list AllSubgraphs.

[0059] Step 7: traversing each query subgraph p in the list AllSubgraphs, if p does not contain an attribute constraint, using a matching result cardinality estimation method based on a topological structure, if p contains an attribute constraint, using a matching cardinality estimation method based on a combination of sparse graph query and formula estimation, estimating the matching result cardinality of the query subgraph p in the attributed graph dataset G, and storing the cardinality estimation value into HigherOrderStatisticMap.

[0060] The implementation steps of the method for estimating the cardinality of the matching result based on the topology structure are as follows: for a query subgraph p, if the key of the query subgraph is stored in the HigherOrderStatisticMap, the cardinality estimation value of the query subgraph is directly obtained from the HigherOrderStatisticMap; if there is a pattern subgraph p' which has only one less edge and one less vertex than p, and the edge and the vertex do not contain attribute filtering conditions, and the two-tuple formed by the label of the edge and the label of the other vertex on the edge exists in the ManyToOneExpansionSet generated in step (2), it is indicated that the matching result of p can be obtained by one-to-one expansion of the matching result of p', and the cardinality estimation result of p is the cardinality estimation result of p' which has been stored in the HigherOrderStatisticMap; otherwise, any existing subgraph quantity estimation method can be used for estimation, all two subgraphs p1 and p2 which can constitute p are enumerated, and the matching cardinality of p is estimated as where F(x) is the cardinality estimation result of the subgraph x.

[0061] The implementation steps of the method for estimating the cardinality of the matching result based on the combination of the sparse graph query and formula estimation are as follows: if the number of vertices contained in the query subgraph p is less than or equal to 3, the number n of the matching results of p in the sparse graph data set SampleG is calculated, and if n is not 0, the cardinality is estimated as where ρ i The implementation steps of the method for estimating the cardinality of the matching result based on the combination of the sparse graph query and formula estimation are as follows: if the number of vertices contained in the query subgraph p is less than or equal to 3, the number n of the matching results of p in the sparse graph data set SampleG is calculated, and if n is not 0, the cardinality is estimated as

[0062] Step 8: Model the subgraph matching query plan as a query tree, each node in the query tree is a subgraph of Q, the root node of the query tree is Q itself, and the leaf node of the query tree is a query subgraph composed of a single vertex; each node of the query tree can be generated by the child nodes according to the EXPAND-OUT, EXPAND-INT O, and JOIN three query operators; if a query subgraph p can be obtained by extending an edge e of a query subgraph p', and one end point of e is not in p', then it is considered that the tree node corresponding to p can be generated by the tree node corresponding to p' according to the EXPAND-OUT query operator; if a query subgraph p can be obtained by extending an edge e of a query subgraph p', and both end points of e are in p', then it is considered that the tree node corresponding to p can be generated by the tree node corresponding to p' according to the EXPAND-INTO query operator; if a query subgraph p can be obtained by merging query subgraph p1 and query subgraph p2, and p1 and p2 have a common vertex, then it is considered that the tree node corresponding to p can be generated by the tree nodes corresponding to p1 and p2 through the JOIN query operator.

[0063] Step 9: According to the cardinality estimation results of all query subgraphs in the list AllSubgraphs, a bottom-up search strategy is adopted to generate a query tree T; each query subgraph p in AllSubgraphs is traversed, and all possible generation methods of the tree node corresponding to p in the query tree are enumerated (including the corresponding child nodes and query operators); according to the cardinality estimation values of the tree node and its child nodes corresponding to the query subgraph obtained in step 7, the execution overhead of each generation method is evaluated, the generation method with the lowest overhead is selected, and the tree node corresponding to p, the child nodes, and the query operator information in the generation method are added to the query tree T; the query tree T obtained with the query graph Q as the root node is the optimal subgraph matching query plan.

[0064] The execution overhead of each generation mode is evaluated, and the implementation steps are as follows: according to different target database systems, a corresponding execution overhead calculation formula is defined for each query operator, if the query subgraph p can be generated by the query subgraph p' through the EXPAND-OUT query operator, the execution overhead of the generation mode is Cost(Plan'(p'))+ExpandCost(p'->p)+F(p), wherein Cos(Plan'(p')) indicates the execution overhead of the query sub-tree corresponding to the query graph p', ExpandCost(p'->p) indicates vertex expansion execution overhead, and different parameters are set according to different target database systems; if the query subgraph p can be generated by the query subgraph p' through the EXPAND-INTO query operator, the execution overhead of the generation mode is Cost(Plan'(p'))+ExpandCost(p'->p)+F(p)+expandIntoFactorxF(p'), wherein ExpandIntoFactor is a factor parameter of the EXPAND-INTO operation; if the query subgraph p can be generated by the query subgraphs p1 and p2 through the JOIN operation, the execution overhead of the generation mode is

[0065] Cost(Plan'(p1))+Cost(Plan'(p2))+F(p)+(F(p1)+F(p2))xIntersectionNumxJoinFactor, wherein JoinFactor is an execution overhead parameter of the JOIN operation, and IntersectionNum is the number of different vertices intersected by p1 and p2.

[0066] Here, the query tree structure is a binary tree, for each node p in the query tree, if the execution overhead of generating p from the child node p' through the EXPAND-OUT or EXPAND-INTO query operator is the lowest, the tree node p' is set as the left child of the node p, the right child of p is set as null, and the query operator of p is updated to EXPAND-OUT or EXPAND-INTO; if the execution overhead of generating p from two child nodes p1 and p2 through the JOIN operation query operator is the lowest, the left child and right child nodes of p are set as p1 and p2 respectively, and the query operator of p is updated to JOIN.

[0067] Step 10: Different query statement generators are configured for different database systems, which can convert EXPAND-OUT, EXPAND-INTO and JOIN query operators in the query tree into specific database query statements, and specify the execution order of each subquery in the query statement, the binary connection to be performed at which vertex in the graph, and the specific connection method to be used when performing binary connection by using the query plan hint function provided by the database system, so that the execution plan of the query statement generated is maximally consistent with the subgraph matching order in the query tree.

[0068] The execution order of each subquery in the query statement is specified by using the query plan hint function provided by the database system, and the implementation steps are as follows: for a node in the query tree and its corresponding query subgraph p, if p is a path structure, the starting vertex and starting table are specified by using the query plan hint function hint of SQL or Cypher language; if p is a tree structure or contains a ring structure, the subquery mechanism provided by SQL or Cypher language is used, or the query statement is split into multiple subquery statements to control the execution plan of the query statement.

[0069] Step 11: The database query statement generated in step 9 is executed in the corresponding database, and the query result generated by the database is returned to the user, so as to realize efficient subgraph matching query.

[0070] As shown in Figure 2 , an example of an attribute graph data set G is given, which is composed of vertices and edges (relations connecting vertices), and each vertex and edge can have multiple key-value pair attributes. Figure 2 In the attribute graph data set shown in Figure 2 , the vertices with label "Person" can be connected to the vertices with label "City" through edges with label "isLocatedIn", and the vertices with label "University" can also be connected to the vertices with label "City" through edges with label "isLocatedIn".

[0071] Taking the attribute graph data set shown in Figure 2 as an example, for the edge label "Person_isLocatedIn_City", the records are grouped according to the dst vertices, and the number of records C in the v3 group is 1, the number of records C in the v4 group is 2, and the maximum value of the number of records C in all groups is 2, so the target vertex cardinality constraint of the edge label "Person_isLocatedIn_City" is "multiple", and similarly, the source vertex cardinality constraint of the edge label "Person_isLocatedIn_City" is "one".Figure 4 The data pattern graph extracted from the attribute graph dataset G is shown in FIG. 6, where the vertices represent the vertex labels of the attribute graph dataset G, and the edges represent the edge labels in the attribute graph dataset, where "*" in the data pattern graph represents that the cardinality constraint of the edge label and the vertex label at this end is "many", and "1" represents that the cardinality constraint of the edge label and the vertex label at this end is "one". Figure 2 The data pattern graph extracted from the attribute graph dataset G is shown in FIG. 6, where the vertices represent the vertex labels of the attribute graph dataset G, and the edges represent the edge labels in the attribute graph dataset, where "*" in the data pattern graph represents that the cardinality constraint of the edge label and the vertex label at this end is "many", and "1" represents that the cardinality constraint of the edge label and the vertex label at this end is "one".

[0072] Taking the pattern graph shown in FIG. 6 as an example, the edge labels with the cardinality constraint of one-to-many or many-to-one are "Person_isLocatedIn_City", "University_isLocatedIn_City", and "City_isPartOf_Country", and these edge labels are added to the set SelectedEdgeLabels, and the binary tuple composed of the edge label and the label of the injective vertex in the cardinality constraint of the edge label is stored in the set ManyToOneExpansionSet. For example, for the edge label "City_isPartOf_Country", "City" is the injective vertex label of the edge label, and the binary tuple stored in ManyToOneExpansionSet is (City_isPartOf_Country, City). Figure 4 Taking the social network attribute graph dataset with a scale factor of 1 under the LDBC standard as an example, the dataset has a total of 33 different edge labels, i.e. l is 33, the total number of edges M is 18970283, the value of a is set to 1%, and then

[0073] is about 5749. For each edge label dataset, if the number of edges is greater than 5749, the number of edges randomly sampled is 5749, otherwise the dataset of the edge label is directly used as the dataset of the edge in the sparse graph.

[0074] Figure 5 The data pattern graph extracted from the attribute graph dataset G is shown in FIG. 6, where the vertices represent the vertex labels of the attribute graph dataset G, and the edges represent the edge labels in the attribute graph dataset, where "*" in the data pattern graph represents that the cardinality constraint of the edge label and the vertex label at this end is "many", and "1" represents that the cardinality constraint of the edge label and the vertex label at this end is "one". Figure 2 ​The example of part of the high-order statistics in the attribute graph dataset G is shown. Level 1 shows the motif pattern subgraph and the total number of motif pattern subgraphs of all 1-layer high-order statistics in G. Level 2 shows the motif pattern subgraph and the total number of motif pattern subgraphs of all 2-layer high-order statistics in G. Level 3 shows the motif pattern subgraph and the total number of motif pattern subgraphs of part of 3-layer high-order statistics in G. All possible 1-3 layer high-order statistics motif pattern subgraphs in the pattern graph are searched by using the depth-first search method. The number of these motif pattern subgraphs is queried in PostgreSQL and stored as high-order statistics in HigherOrderStatisticMap. The key is the motif pattern subgraph, and the value is the total number of motif pattern subgraphs matched in the attribute graph dataset.

[0075] As shown in Figure 6 For the query subgraph p', the key of the query subgraph is stored in the HigherOrderStatisticMap stored in step (5). The cardinality estimation value 1 of the query subgraph can be directly obtained from HigherOrderStatisticMap; for the query subgraph p, it has one more edge and one more vertex than the motif subgraph p'. Neither the edge nor the vertex contains the attribute filter condition, and the binary tuple (City_isPartOf_Country, City) formed by the label of the edge and the label of the other vertex on the edge exists in the ManyToOneExpansionSet generated in step (2). Therefore, the cardinality estimation result F(p) of p is F(p').

[0076] As shown in Figure 7 In the estimation of the motif subgraph p1 with attribute filter conditions, because the number of vertices contained in p1 is 3, the sparse graph can be directly used for estimation. Assuming that the number of pattern graphs p1 matched in the database where the sparse graph data is imported in step (3) is 1, the sparsification ratio of the edge label "Person_isLocatedIn_City" obtained from SparsificationRatioMap is 0.5, and the sparsification ratio of the edge label "City_isPartOf_Country" is 0.25, the cardinality estimation value of the motif subgraph p1 calculated according to the formula is If no pattern graph p1 is matched, the cardinality estimation value of the motif subgraph p1 calculated according to the formula is The number of vertices of the motif graph p is greater than 3, and the union of its two sub-patterns p1 and p2 can constitute the complete p, and the intersection is p3. The estimation value of the cardinality F(p) of the motif graph p obtained by using the sub-patterns p1 and p2 is Because the subgraphs are estimated in ascending order of the number of vertices and edges, F(p1), F(p2), and F(p3) are already stored in the HigherOrderStatisticMap when estimating the cardinality of the pattern graph p.

[0077] Abstract execution plan tree as follows Figure 8 As shown, a greedy algorithm is used to build a query tree from bottom to top. Each time, the root node of the tree with the lowest execution cost is selected as the optimal execution plan for the current subgraph. At the same time, the operation steps and execution cost of the optimal execution plan are also stored in the root node. Figure 8 The query tree shown illustrates that the optimal execution plan for schema p6 is obtained by JOINing schemas p3 and p5. p3 is obtained from p2 via EXPAND-OUT, p2 from p1 via EXPAND-OUT, and p5 from p4 via EXPAND-OUT. The execution cost for p6 is Cost6 = Cost3 + Cost5 + F(p6) + (F(p3) + F(p5)) × 1 × JoinFactor. When used on Neo4j systems, the recommended value for JoinFactor is 3. Similarly, Cost5 can be calculated using the formula Cost5 = Cost4 + ExpandCost(p4->p5) + F(p5), where ExpandCost(p4->p5) = F(p4) × DBHits(Label). Person Label Person_knows_Person ), where DBHits(Label Person Label Person_knows_Person The value of Cost(plan(p3)) is the number of DBHits generated by expanding the edge labeled Person_knows_Person from a vertex labeled Person in advance, collected from the Neo4j system. Similarly, the value of Cost(plan(p3)) can also be calculated based on the estimated value of the pattern graph in the existing storage structure.

[0078] Neo4j supports advanced hints including Index hints, Scan hints and Join hints, where Index hints are used to specify which index the query optimizer should use as a starting point, Scan hints can force Cypher to not use an index that it might have used, and also specify a starting point, and Join hints can force a join at a specified point. PostgreSQL also provides the pg_hint_plan plugin to support the use of advanced hints such as SeqScan(table), NestLoop(table table[table...]) to adjust the execution plan. Taking the Neo4j system as an example, according to the execution plan tree of Figure 8 , the Cypher query generated by the query statement generator is as follows:

[0079] “MATCH(city:City)-[e1:City_isPartOf_Country]->(country:Country),

[0080] (person1:Person)-[e2:Person_isLocatedIn_City]->(city:City),

[0081] (person2:Person)-[e3:Person_knows_Person]-(person1:Person)

[0082] USING SCAN country:Country USING SCAN person2:Person USING JOIN ONperson1

[0083] WHERE country.name='Germany'

[0084] RETURN count(*)AS count”

[0085] To test the performance advantage of the method proposed in the present application compared with the existing method, the LDBC SNB dataset is used for testing, and the optimization effect of the method proposed in the present application on Neo4j system and PostgreSQL system in 6 typical BI queries given by the official under the SF1 scale dataset is measured. The 6 typical queries are long path pattern BI-3, BI-4, tree pattern BI-10a, single ring graph BI-14a, complex multi-ring graph BI-11, BI-17, and for each query, the standard query condition provided by the benchmark test and the special parameters combined with high-low filtering query condition are used for experiment, wherein high filtering refers to the remaining matching number after filtering is less than 20%, and low filtering refers to the remaining matching number after filtering is greater than 80%. Figure 9 、 Figure 10 、 Figure 11 、 Figure 12 The execution time ratio of the present application before and after optimization on Neo4j and PostgreSQL systems is given respectively, and the query time of the native execution plan of the system is set to 1. Under the standard query condition provided by the benchmark test, the query plan generated by the method of the present application compared with the query plan generated by the default optimizer of Neo4j on the Neo4j database, the query time is shortened by 32.66% on average, and the query time is shortened by 87.55% on BI-11. On the PostgreSQL database, the query plan generated by the method of the present application compared with the query plan generated by the default query optimizer of PostgreSQL, the query time is shortened by 65.29% on average, and the query time is shortened by 99.13% on BI-11. Under the query parameter combined with high-low filtering query condition, the query time of the method of the present application compared with the default query optimizer of Neo4j on the Neo4j database is shortened by 70.31% on average, and the query time is shortened by 81.68% on BI-3. The query time of the method of the present application compared with the default query optimizer of PostgreSQL on the PostgreSQL database is shortened by 42.76% on average, and the query time is shortened by 97.25% on BI-17. The experimental results show that the method proposed in the present application can effectively improve the matching efficiency, especially under part of the constraint condition, which can produce good optimization effect.

[0086] The above describes in detail the method for efficient subgraph matching query across database systems according to the present application, but apparently the specific implementation form of the present application is not limited to this. For those skilled in the art, various obvious changes made to it without departing from the spirit and scope of the claims of the present application are within the scope of protection of the present application.

Claims

1. A method for efficient subgraph matching query across database systems, characterized in that, Includes the following steps: (1) An information query system is preset. The information query system receives the attribute graph dataset G to be queried, processes the edge labels of the attribute graph dataset G so that each edge label is unique in the attribute graph dataset, and imports the processed attribute graph dataset into the target database. (2) Extract the data pattern graph S from the attribute graph dataset G. The data pattern graph S stores the metadata of the attribute graph dataset; it includes the vertex label set, edge label set, and cardinality constraint relationship between vertices and edges in the attribute graph dataset; extract each edge label. The cardinality constraints of edge labels are divided into four categories: one-to-many, many-to-one, one-to-one, and many-to-many. All edge labels with cardinality constraints of one-to-many and many-to-one are added to the set SelectedEdgeLabels, and the tuples consisting of the edge labels and the labels of the injective vertices in their cardinality constraints are stored in the set ManyToOneExpansionSet. (3) Sparsification sampling is performed on the attribute graph dataset G. The edge set of each edge label is sampled, and all the sampled edges and their associated vertices constitute the sparse graph dataset SampleG. (4) Calculate the first to third order Motif statistics of the attribute graph dataset G. The k-th order Motif statistics refer to the number of times the Motif pattern subgraph containing k vertices appears in the original attribute graph dataset G. (5) Store the Motif statistical features calculated in step (4) into the key-value storage structure HigherOrderStatisticMap, where the key is the Motif pattern map and the value is the Motif statistical feature value. (6) For the query graph Q input by the user, enumerate the query subgraphs of the query graph Q, arrange the query subgraphs in ascending order of the number of vertices, and if the number of vertices is the same, arrange them in ascending order of the number of edges. Store the arrangement results in an ordered list AllSubgraphs. (7) Traverse each query subgraph p in the list AllSubgraphs. If p does not contain attribute constraints, use the cardinality estimation method for matching results based on topology. If p contains attribute constraints, a matching cardinality estimation method based on sparse graph query and formula estimation is adopted to estimate the cardinality of the matching results of the query subgraph p in the attribute graph dataset G, and the cardinality estimate is stored in HigherOrderStaticMap; (8) Model the query plan matching subgraph of query graph Q as a query tree. Each node in the query tree is a subgraph of Q. The root node of the query tree is Q itself, and the leaf nodes of the query tree are query subgraphs composed of a single vertex. Each node of the query tree is generated by the child nodes according to the three query operators EXPAND-OUT, EXPAND-INTO, and JOIN. If a query subgraph p can be obtained by extending an edge e from the query subgraph p', and one of the endpoints of e is not in p', then the tree node corresponding to p is considered to be able to be obtained from the query subgraph p'. The tree nodes are generated according to the EXPAND-OUT query operator; if a query subgraph p can be obtained by extending an edge e from a query subgraph p', and both endpoints of e are in p', then the tree node corresponding to p is considered to be generated by the tree node corresponding to the query subgraph p' according to the EXPAND-INTO query operator; if a query subgraph p can be obtained by merging query subgraphs p1 and p2, and p1 and p2 have a common vertex, then the tree node corresponding to p is considered to be generated by the tree nodes corresponding to p1 and p2 through the JOIN query operator; (9) Based on the cardinality estimation results of all query subgraphs in the list AllSubgraphs, a bottom-up search strategy is adopted to generate a query tree T; traverse each query subgraph p in AllSubgraphs, enumerate all possible ways to generate the tree nodes corresponding to p in the query tree; based on the cardinality estimation values ​​of the query subgraphs corresponding to the obtained tree nodes and their child nodes, evaluate the execution cost of each generation method, select the generation method with the lowest cost, and add the tree nodes, child nodes and query operator information corresponding to p under this generation method to the query tree T; the obtained query tree T with the query graph Q as the root node is the optimal subgraph matching query plan; (10) Configure different query statement generators for different database systems. The query statement generator can convert the EXPAND-OUT, EXPAND-INTO and JOIN query operators in the query tree into specific database query statements. It can also use the query plan prompt function provided by the database system to specify the execution order of each subquery in the query statement, which vertex in the graph needs to be joined, and the specific connection method used when joining, so that the execution plan generated by the query statement conforms to the subgraph matching order in the query tree to the greatest extent. (11) Execute the database query statement generated in step (10) in the backend of the corresponding database system, and return the query results generated by the database to the user to achieve efficient subgraph matching query; The label of the injective vertex in step (2) is that the cardinality constraint type of the edge label is one of many-to-one or one-to-many, and the label of the vertex whose cardinality constraint of the source vertex or the target vertex in the edge label is "many" is the injective vertex label, indicating that the vertex of the label can be connected to at most one edge of the label. The process of using the cardinality estimation method based on topology for matching results in step (7) is as follows: For a query subgraph p, if the key of the query subgraph is stored in HigherOrderStatisticMap, the cardinality estimate of the query subgraph is directly obtained from HigherOrderStatisticMap; if there exists a pattern subgraph p' that is only one edge and one vertex less than p, and neither the edge nor the vertex contains attribute filtering conditions, and the tuple formed by the label of the edge and the label of the other vertex on the edge exists in the ManyToOneExpansionSet generated in step (2), then it means that the matching result of p can be obtained by expanding the matching result of p' one by one, and the cardinality estimate of p is the cardinality estimate of p' already stored in HigherOrderStatisticMap; otherwise, any existing subgraph number estimation method is used for estimation, and all two subgraphs p1 and p2 that can form p are enumerated, and the matching cardinality of p is estimated as follows. Where F(x) is the cardinality estimate of subgraph x; The matching cardinality estimation method adopted in step (7) that combines sparse graph query and formula estimation is as follows: If the number of vertices in the subgraph p is less than or equal to 3, then calculate the number of matching results n for p in the sparse graph dataset SampleG. If n is not 0, then estimate the cardinality. Where k is the number of edges in p, ρ i This refers to the sparsity ratio of the edge label of the i-th edge involved in p. If the value of n is 0, meaning the pattern graph cannot be found in the sparse graph dataset, then the estimated cardinality is...

2. The efficient subgraph matching query method across database systems according to claim 1, characterized in that, In step (2), the cardinality constraint of each edge label refers to the allowed mapping multiplicity relationship between the two vertices connected by the edge, including the cardinality constraint of the source vertex and the cardinality constraint of the target vertex. The cardinality constraint is one of two values: "multiple" or "one".

3. The efficient subgraph matching query method across database systems according to claim 1, characterized in that, Step (2) involves extracting each edge label. The implementation process of the cardinality constraint is as follows: For each side label Label e All edges with the specified label are stored in a two-dimensional table T with two attributes, src and dst. Each edge corresponds to a record in the table, and the source and destination vertex numbers of the edge are the src and dst attributes of that record, respectively. All records are grouped according to the dst attribute in table T, and the number of records C in each group is calculated. The maximum value of the number of records C in all groups is calculated. If the maximum value is greater than 1, the edge label is then assigned. e The source vertex cardinality constraint is "many", otherwise "one"; group all records according to the src attribute in the two-dimensional table T, calculate the number of records C in each group, calculate the maximum value of the number of records C in all groups, and if the maximum value is greater than 1, then the edge label is... e The target vertex cardinality constraint is "many", otherwise it is "one".

4. The efficient subgraph matching query method across database systems according to claim 1, characterized in that, The implementation process of step (3) is as follows: Each edge set is randomly sampled The edge, where α is the sampling ratio, Indicates that the label is The frequency of edges appearing in the attribute graph dataset G, M is the total number of edges in the attribute graph dataset, and l is the total number of different edge labels; calculate the sparsity rate of the edge labels:

5. The efficient subgraph matching query method across database systems according to claim 1, characterized in that, The process for evaluating the execution overhead of each generation method in step (9) is as follows: Based on different target database systems, a corresponding execution cost calculation formula is defined for each query operator. If a query subgraph p can be generated from a query subgraph p' using the EXPAND-OUT query operator, then the execution cost of this generation method is Cost(Plan'(p')) + ExpandCost(p'->p) + F(p), where Cost(Plan'(p')) refers to the execution cost of the query subtree corresponding to query graph p', and ExpandCost(p'->p) refers to the vertex expansion execution cost, which is set according to different target database systems. Different parameters; if the query subgraph p can be generated from the query subgraph p' using the EXPAND-INTO query operator, then the execution cost of this generation method is Cost(Plan'(p')) + ExpandCost(p'->p) + F(p) + ExpandIntoFactor × F(p'), where ExpandIntoFactor is the factor parameter of the EXPAND-INTO operation; if the query subgraph p can be generated from the query subgraphs p1 and p2 using the JOIN operation, then the execution cost of this generation method is: Cost(Plan'(p1))+Cost(Plan'(p2))+F(p)+(F([1)+F([2))×IntersectionNum×JoinFactor, where JoinFactor is the execution cost parameter of the Join operation, and IntersectionNum is the number of different vertices that p1 and p2 intersect.

6. The efficient subgraph matching query method across database systems according to claim 1, characterized in that, The query tree described in step (9) is a binary tree. For each node p in the query tree, if the execution cost of generating p from child node p' through the EXPAND-OUT or EXPAND-INTO query operator is the lowest, then the tree node p' is set as the left child of node p, the right child of p is set to null, and the query operator of p is updated to EXPAND-OUT or EXPAND-INTO. If the execution cost of generating p from two child nodes p1 and p2 through the JOIN operation query operator is the lowest, then the left and right child nodes of p are set to p1 and p2 respectively, and the query operator of p is updated to JOIN.

7. The efficient subgraph matching query method across database systems according to claim 1, characterized in that, Step (10) describes using the query plan hint function provided by the database system to specify the execution order of each subquery in the query statement as follows: For a node in the query tree and its corresponding query subgraph p, if p is a path structure, the query starting vertex and starting table are specified using the query plan hint function of SQL or Cypher language; If p is a tree structure or contains a ring structure, then the subquery mechanism provided by SQL or Cypher language is used, or a query statement is split into multiple subquery statements to control the execution plan of the query statement.

Images