Efficient sub-graph matching query method of cross-database system

Abstract

The invention discloses an efficient sub-graph matching query method of a cross-database system. The method comprises the following steps: reducing the influence of vertex number and edge number increase on sub-graph cardinal number estimation precision by utilizing pattern graph cardinal number constraint information; the accuracy of matching cardinal number estimation of the query sub-graph with attribute filtering is improved by utilizing a sparse sampling graph technology; calculating execution overhead on the basis of a query sub-graph cardinal number estimated value, and searching an optimal execution plan tree from bottom to top; on the basis of the optimal execution plan tree, optimized query statements capable of guiding execution plans are generated for different database systems, and efficient sub-graph matching query across the database systems is achieved. According to the method, the problems that cross-database unified management and optimization are difficult to realize and the advantages of each database system cannot be brought into full play in the existing method are solved, the query efficiency when the existing database system executes complex graph query is effectively improved, and the query time is shortened.

Description

Technical Field

[0001] The present invention belongs to the field of database query performance optimization, and in particular relates to an efficient subgraph matching query method across database systems. Background Art

[0002] Graph database systems are widely used in various industries to store and manage large amounts of graph structured data. However, with the rapid growth of data volume and the diversification of business needs, the subgraph matching performance of graph databases is gradually becoming the main bottleneck in the query process. Graph database systems based on different types of query engines (such as database AQL based on the PostgreSQL relational engine and Neo4j based on the native graph engine, etc.) have different query optimization strategies and grammatical structures. Traditional graph databases often only support a single query language, and their query optimization methods are often only applicable to a single system and cannot meet complex cross-database or multi-database application scenarios. Therefore, how to achieve efficient and unified query optimization for multiple graph databases has become an important technical problem. At the same time, the subgraph cardinality estimation method built into the existing graph database is too simple, and the cardinality estimation results of attribute graphs with high or low filtering attributes have large deviations, resulting in unreasonable execution plans generated by the database. How to optimize the accuracy of cardinality estimation and generate better subgraph matching query plans is also a major difficulty.

[0003] Existing subgraph matching query optimization methods can be mainly divided into two categories: subgraph matching algorithm optimization and parallel subgraph matching query optimization methods. Among them, subgraph matching algorithm optimization is usually implemented based on a backtracking framework, and various heuristic filtering strategies, index structures and connection types are used to improve the computational efficiency of subgraph matching. The parallel subgraph matching query optimization method decomposes the subgraph matching calculation process into multiple independently executable sub-computing tasks, distributes the graph data to multiple computing nodes, and reduces the query time through multi-core multi-linear parallel execution or multi-machine multi-process parallel execution.

[0004] Subgraph matching query optimization methods estimate the cardinality of intermediate matching results and select high-quality matching sequences to generate query plans with low computational overhead. High-precision matching result cardinality estimation methods are the basis for generating high-quality subgraph matching query plans. Existing subgraph matching cardinality estimation methods can be divided into three categories: traditional methods, machine learning-based methods, and cardinality bound-based methods. Traditional cardinality estimation methods support all mainstream database systems. They rely on sparse graphs, distinct value counts, and most common value lists to optimize the accuracy of cardinality estimation. Machine learning-based methods use machine learning models to help determine the query cardinality of subgraph matching; cardinality bound methods estimate the upper or lower bound of the query graph cardinality based on degree sequences.

[0005] However, existing methods for improving subgraph matching query performance optimization only optimize the performance of a single graph database, and require writing specific query optimization logic for each database; when multiple graph database systems are involved, it is necessary to be familiar with and master the query optimization rules of multiple graph database systems at the same time, which has high development costs and is difficult to maintain. In addition, the query syntax and indexing mechanisms of different graph databases vary greatly, resulting in complex compatibility issues during the query optimization process. Traditional subgraph matching performance optimization methods make it difficult to reuse common subgraph matching query optimization logic to achieve unified management and optimization across graph databases, and are unable to give full play to the advantages of each graph database system. Summary of the invention

[0006] Purpose of the invention: The present invention provides an efficient subgraph matching query method across database systems, which supports generating an optimal logical execution plan for an input query graph, and generates corresponding graph query statements for multiple database systems based on the optimal logical execution plan, thereby supporting reducing the computational overhead of complex subgraph matching queries in multiple graph database systems and reducing the difficulty of queries across database systems.

[0007] Technical solution: The present invention provides an efficient subgraph matching query method across database systems, comprising the following steps:

[0008] (1) An information query system is pre-set, which receives a property graph dataset G to be queried, processes edge labels of the property graph dataset G so that each edge label is unique in the property graph dataset, and imports the processed property graph dataset into a target database;

[0009] (2) Extract the data pattern graph S from the attribute graph dataset G. The data pattern graph S stores the metadata of the attribute graph dataset; it includes the vertex label set, edge label set, and cardinality constraint relationship between vertices and edges in the attribute graph dataset; extract each edge label The cardinality constraints of edge labels are divided into four categories: one-to-many, many-to-one, one-to-one, and many-to-many. All edge labels with one-to-many and many-to-one cardinality constraints are added to the set SelectedEdgeLabels, and the tuples consisting of the edge label and the labels of the injective vertices in its cardinality constraints are stored in the set ManyToOneExpansionSet;

[0010] (3) Sparsely sample the attribute graph dataset G and sample the edge set of each edge label. All sampled edges and their associated vertices constitute the sparse graph dataset SampleG.

[0011] (4) Calculate the 1st to 3rd order Motif statistical features of the attribute graph dataset G. The k-order Motif statistical feature refers to the number of occurrences of the Motif pattern subgraph containing k vertices in the original attribute graph dataset G;

[0012] (5) The Motif statistical features calculated in step (4) are stored in the key-value storage structure HigherOrderStatisticMap, where the key is the Motif pattern map and the value is the Motif statistical feature value;

[0013] (6) For the query graph Q input by the user, enumerate the query subgraphs of the query graph Q, and arrange the query subgraphs in ascending order according to the number of vertices. If the number of vertices is the same, arrange them in ascending order according to the number of edges. Store the arrangement results in an ordered list AllSubgraphs.

[0014] (7) Traverse each query subgraph p in the list AllSubgraphs. If p does not contain attribute constraints, use the matching result cardinality estimation method based on the topological structure; if p contains attribute constraints, use the matching cardinality estimation method based on the combination of sparse graph query and formula estimation to estimate the matching result cardinality of the query subgraph p in the attribute graph dataset G, and store the cardinality estimation value in HigherOrderStaticMap.

[0015] (8) The subgraph matching query plan of the query graph Q is modeled as a query tree. Each node in the query tree is a subgraph of Q. The root node of the query tree is Q itself, and the leaf nodes of the query tree are query subgraphs composed of a single vertex. Each node of the query tree is generated by a subnode according to the three query operators EXPAND-OUT, EXPAND-INTO, and JOIN. If a query subgraph p can be obtained by expanding an edge e from the query subgraph p', and one of the endpoints of e is not in p', then it is considered that the tree node corresponding to p can be obtained by the query subgraph p'. The tree nodes are generated according to the EXPAND-OUT query operator; if a query subgraph p can be obtained by expanding an edge e through the query subgraph p', and both endpoints of e are in p', then it is considered that the tree node corresponding to p can be generated by the tree node corresponding to the query subgraph p' according to the EXPAND-INTO query operator; if a query subgraph p can be obtained by merging the query subgraph p1 and the query subgraph p2, and p1 and p2 have common vertices, then it is considered that the tree node corresponding to p can be generated by the tree nodes corresponding to p1 and p2 through the JOIN query operator;

[0016] (9) Based on the cardinality estimation results of all query subgraphs in the list AllSubgraphs, a bottom-up search strategy is adopted to generate a query tree T; each query subgraph p in AllSubgraphs is traversed, and all possible tree node generation methods corresponding to p are enumerated in the query tree; based on the cardinality estimation values ​​of the obtained tree nodes and their child nodes corresponding to the query subgraphs, the execution overhead of each generation method is evaluated, and the generation method with the lowest overhead is selected, and the tree nodes, child nodes, and query operator information corresponding to p under the generation method are added to the query tree T; the query tree T obtained with the query graph Q as the root node is the optimal subgraph matching query plan;

[0017] (10) Different query statement generators are configured for different database systems. The query statement generator can convert the EXPAND-OUT, EXPAND-INTO and JOIN query operators in the query tree into specific database query statements, and use the query plan prompt function provided by the database system to specify the execution order of each subquery in the query statement, which vertex in the graph needs to be binary connected, and the specific connection method used when performing the binary connection, so that the execution plan generated by the query statement conforms to the subgraph matching order in the query tree to the greatest extent;

[0018] (11) Execute the database query statement generated in step (10) in the backend of the corresponding database system, and return the query results generated by the database to the user, thereby realizing efficient subgraph matching query.

[0019] Furthermore, the cardinality constraint of each edge label in step (2) refers to the mapping multiplicity relationship allowed by the edge label between the two vertices connected by the edge, including the cardinality constraint of the source vertex and the cardinality constraint of the target vertex, and the cardinality constraint is one of the two values ​​of "many" or "one".

[0020] Furthermore, step (2) extracts each edge label The cardinality constraint implementation process is as follows:

[0021] For each edge label Label e , store all edges with this label in a two-dimensional table T with two attributes src and dst. Each edge corresponds to a record in the table. The numbers of the source vertex and the target vertex of the edge are the src and dst attributes of the record respectively; group all records according to the dst attribute in the two-dimensional table T, and calculate the number of records C in each group, and calculate the maximum number of records C in all groups. If the maximum value is greater than 1, the edge label Label eThe source vertex cardinality constraint is "many", otherwise it is "one"; group all records according to the src attribute in the two-dimensional table T, and calculate the number of records C in each group, and calculate the maximum value of the number of records C in all groups. If the maximum value is greater than 1, the edge label Label e The target vertex cardinality constraint is "many" if , otherwise it is "one".

[0022] Furthermore, the label of the injective vertex in step (2) is a vertex label whose cardinality constraint type is one of many-to-one or one-to-many, and the cardinality constraint of the source vertex or the target vertex in the edge label is "many", indicating that the vertex with this label can be connected to at most one edge with this label.

[0023] Furthermore, the implementation process of step (3) is as follows:

[0024] Each edge set is randomly sampled edges, where α is the sampling ratio, Indicates that the label is The frequency of the edge in the attribute graph dataset G, M is the total number of edges in the attribute graph dataset, and l is the total number of different edge labels; calculate the sparsification rate of edge labels:

[0025] Furthermore, the implementation process of the matching result cardinality estimation method based on topological structure in step (7) is as follows:

[0026] For the query subgraph p, if the key of the query subgraph is stored in HigherOrderStatisticMap, the cardinality estimation value of the query subgraph is directly obtained from HigherOrderStatisticMap; if there is a pattern subgraph p' that has only one less edge and one less vertex than p, and neither the edge nor the vertex contains the attribute filtering condition, and the tuple consisting of the label of the edge and the label of another vertex on the edge exists in the ManyToOneExpansionSet generated in step (2), then it means that the matching result of p can be obtained by expanding the matching result of p' one by one, and the cardinality estimation result of p is the cardinality estimation result of p' already stored in HigherOrderStatisticMap; otherwise, any existing subgraph number estimation method is used for estimation, and all two subgraphs p1 and p2 that can form p are enumerated, and the matching cardinality of p is estimated to be Among them, F(x) is the cardinality estimation result of subgraph x.

[0027] Furthermore, the matching cardinality estimation method based on sparse graph query combined with formula estimation in step (7) is:

[0028] If the number of vertices contained in the query subgraph p is less than or equal to 3, then calculate the number of matching results n of p in the sparse graph dataset SampleG. If n is not 0, the estimated cardinality is where k is the number of edges in p, ρ i Refers to the sparsification ratio of the edge label of the ith edge involved in p. If the value of n is 0, that is, the pattern graph cannot be found in the sparse graph dataset, the estimated cardinality is

[0029] Furthermore, the implementation process of evaluating the execution overhead of each generation method in step (9) is as follows:

[0030] According to different target database systems, corresponding execution cost calculation formulas are defined for each query operator. If query subgraph p can be generated by query subgraph p' through EXPAND-OUT query operator, the execution cost of this generation method is Cost(Plan'(p'))+ExpandCost(p'->p)+F(p), where Cost(Plan'(p')) refers to the execution cost of the query subtree corresponding to query graph p', and ExpandCost(p'->p) refers to the vertex expansion execution cost. Different parameters; if the query subgraph p can be generated by the query subgraph p' through the EXPAND-INTO query operator, the execution cost of this generation method is Cost(Plan'(p'))+ExpandCost(p'->p)+F(p)+ExpandIntoFactor×F(p'), where ExpandIntoFactor is the factor parameter of the EXPAND-INTO operation; if the query subgraph p can be generated by the query subgraphs p1 and p2 through the JOIN operation, the execution cost of this generation method is:

[0031] Cost(Plan'(p1))+Cost(Plan'(p2))+F(p)+(F(p1)+F(p2))×IntersectionNum×JoinFactor, where JoinFactor is the execution cost parameter of the Join operation, and IntersectionNum is the number of different vertices intersected by p1 and p2.

[0032] Furthermore, the query tree described in step (9) is a binary tree. For each node p in the query tree, if the execution overhead of generating p from the child node p' through the EXPAND-OUT or EXPAND-INTO query operator is the lowest, then the tree node p' is set to the left child of the node p, the right child of p is set to null, and the query operator of p is updated to EXPAND-OUT or EXPAND-INTO; if the execution overhead of generating p from the two child nodes p1 and p2 through the JOIN operation query operator is the lowest, then the left child and right child nodes of p are set to p1 and p2 respectively, and the query operator of p is updated to JOIN.

[0033] Furthermore, in step (10), the query plan prompt function provided by the database system is used to specify the execution order of each sub-query in the query statement as follows:

[0034] For a node in the query tree and its corresponding query subgraph p, if p is a path structure, the query starting vertex and starting table are specified through the query plan hint function of the SQL or Cypher language; if p is a tree structure or contains a ring structure, the subquery mechanism provided by the SQL or Cypher language is used, or a query statement is split into multiple subquery statements to control the execution plan of the query statement.

[0035] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects: the present invention utilizes the information provided by the data pattern graph S extracted from the property graph dataset G and the sparse sampling graph SampleG, reduces the influence of the increase in the number of vertices and the number of edges on the accuracy of the subgraph cardinality estimation, and improves the accuracy of the matching cardinality estimation of the query subgraph with attribute filtering; the present invention can generate high-quality query trees for query graphs with attribute constraints, and generate efficient query statements for different target database systems, thereby improving the query efficiency and reducing the query time of existing database systems (such as Neo4j and PostgreSQL) when executing complex graph queries; the present invention can generate the optimal execution plan for multiple database systems according to the input query pattern graph, and at the same time configures an optimized query statement generator for each database system, which can generate corresponding query statements according to the abstract execution plan tree, control the database execution plan, and solve the problem that the existing method is difficult to achieve unified management and optimization across databases and cannot give full play to the advantages of each database system. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flow chart of the present invention;

[0037] Figure 2 It is a schematic diagram of the property graph dataset;

[0038] Figure 3It is a schematic diagram of the cardinality constraint calculation process of the source and target vertices of the edge label;

[0039] Figure 4 yes Figure 2 Schematic diagram of the pattern graph cardinality constraint information corresponding to the property graph dataset shown;

[0040] Figure 5 It is a schematic diagram of the storage structure of first to third order statistics;

[0041] Figure 6 This is a schematic diagram of a matching result cardinality estimation method based on topological structure;

[0042] Figure 7 This is a schematic diagram of a matching cardinality estimation method based on a combination of sparse graph query and formula estimation;

[0043] Figure 8 It is a schematic diagram of the abstract execution plan tree;

[0044] Fig. 9 This is a comparison chart of query time between the present invention and Neo4j native execution plan under LDBC standard query conditions;

[0045] Fig.10 It is a comparison chart of query time between the present invention and Neo4j native execution plan under high-low filter query conditions;

[0046] Fig.11 This is a comparison chart of query time between the present invention and the native execution plan of PostgreSQL under the LDBC standard query condition;

[0047] Fig.12 This is a comparison chart of query time between the present invention and the native execution plan of PostgreSQL under high-low filtering query conditions. DETAILED DESCRIPTION

[0048] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0049] like Figure 1 As shown, the present invention provides an efficient subgraph matching query method across database systems, and the specific implementation process is as follows:

[0050] Step 1: Run an information query system on a computing device, which receives the property graph dataset G to be queried, processes the edge labels of the property graph dataset G so that each edge label is unique in the property graph dataset, and imports the processed property graph dataset into the target database.

[0051] The target database system can be a relational database (such as PostgreSQL, MySQL, etc.) or a graph database (such as Neo4j, etc.).

[0052] Step 2: Extract the data pattern graph S from the attribute graph dataset G. The data pattern graph S stores the metadata of the attribute graph dataset, including the vertex label set, edge label set, and cardinality constraint relationship between vertices and edges in the attribute graph dataset. Extract the cardinality constraint of the edge label. The cardinality constraint of the edge label describes the limit on the number of edges between the vertex labels connected by the edge label. The cardinality constraints of the edge label are divided into four categories: one-to-many, many-to-one, one-to-one, and many-to-many. Add all edge labels with cardinality constraints of one-to-many and many-to-one to the set SelectedEdgeLabels, and store the tuples consisting of the edge label and the labels of the injective vertices in its cardinality constraint into the set ManyToOneExpansionSet. The label of an injective vertex refers to the label of the vertex whose cardinality constraint type of the edge label is one of the two types of many-to-one or one-to-many, and the cardinality constraint of the source vertex or the target vertex in the edge label is "many", indicating that the vertex of the label can be connected to at most one edge of the label.

[0053] The cardinality constraint of each edge label refers to the mapping multiplicity relationship allowed by the edge label between the two vertices connected by the edge, including the cardinality constraint of the source vertex and the cardinality constraint of the target vertex. The cardinality constraint can be one of the two values ​​​​"many" or "one".

[0054] The cardinality constraint for each edge label in the property graph dataset is calculated using Figure 3 The relational algebra operation shown is calculated, for each edge label Label e , store all edges with this label in a two-dimensional table T with two attributes src and dst. Each edge corresponds to a record in the table. The numbers of the source vertex and the target vertex of the edge are the src and dst attributes of the record respectively; group all records according to the dst attribute in the two-dimensional table T, and calculate the number of records C in each group, and calculate the maximum number of records C in all groups. If the maximum value is greater than 1, the edge label Label e The source vertex cardinality constraint is "many", otherwise it is "one"; group all records according to the src attribute in the two-dimensional table T, and calculate the number of records C in each group, and calculate the maximum value of the number of records C in all groups. If the maximum value is greater than 1, the edge label Label e The target vertex cardinality constraint is "many" if , otherwise it is "one".

[0055] Step 3: Sparsely sample the attribute graph dataset G. According to the sampling ratio parameter α set by the administrator, sample the edge set of each edge label. Each edge set is randomly sampled. edge, where Indicates that the label is The frequency of the edge in the attribute graph dataset G, M is the total number of edges in the attribute graph dataset, l is the total number of different edge labels; calculate the sparsification rate of edge labels All sampled edges and their associated vertices constitute the sparse graph dataset SampleG.

[0056] Step 4: Calculate the 1st to 3rd order Motif statistical features of the original attribute graph dataset G. The k-order Motif statistical feature refers to the number of occurrences of the Motif pattern subgraph containing k vertices in the original attribute graph dataset G. The Motif pattern subgraph only considers the constraints of vertex labels and edge labels. k is called the high-order level. The relational database is used to calculate all 1 to 3 layers that the original attribute graph dataset can be composed of without any attribute constraints.

[0057] Step 5: Store the Motif statistical features calculated in step 4 into the key-value storage structure HigherOrderStatisticMap, where the key is the Motif pattern map and the value is the Motif statistical feature value.

[0058] Step 6: For the query graph Q input by the user, enumerate all query subgraphs of the query graph Q, and arrange the query subgraphs from small to large according to the number of vertices. If the number of vertices is the same, arrange them from small to large according to the number of edges, and store the arrangement results in an ordered list AllSubgraphs.

[0059] Step 7: Traverse each query subgraph p in the list AllSubgraphs. If p does not contain attribute constraints, use the matching result cardinality estimation method based on the topological structure. If p contains attribute constraints, use the matching cardinality estimation method based on the combination of sparse graph query and formula estimation to estimate the matching result cardinality of the query subgraph p in the attribute graph dataset G, and store the cardinality estimation value in HigherOrderStatisticMap.

[0060] The cardinality estimation method of matching results based on topological structure is implemented as follows: for a query subgraph p, if the key of the query subgraph is stored in HigherOrderStatisticMap, the cardinality estimation value of the query subgraph is directly obtained from HigherOrderStatisticMap; if there is a pattern subgraph p' that has only one edge and one vertex less than p, and neither the edge nor the vertex contains attribute filtering conditions, and the tuple consisting of the label of the edge and the label of another vertex on the edge exists in the ManyToOneExpansionSet generated in step (2), then it means that the matching result of p can be obtained by expanding the matching result of p' one by one, and the cardinality estimation result of p is the cardinality estimation result of p' already stored in HigherOrderStatisticMap; otherwise, any existing subgraph number estimation method can be used for estimation, and all two subgraphs p1 and p2 that can form p are enumerated, and the matching cardinality of p is estimated to be Where F(x) is the cardinality estimation result of subgraph x.

[0061] The matching cardinality estimation method based on sparse graph query and formula estimation is implemented as follows: if the number of vertices contained in the query subgraph p is less than or equal to 3, then the number of matching results n of p in the sparse graph dataset SampleG is calculated. If n is not 0, the estimated cardinality is where ρ i It refers to the sparsification ratio of edge label i stored in the SparsificationRatioMap generated in step (3). If the value of n is 0 (i.e., the pattern graph cannot be found in the sparse graph dataset), the estimated cardinality is

[0062] Step 8: Model the subgraph matching query plan as a query tree. Each node in the query tree is a subgraph of Q. The root node of the query tree is Q itself. The leaf nodes of the query tree are query subgraphs composed of a single vertex. Each node in the query tree can be represented by a subnode according to EXPAND-OUT, EXPAND-INT O, JOIN three query operators; if a query subgraph p can be obtained by extending an edge e through the query subgraph p', and one of the endpoints of e is not in p', then it is considered that the tree node corresponding to p can be generated by the tree node corresponding to the query subgraph p' according to the EXPAND-OUT query operator; if a query subgraph p can be obtained by extending an edge e through the query subgraph p', and both endpoints of e are in p', then it is considered that the tree node corresponding to p can be generated by the tree node corresponding to the query subgraph p' according to the EXPAND-INTO query operator; if a query subgraph p can be obtained by merging query subgraph p1 and query subgraph p2, and p1 and p2 have common vertices, then it is considered that the tree node corresponding to p can be generated by the tree nodes corresponding to p1 and p2 through the JOIN query operator.

[0063] Step 9: Based on the cardinality estimation results of all query subgraphs in the list AllSubgraphs, adopt a bottom-up search strategy to generate a query tree T; traverse each query subgraph p in AllSubgraphs, enumerate all possible tree node generation methods (including corresponding child nodes and query operators) in the query tree to generate p, and evaluate the execution overhead of each generation method based on the cardinality estimation value of the tree node and its child nodes corresponding to the query subgraph obtained in step 7, select the generation method with the lowest overhead, and add the tree nodes, child nodes and query operator information corresponding to p under this generation method to the query tree T; the query tree T obtained with the query graph Q as the root node is the optimal subgraph matching query plan.

[0064] The execution cost of each generation method is evaluated. The implementation steps are as follows: according to different target database systems, a corresponding execution cost calculation formula is defined for each query operator. If the query subgraph p can be generated by the query subgraph p' through the EXPAND-OUT query operator, the execution cost of this generation method is Cost(Plan'(p'))+ExpandCost(p'->p)+F(p), where Cos(Plan'(p')) refers to the execution cost of the query subtree corresponding to the query graph p', and ExpandCost(p'->p) refers to the vertex expansion execution cost. Different parameters are set according to different target database systems; if the query subgraph p can be generated by the query subgraph p' through the EXPAND-INTO query operator, the execution cost of this generation method is Cost(Plan'(p'))+ExpandCost(p'->p)+F(p)+expandIntoFactor×F(p'), where ExpandIntoFactor is the factor parameter of the EXPAND-INTO operation; if the query subgraph p can be generated by the query subgraph p' through the EXPAND-INTO query operator, the execution cost of this generation method is Cost(Plan'(p'))+ExpandCost(p'->p)+F(p)+expandIntoFactor×F(p'), where ExpandIntoFactor is the factor parameter of the EXPAND-INTO operation; 1. p2 is generated through JOIN operation. The execution cost of this generation method is

[0065] Cost(Plan'(p1))+Cost(Plan'(p2))+F(p)+(F(p1)+F(p2))×IntersectionNum×JoinFactor, where JoinFactor is the execution cost parameter of the Join operation, and IntersectionNum is the number of different vertices intersected by p1 and p2.

[0066] The query tree structure here is a binary tree. For each node p in the query tree, if the execution cost of generating p from the child node p' through the EXPAND-OUT or EXPAND-INTO query operator is the lowest, then the tree node p' is set to the left child of the node p, the right child of p is set to null, and the query operator of p is updated to EXPAND-OUT or EXPAND-INTO; if the execution cost of generating p from the two child nodes p1 and p2 through the JOIN operation query operator is the lowest, then the left child and right child nodes of p are set to p1 and p2 respectively, and the query operator of p is updated to JOIN.

[0067] Step 10: Different query statement generators are configured for different database systems. The query statement generator can convert the EXPAND-OUT, EXPAND-INTO and JOIN query operators in the query tree into specific database query statements, and use the query plan prompt function provided by the database system to specify the execution order of each subquery in the query statement, which vertex in the graph needs to be binary connected, and the specific connection method used when performing binary connection, so that the execution plan generated by the query statement conforms to the subgraph matching order in the query tree to the greatest extent.

[0068] The query plan hint function provided by the database system is used to specify the execution order of each subquery in the query statement. The implementation steps are as follows: for a node in the query tree and its corresponding query subgraph p, if p is a path structure, the query plan hint function of the SQL or Cypher language is used to specify the query starting vertex and starting table; if p is a tree structure or contains a ring structure, the subquery mechanism provided by the SQL or Cypher language is used, or a query statement is split into multiple subquery statements to control the execution plan of the query statement.

[0069] Step 11: The database query statement generated in step 9 is executed in the corresponding database, and the query result generated by the database is returned to the user, thereby realizing efficient subgraph matching query.

[0070] like Figure 2 As shown in Figure 1, an example of a property graph dataset G is given. The property graph consists of vertices and edges (relationships connecting vertices). Each vertex and edge can have multiple key-value pair attributes associated with it. Figure 2 In the property graph dataset shown, the vertex labeled "Person" can be connected to the vertex labeled "City" through the edge labeled "isLocatedIn", and the vertex labeled "University" can also be connected to the vertex labeled "City" through the edge labeled "isLocatedIn". Figure 2 Edge labels are placed on the edges of the graph.

[0071] by Figure 2 Taking the property graph dataset shown as an example, for the edge label "Person_isLocatedIn_City", all records are grouped according to the dst vertex, and the number of records C in the v3 group is 1, the number of records C in the v4 group is 2, and the maximum value of the number of records C in all groups is 2. Therefore, the target vertex cardinality constraint of the edge label "Person_isLocatedIn_City" is "many". Similarly, the source vertex cardinality constraint of the edge label "Person_isLoc atedIn_City" is calculated to be "one". Figure 4 Given from Figure 2 The data pattern graph extracted from the property graph dataset shown in the figure, the vertices in the data pattern graph represent the vertex labels of the property graph dataset G, and the edges in the data pattern graph represent the edge labels in the property graph dataset, where "*" in the data pattern graph indicates that the cardinality constraint between the edge label and the vertex label at that end is "many", and "1" indicates that the cardinality constraint between the edge label and the vertex label at that end is "one".

[0072] by Figure 4 Take the pattern graph shown in the figure as an example, where the edge labels with cardinality constraints of one-to-many or many-to-one are "Person_isLocatedIn_City", "University_isLocatedIn_City", and "City_isPartOf_Co untry", add these edge labels to the set SelectedEdgeLabels, and store the tuples consisting of the edge label and the labels of the injective vertices in its cardinality constraints into the set ManyToOneExpansionSet. For example, for the edge label "City_isPartOf_Country", "City" is the injective vertex label of the edge label, and the tuple stored in ManyToOneExpansionSet is (City_isPartOf_Country, City).

[0073] Taking the social network attribute graph dataset with a scale factor of 1 under the LDBC standard as an example, the dataset has 33 different edge labels, that is, l is 33, the total number of edges M is 18970283, and the value of α is set to 1%, then It is approximately 5749. For each labeled edge data set, if the number of its edges is greater than 5749, the number of edges randomly sampled is 5749. Otherwise, the data set of the edge label is directly used as the data set of the edge in the sparse graph.

[0074] Figure 5 Given Figure 2The following are examples of some high-order statistics in the property graph dataset G. Level 1 shows all Motif pattern subgraphs of 1-level high-order statistics and the total number of Motif pattern subgraphs in G. Level 2 shows all Motif pattern subgraphs of 2-level high-order statistics and the total number of Motif pattern subgraphs in G. Level 3 shows some Motif pattern subgraphs of 3-level high-order statistics and the total number of Motif pattern subgraphs in G. The depth-first search method is used to search for all possible Motif pattern subgraphs of 1 to 3 levels of high-order statistics in the pattern graph. The number of these Motif pattern subgraphs is queried in PostgreSQL and stored as their high-order statistics in HigherOrderStatisticMap. The key is the Motif pattern subgraph, and the value is the total number of Motif pattern subgraphs matched in the property graph dataset.

[0075] like Figure 6 As shown, for the query subgraph p', the key of the query subgraph is stored in the HigherOrderStatisticMap stored in step (5), and the cardinality estimate of the query subgraph 1 can be directly obtained from the HigherOrderStatisticMap; for the query subgraph p, it has one more edge and one more vertex than the pattern subgraph p', neither the edge nor the vertex contains attribute filtering conditions, and the tuple (City_isPartOf_Country, City) composed of the label of the edge and the label of another vertex on the edge exists in the ManyTo OneExpansionSet generated in step (2), so the cardinality estimate result of p is F(p) = F(p').

[0076] like Figure 7 As shown in the figure, when estimating the pattern subgraph p1 with attribute filtering conditions, because the number of vertices contained in p1 is 3, the sparse graph can be directly used for estimation. Assuming that the number of pattern graphs p1 matched in the database imported with sparse graph data in step (3) is 1, the sparsification ratio of the edge label "Person_isLocatedIn_City" obtained from SparsificationRatioMap is 0.5, and the sparsification ratio of the edge label "City_isPartOf_Country" is 0.25, then the cardinality estimate of the pattern subgraph p1 calculated according to the formula is If the pattern graph p1 is not matched, the estimated cardinality of the pattern subgraph p1 calculated according to the formula is The number of vertices of the pattern graph p is greater than 3. The union of its two sub-patterns p1 and p2 can form a complete p, and the intersection is p3. The estimated cardinality F(p) of the pattern graph p obtained using sub-patterns p1 and p2 is Because the estimation order of the subgraphs is sorted from small to large according to the number of points and edges, when estimating the cardinality of the pattern graph p, F(p1), F(p2), and F(p3) are all stored in the higher-order statistics HigherOrderStatisticMap.

[0077] The abstract execution plan tree is as follows Figure 8 As shown, a greedy algorithm is used to build a query tree from bottom to top. Each time, the root node of the tree with the lowest execution cost is selected as the optimal execution plan of the current subgraph. At the same time, the operation steps and execution cost of the optimal execution plan are also stored in the root node. Figure 8 The query tree shown shows that the optimal execution plan of pattern graph p6 is obtained by JOIN operation of pattern graphs p3 and p5, p3 is obtained by EXPAND-OUT operation of p2, p2 is obtained by EXPAND-OUT operation of p1, and p5 is obtained by EXPAND-OUT operation of p4. The execution cost corresponding to p6 is Cost6 = Cost3 + Cost5 + F(p6) + (F(p3) + F(p5)) × 1 × JoinFactor. When used in Neo4j system, the recommended value of JoinFactor is 3. Similarly, according to the formula, Cost5 = Cost4 + ExpandCost(p4->p5) + F(p5), where ExpandCost(p4->p5) = F(p4) × DBHits(Label Person ,Label Person_knows_Person ), where DBHits(Label Person ,Label Person_knows_Person ) is the number of DBHits generated by extending an edge labeled Person_knows_Person from a vertex labeled Person, which is collected in advance from the Neo4j system. Similarly, the value of Cost(plan(p3)) can also be calculated based on the estimated value of the pattern graph in the existing storage structure.

[0078] Use advanced hints and temporary subgraph construction methods in different databases to control the execution plan. Neo4j supports advanced hints including Index hints, Scan hints, and Join hints. Index hints are used to specify which index the query optimizer should use as the starting point. Scan hints can force Cypher to scan instead of using the index that could have been used. It can also specify a starting point. Join hints can force a connection at a specified point. PostgreSQL also provides the pg_hint_plan plug-in, which supports the use of advanced hints such as SeqScan(table) and NestLoop(tabletable[table...]) to adjust the execution plan. Taking the Neo4j system as an example, according to Figure 8 The execution plan tree of the query statement generator is as follows:

[0079] "MATCH(city:City)-[e1:City_isPartOf_Country]->(country:Country),

[0080] (person1:Person)-[e2:Person_isLocatedIn_City]->(city:City),

[0081] (person2:Person)-[e3:Person_knows_Person]-(person1:Person)

[0082] USING SCAN country:Country USING SCAN person2:Person USING JOIN ONperson1

[0083] WHERE country.name='Germany'

[0084] RETURN count(*)AS count"

[0085] In order to test the performance advantage of the method proposed in the present invention over the existing methods, the LDBC SNB data set was used for testing, and the optimization effect of the method proposed in the present invention on the Neo4j system and the PostgreSQL system in the 6 typical BI queries given by the official under the SF1 scale data set was measured. The 6 typical queries are long path mode BI-3, BI-4, tree mode BI-10a, single ring graph BI-14a, complex multi-ring graph BI-11, BI-17. For each query, the standard query conditions provided by the benchmark and the special parameters of the combination of high-low filtering query conditions were used for experiments, where high filtering refers to the number of remaining matches after filtering is less than 20%, and low filtering refers to the number of remaining matches after filtering is greater than 80%. Fig. 9 , Fig.10 , Fig.11 , Fig.12 The execution time ratios of the method before and after optimization on the Neo4j and PostgreSQL systems are given respectively. The query time of the native execution plan of the system is set to 1. Under the standard query conditions provided by the benchmark test, the query plan generated by the method of the present invention on the Neo4j database is 32.66% shorter than the query plan generated by the Neo4j default optimizer, and the query time on BI-11 is 87.55% shorter; on the PostgreSQL database, the query plan generated by the method of the present invention is 65.29% shorter than the query plan generated by the PostgreSQL default query optimizer, and the query time on BI-11 is 99.13% shorter. Under the query parameters of the high-low filter query condition combination, the query time of the method of the present invention on the Neo4j database is 70.31% shorter than the Neo4j default query optimizer on average, and the query time on BI-3 is 81.68% shorter; on the PostgreSQL database, the query time of the method of the present invention is 42.76% shorter than the PostgreSQL default query optimizer on average, and the query time on BI-17 is 97.25% shorter. Experimental results show that the method proposed in the present invention can effectively improve the matching efficiency, especially can produce better optimization effect under some constraints.

[0086] The above content has described in detail an efficient subgraph matching query method across database systems described in the present invention, but it is obvious that the specific implementation form of the present invention is not limited thereto. For those skilled in the art, various obvious changes made to it without departing from the spirit of the present invention and the scope of the claims are within the protection scope of the present invention.

Claims

1. An efficient subgraph matching query method across database systems, characterized in that: The following steps are involved: (1) An information query system is pre-set, which receives a property graph dataset G to be queried, processes edge labels of the property graph dataset G so that each edge label is unique in the property graph dataset, and imports the processed property graph dataset into a target database; (2) Extract the data pattern graph S from the attribute graph dataset G. The data pattern graph S stores the metadata of the attribute graph dataset; it includes the vertex label set, edge label set, and cardinality constraint relationship between vertices and edges in the attribute graph dataset; extract each edge label Label Ei The cardinality constraints of edge labels are divided into four categories: one-to-many, many-to-one, one-to-one, and many-to-many. All edge labels with one-to-many and many-to-one cardinality constraints are added to the set SelectedEdgeLabels, and the tuples consisting of the edge label and the labels of the injective vertices in its cardinality constraints are stored in the set ManyToOneExpansionSet; (3) Sparsely sample the attribute graph dataset G and sample the edge set of each edge label. All sampled edges and their associated vertices constitute the sparse graph dataset SampleG. (4) Calculate the 1st to 3rd order Motif statistical features of the attribute graph dataset G. The k-order Motif statistical feature refers to the number of occurrences of the Motif pattern subgraph containing k vertices in the original attribute graph dataset G; (5) The Motif statistical features calculated in step (4) are stored in the key-value storage structure HigherOrderStatisticMap, where the key is the Motif pattern map and the value is the Motif statistical feature value; (6) For the query graph Q input by the user, enumerate the query subgraphs of the query graph Q, and arrange the query subgraphs in ascending order according to the number of vertices. If the number of vertices is the same, arrange them in ascending order according to the number of edges. Store the arrangement results in an ordered list AllSubgraphs. (7) Traverse each query subgraph p in the list AllSubgraphs. If p does not contain attribute constraints, use the matching result cardinality estimation method based on the topological structure. If p contains attribute constraints, the matching cardinality estimation method based on sparse graph query and formula estimation is used to estimate the matching result cardinality of the query subgraph p in the attribute graph dataset G, and the cardinality estimation value is stored in HigherOrderStaticMap; (8) The subgraph matching query plan of the query graph Q is modeled as a query tree. Each node in the query tree is a subgraph of Q. The root node of the query tree is Q itself, and the leaf nodes of the query tree are query subgraphs composed of a single vertex. Each node of the query tree is generated by a subnode according to the three query operators EXPAND-OUT, EXPAND-INTO, and JOIN. If a query subgraph p can be obtained by expanding an edge e from the query subgraph p', and one of the endpoints of e is not in p', then it is considered that the tree node corresponding to p can be obtained by the query subgraph p'. The tree nodes are generated according to the EXPAND-OUT query operator; if a query subgraph p can be obtained by expanding an edge e through the query subgraph p', and both endpoints of e are in p', then it is considered that the tree node corresponding to p can be generated by the tree node corresponding to the query subgraph p' according to the EXPAND-INTO query operator; if a query subgraph p can be obtained by merging the query subgraph p1 and the query subgraph p2, and p1 and p2 have common vertices, then it is considered that the tree node corresponding to p can be generated by the tree nodes corresponding to p1 and p2 through the JOIN query operator; (9) Based on the cardinality estimation results of all query subgraphs in the list AllSubgraphs, a bottom-up search strategy is adopted to generate a query tree T; each query subgraph p in AllSubgraphs is traversed, and all possible tree node generation methods corresponding to p are enumerated in the query tree; based on the cardinality estimation values ​​of the obtained tree nodes and their child nodes corresponding to the query subgraphs, the execution overhead of each generation method is evaluated, and the generation method with the lowest overhead is selected, and the tree nodes, child nodes, and query operator information corresponding to p under the generation method are added to the query tree T; the query tree T obtained with the query graph Q as the root node is the optimal subgraph matching query plan; (10) Different query statement generators are configured for different database systems. The query statement generator can convert the EXPAND-OUT, EXPAND-INTO and JOIN query operators in the query tree into specific database query statements, and use the query plan prompt function provided by the database system to specify the execution order of each subquery in the query statement, which vertex in the graph needs to be binary connected, and the specific connection method used when performing the binary connection, so that the execution plan generated by the query statement conforms to the subgraph matching order in the query tree to the greatest extent; (11) Execute the database query statement generated in step (10) in the backend of the corresponding database system, and return the query results generated by the database to the user, thereby realizing efficient subgraph matching query.

2. According to claim 1, an efficient subgraph matching query method across database systems is characterized in that: The cardinality constraint of each edge label in step (2) refers to the mapping multiplicity relationship allowed by the edge label between the two vertices connected by the edge, including the cardinality constraint of the source vertex and the cardinality constraint of the target vertex. The cardinality constraint is one of the two values ​​of "many" or "one".

3. According to the efficient subgraph matching query method across database systems of claim 1, it is characterized in that: Step (2) extracts each edge label The cardinality constraint implementation process is as follows: For each edge label Label e , store all edges with this label in a two-dimensional table T with two attributes src and dst. Each edge corresponds to a record in the table. The numbers of the source vertex and the target vertex of the edge are the src and dst attributes of the record respectively; group all records according to the dst attribute in the two-dimensional table T, and calculate the number of records C in each group, and calculate the maximum number of records C in all groups. If the maximum value is greater than 1, the edge label Label e The source vertex cardinality constraint is "many", otherwise it is "one"; group all records according to the src attribute in the two-dimensional table T, and calculate the number of records C in each group, and calculate the maximum value of the number of records C in all groups. If the maximum value is greater than 1, the edge label Label e The target vertex cardinality constraint is "many" if , otherwise it is "one".

4. According to claim 1, an efficient subgraph matching query method across database systems is characterized in that: The label of the injective vertex in step (2) is one of the cardinality constraint types of the edge label, which is many-to-one or one-to-many, and the label of the vertex whose cardinality constraint of the source vertex or the target vertex in the edge label is "many" is an injective vertex label, indicating that the vertex with this label can be connected to at most one edge with this label.

5. According to claim 1, an efficient subgraph matching query method across database systems is characterized in that: The implementation process of step (3) is as follows: Each edge set is randomly sampled edges, where α is the sampling ratio, Indicates that the label is The frequency of the edge in the attribute graph dataset G, M is the total number of edges in the attribute graph dataset, and l is the total number of different edge labels; calculate the sparsification rate of edge labels:

6. According to claim 1, an efficient subgraph matching query method across database systems is characterized in that: The implementation process of the matching result cardinality estimation method based on topological structure in step (7) is as follows: For the query subgraph p, if the key of the query subgraph is stored in HigherOrderStatisticMap, the cardinality estimation value of the query subgraph is directly obtained from HigherOrderStatisticMap; if there is a pattern subgraph p' that has only one less edge and one less vertex than p, and neither the edge nor the vertex contains the attribute filtering condition, and the tuple consisting of the label of the edge and the label of another vertex on the edge exists in the ManyToOneExpansionSet generated in step (2), then it means that the matching result of p can be obtained by expanding the matching result of p' one by one, and the cardinality estimation result of p is the cardinality estimation result of p' already stored in HigherOrderStatisticMap; otherwise, any existing subgraph number estimation method is used for estimation, and all two subgraphs p1 and p2 that can form p are enumerated, and the matching cardinality of p is estimated to be Among them, F(x) is the cardinality estimation result of subgraph x.

7. According to claim 1, an efficient subgraph matching query method across database systems is characterized in that: The matching cardinality estimation method based on sparse graph query and formula estimation in step (7) is: If the number of vertices contained in the query subgraph p is less than or equal to 3, then calculate the number of matching results n of p in the sparse graph dataset SampleG. If n is not 0, the estimated cardinality is where k is the number of edges in p, and ρ i Refers to the sparsification ratio of the edge label of the i-th edge involved in p. If the value of n is 0, that is, the pattern graph cannot be found in the sparse graph dataset, the estimated cardinality is 8. According to claim 1, an efficient subgraph matching query method across database systems is characterized in that: The implementation process of evaluating the execution overhead of each generation method in step (9) is as follows: According to different target database systems, corresponding execution cost calculation formulas are defined for each query operator. If query subgraph p can be generated by query subgraph p' through EXPAND-OUT query operator, the execution cost of this generation method is Cost(Plan'(p'))+ExpandCost(p'->p)+F(p), where Cost(Plan'(p')) refers to the execution cost of the query subtree corresponding to query graph p', and ExpandCost(p'->p) refers to the vertex expansion execution cost. Different parameters; if the query subgraph p can be generated by the query subgraph p' through the EXPAND-INTO query operator, the execution cost of this generation method is Cost(Plan'(p'))+ExpandCost(p'->p)+F(p)+ExpandIntoFactor×F(p'), where ExpandIntoFactor is the factor parameter of the EXPAND-INTO operation; if the query subgraph p can be generated by the query subgraphs p1 and p2 through the JOIN operation, the execution cost of this generation method is: Cost(Plan'(p1))+Cost(Plan'(p2))+F(p)+(F(p1)+F(p2))×IntersectionNum×JoinFactor, where JoinFactor is the execution cost parameter of the Join operation, and IntersectionNum is the number of different vertices intersected by p1 and p2.

9. The efficient subgraph matching query method across database systems according to claim 1, characterized in that: The query tree described in step (9) is a binary tree. For each node p in the query tree, if the execution overhead of generating p from the child node p' through the EXPAND-OUT or EXPAND-INTO query operator is the lowest, then the tree node p' is set to the left child of the node p, the right child of p is set to null, and the query operator of p is updated to EXPAND-OUT or EXPAND-INTO; if the execution overhead of generating p from the two child nodes p1 and p2 through the JOIN operation query operator is the lowest, then the left child and right child nodes of p are set to p1 and p2 respectively, and the query operator of p is updated to JOIN.

10. The efficient subgraph matching query method across database systems according to claim 1, characterized in that: The query plan hint function provided by the database system in step (10) specifies the execution order of each sub-query in the query statement as follows: For a node in the query tree and its corresponding query subgraph p, if p is a path structure, the query starting vertex and starting table are specified through the query plan hint function of the SQL or Cypher language; If p is a tree structure or contains a ring structure, the subquery mechanism provided by the SQL or Cypher language is used, or a query statement is split into multiple subquery statements to control the execution plan of the query statement.

Images