Kleen closure regular path query optimization method based on recursive index tree

Abstract

The present invention discloses a Kleen closure regular path query optimization method based on a recursive index tree. The method comprises the following steps: step 1: performing key-value pair conversion on an RDF data graph consisting of (s, p, o) triples; step 2: converting character strings into encoded data; step 3: storing key-value pairs of the entire data graph; step 4: subdividing Kleen closures into single-predicate Kleen closures and expression Kleen closures; and step 5: traversing the entire key-value pair storage. By extracting the information required for regular path queries modified by Kleen closures into a recursive index tree or other form, regular path queries in two Kleen closure forms, namely, predicate and single-predicate Kleen closures and expression Kleen closures, are implemented. Compared with the prior art, the present invention significantly shortens the execution time of Kleen closure queries and is suitable for a wide range of applications.

Description

Technical Field

[0001] The present invention relates to the field of search technology, and in particular to an optimization method for regular path queries modified by Kleen closures. Background Art

[0002] The Resource Description Framework (RDF) is a data model developed by the World Wide Web Consortium (W3C) to represent linked data on the Web. There are three types of data in an RDF graph: IRIs, empty nodes, and literals. The type constraints for each part of the SPO include: (1) The subject can be an IRI or an empty node. (2) The predicate is an IRI. (3) The object can be any of the three types. IRI is a generalization and promotion of URI or URL, which uniquely defines an entity / resource in the entire network or graph, similar to an ID card number; literals are literals, which are plain text with data types, and empty nodes are resources without IRIs and literals, or anonymous resources. The tuple (s, p, o) is called an RDF triple, where: s is the subject, p is the predicate (also called a property), and o is the object. An RDF data graph G(V,E,L,Ψ) is a directed, labeled multigraph, where V is a set of data nodes representing entities and objects, G(V,E,L,Ψ) is a set of directed edges, E is the total number of triples in G, L is a set of predicates, and Ψ is a labeling function with Ψ:E→L. Let Q = (x,r,y) be a regular path query on the RDF graph G(V,E,L,Ψ), where x,y∈V are variables and r is a regular expression over the alphabet L. The regular expression r is recursively defined as r:= |ε||p||r / r||r * |, where p∈L, where / represents the connection of paths, / represents or, * represents Kleen closure, and / represents or.

[0003] The evaluation result of the regular path query Q on G is given a graph G and a regular path query Q, RG = {(v i ,v j )|(v i ,v j )}The two-node path satisfies the regular expression r in Q. Summary of the Invention

[0004] In order to overcome the technical problems existing in the existing technology, the present invention proposes a Kleen closure regular path query optimization method based on a recursive index tree, focusing on regular path queries modified by Kleen closure. Through data preprocessing, the results required by Kleen closure are stored in the form of a recursive index tree. When executing the Kleen closure query, the answer branch can be directly obtained from the tree.

[0005] The present invention is achieved by utilizing the following technical solutions:

[0006] A Kleen closure regular path query optimization method based on a recursive index tree, the method comprising the following steps:

[0007] Step 1: Perform key-value pair conversion on the RDF data graph consisting of (s, p, o) triples, using the predicate p as the key and the subject s and object o pair as the value;

[0008] Step 2: Convert the string into encoded data;

[0009] Step 3: Perform key-value pair storage of the entire data graph, and then divide the key-value pair storage of the entire data graph into multiple data clusters, where each data cluster is used to store subject and object pairs with the same predicate;

[0010] Step 4: Subdivide the query modified by Kleen closure into single predicate Kleen closure and expression Kleen closure;

[0011] Step 5. First, traverse the entire key-value pair storage and pre-process the Kleen closure of each data cluster, that is: traverse the entire key-value pair storage and pre-process the Kleen closure of each data cluster, the most important of which is the single-predicate Kleen closure, and generate a recursive index tree for it to facilitate subsequent query execution; for the expression Kleen closure, obtain its maximum recursive step number, and convert the infinite matching operation of the Kleen closure into bounded recursion, thereby reducing the complexity of the Kleen closure query; secondly, implement regular path queries in the two Kleen closure forms of predicate and single-predicate Kleen closure and expression Kleen closure, distinguish the answers generated by the predicate and the answers generated by the Kleen closure to perform join operations, split the results by planning the result table space, and mark the answers generated in the same generation to distinguish them.

[0012] Compared with the prior art, the present invention can achieve the following beneficial technical effects:

[0013] By extracting the information required for regular path queries modified by Kleen closures into forms such as recursive index trees, this method can be used to optimize this type of query, greatly shortening the execution time of Kleen closure queries and making them suitable for wide applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is an overall flow chart of the Kleen closure regular path query optimization method based on recursive index tree of the present invention;

[0015] Figure 2 This is a block diagram of an embodiment of a Kleen closure regular path query optimization method based on a recursive index tree according to the present invention;

[0016] Figure 3 This is an example diagram of an RDF data graph according to an embodiment of the present invention;

[0017] Figure 4 A schematic diagram of storing an RDF data graph according to an embodiment of the present invention;

[0018] Figure 5 This is an example diagram of the subject and object of the predicate know in an embodiment of the present invention;

[0019] Figure 6 A schematic diagram of a recursive answer tree constructed by predicates according to an embodiment of the present invention;

[0020] Figure 7 Schematic diagram of a recursive tree branch according to an embodiment of the present invention;

[0021] Figure 8 A schematic diagram of Kleen closure and non-Kleen closure query execution according to an embodiment of the present invention;

[0022] Figure 9 The following is a schematic diagram of some specific query executions according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] like Figure 1 FIG. 1 is a flow chart of the overall Kleen closure regular path query optimization method based on a recursive index tree of the present invention, which specifically includes the following steps:

[0025] Step 1: Perform key-value pair conversion on the resource description framework (RDF data graph) consisting of (s, p, o) triples, where s is the subject, p is the predicate (also called attribute), and o is the object;

[0026] Step 2: Convert the string into coded data (number). The entire query is to process the coded data.

[0027] Step 3: Use the predicate in the (s, p, o) triple as the key and the subject and object pair as the value to divide the entire data graph into multiple data clusters, each of which stores the subject and object pairs with the same predicate.

[0028] Step 4: Subdivide the Kleen closure into single-predicate Kleen closure (a)* and expression Kleen closure (a / b / c)* according to the different components modified by the Kleen closure;

[0029] Step 5: Traverse the entire key-value pair store to implement regular path queries in the form of predicates and two Kleen closures. Distinguish the answers generated by the predicates from those generated by the Kleen closures and perform join operations. The results are mainly divided by planning the result table space, and different results with different iteration times are processed differently.

[0030] Kleen closure preprocessing is performed on each data cluster. The specific process is as follows:

[0031] Traverse the entire key-value pair store and perform Kleen closure preprocessing on each data cluster, the most important of which is single-predicate Kleen closure, to generate a recursive index tree for it to facilitate subsequent query execution;

[0032] For the Kleen closure of an expression, the maximum number of recursive steps is obtained, and the infinite matching operation of the Kleen closure is transformed into a bounded recursion, thereby reducing the complexity of the Kleen closure query.

[0033] The present invention addresses regular path queries containing predicates and two forms of Kleen closure. Therefore, the result join operation requires distinguishing between answers generated by ordinary predicates and answers generated by Kleen closure forms. This invention primarily partitions the results by planning the result table space, processing results with different iteration times differently. Therefore, the present invention handles three different forms of regular path queries and solves the problem of joining results for mixed-form queries.

[0034] Among them, Kleen closure modification is divided into two forms, specifically:

[0035] The Kleene closure modifier is *. The predicate or expression modified by it is transformed into an infinite loop matching process. Normally, an unmodified predicate or expression will end after executing a query, but after being modified, it is necessary to continue to use the predicate or expression to repeatedly query the data graph to obtain the answer until no new answer is generated. Therefore, the regular path query modified by the Kleene closure is an infinite matching process.

[0036] Among them, step 5 also includes preprocessing operations on the Kleen closure query. The preprocessing operation is an operation of analyzing the data before the query, which specifically includes:

[0037] For single-predicate Kleen closure, the present invention chooses to traverse the entire data graph and perform infinite recursive query on each data cluster, thereby obtaining the self-recursive query answer of each predicate and storing it in the form of a tree;

[0038] For the Kleen closure of the expression, all predicates are selected for permutation and combination, and then these generated queries are traversed in the graph during the data processing phase, and the number of possible traversals is recorded. Thus, when the query is actually executed, the number of recursions is directly obtained, and the infinite matching process is converted into bounded recursion.

[0039] RDF data graphs include social relationship RDF data graphs, bioinformatics RDF data graphs, or transportation network RDF data graphs. Social relationship RDF data graphs can include RDF data graphs containing person relationships, RDF data graphs containing school relationships, and so on. This invention is applicable to the construction of knowledge graphs with complex relationships, such as university knowledge graphs, social knowledge graphs, and transportation knowledge graphs.

[0040] In conjunction with the embodiments, the present invention is specifically described as follows:

[0041] like Figure 2 The figure shows a block diagram of an embodiment of the Kleen closure regular path query optimization method based on a recursive index tree of the present invention. A regular path query is denoted as RPQ = (x, r, y), where the regular expression r = (a / b / (c / d)* / g), where a, b, c, d, g represent paths, / represents the connection of the paths, and * represents that (c / d) is modified by the Kleen closure. The processing of RPQ is the processing of r. After the query instruction is input, the query instruction is parsed and the query is executed in sequence until the query result is output. After the relevant RDF data is input, the query optimization is achieved through the following implementation process:

[0042] (1) Construct a database, pre-process the RDF data graph, obtain a key-value pair data graph in the form of Kleen closure feature data, and store it in the database:

[0043] First, the collected RDF data graph is preprocessed and stored in the form of key-value pairs as the query object. The RDF data graph can be a social relationship RDF data graph, a university relationship RDF data graph, a transportation network RDF data graph, etc.

[0044] The construction of the database in this embodiment specifically includes the following processes:

[0045] (1) Hash-encode the RDF data graph to obtain a hash value data graph; use C++ to write a hash value encoding method and a corresponding hash value decoding method to ensure data reliability. There is no restriction on the hash value encoding method and the hash value decoding method, and commonly used hash value encoding and decoding methods can be used;

[0046] (2) Using the predicate of the hash value data graph as the key and the subject and object pair as the value, the hash value data graph is stored using a key-value pair distributed storage method to obtain a key-value pair data graph.

[0047] like Figure 3 As shown in FIG, it is an example diagram of RDF data graph in the embodiment. Figure 4Figure 2 shows a schematic diagram of RDF data graph storage according to an embodiment of the present invention. After processing, the RDF data graph is stored as key-value pairs, i.e., a key-value data graph. The predicates serving as keys carry their own labels, while the subject-object pairs serving as values ​​are clustered. Key-value storage utilizes the triple data characteristics of hash value data graphs. Multiple subjects and objects are always present for each predicate. By segmenting the entire RDF data graph with the predicate as the focus, processing can be prevented from being interrupted by triples of unrelated predicates.

[0048] (3) The recursive index tree is constructed before the query is run. Specifically, the predicates that could potentially form a Kleen closure query are pre-processed with infinite recursive queries. The resulting answers are stored as a tree. When the query is executed, the tree branches are directly retrieved based on the current result, eliminating the need to execute infinite queries to obtain the result. The recursive index tree is primarily used to optimize single-predicate Kleen closure queries, namely KSone. This invention also provides optimized forms for KSfm. The following describes the process of generating the recursive index tree.

[0049] For KSone, when executing this query, the normal operation is to perform an infinite number of recursive queries on the predicate, and each answer is used as the correct answer to the Kleen closure query for the predicate until no answer is generated. Considering that this step is actually an infinite recursive process, and for the entire data graph, the path of the predicate extending infinitely at a certain node already exists and is unaffected by other factors, it is theoretically possible to perform recursive queries on the predicate in advance and construct the existing Kleen closure path. A predicate with Kleen closure refers to: repeatedly recursively querying a predicate or a predicate expression until no results are found, where all results found are the Kleen closure answers of the predicate. This predicate is a Kleen closure predicate, usually marked with '*', such as a* indicating that predicate a is a Kleen closure predicate. A recursive query refers to the result generated by a predicate or predicate expression, and then continuing to query with the same predicate or predicate expression.

[0050] by Figure 3 Taking the RDF data graph shown in the figure as an example, the present invention divides triples into multiple triple clusters based on predicates, takes the triple cluster corresponding to each predicate as an object, and extracts triple data graphs that may have Kleene closures from them. Taking the know predicate as an example, Figure 5Figure 2 shows the subject and object of the predicate "knows" in an embodiment of the present invention. If "knows" is modified by a Kleen closure * in a query, this means that the predicate must be queried recursively infinitely until no answer is found. Therefore, the present invention improves query speed by pre-generating a complete recursive answer tree. First, define the starting point and end point of the recursive answer tree. Each node of the tree is composed of the subject or object of the predicate. The starting point indicates that the entity does not appear in the object, and the end point indicates that the entity does not appear in the subject. Traverse each triple in the triple cluster, set the current triple to T_c, its subject to S_c, and its object to O_c, and search for triples with S_c as the object or O_c as the subject from the current triple; if the triple described in the former appears, and no triple with the object of the triple as the subject can be found in the triple sequence, it means that T_c is the end point of a recursive path; if the triple described in the latter appears, and no triple with the subject of the triple as the object can be found in the triple sequence, it means that T_c is the starting point of a recursive path; if both situations occur at the same time, it means that T_c is an intermediate member of the recursive path. Figure 6 As shown in FIG, it is a schematic diagram of a recursive answer tree composed of predicates according to an embodiment of the present invention. As can be seen from the figure, the starting points are kim, ada, and jan, and the end points are liz, sue, and sam; the recursive index tree is completed by constructing branches of the recursive index tree. First, the starting point column is traversed, and the entities therein are subjected to infinite recursive queries using the knows predicate until there are no results. Then, a branch of the recursive index tree is generated, and the end point of the branch is matched with the entity in the end point table. If the match is the same, it is deleted from the end point table. Figure 7 , which is a schematic diagram of a recursive tree branch according to an embodiment of the present invention, including two branches starting from ada and jan.

[0051] For KSfm, the possible forms are the number of combinations of all predicates. Without knowing the preprocessing stage of the specific query, it is impossible to obtain a specific answer recursive index tree like with KSone. During the query process, the expression in the form of Kleen closure will be expanded infinitely and the query executed. The present invention obtains the number of expansions of KSfm through preprocessing, thus turning an infinite recursive query process into a finite number of query processes. However, due to the excessive number of combinations mentioned above, the present invention only performs preprocessing operations on expression queries with fewer than or equal to 3 predicates.

[0052] The present invention first obtains all expressions with a predicate count of 3 or less and sequentially searches the entire dataset to determine the maximum number of recursions. The expression is defined as a path, and each path is run through a path-finding algorithm to find its step count. If the step count is zero, it means that the path cannot have a Kleen closure, thereby shortening the time for resultless queries. The step count also tells the query processor the maximum number of recursions for the current path and the specific position of the current triple when generating the result.

[0053] (2) When a query statement RPQ (including ordinary predicates and regular path queries in two Kleen closure forms) is received, the following search process is performed:

[0054] (1) Decompose the query statement RPQ:

[0055] A complete query consists of multiple common path queries and multiple Kleen closure expression queries. Kleen closure expressions serve as the decomposition boundaries for the query, splitting the large query into multiple subqueries. This decomposition is performed using Kleen closure expressions, which are brackets with superscript * in the query, as delimiters. Each subquery group contains a path.

[0056] (2) The join operation of the query results needs to distinguish the answers generated by ordinary predicates from the answers generated by predicates modified by Kleen closures for the join operation. The answers generated by ordinary predicates are the answers obtained by performing path queries on predicates that are not modified by Kleen closures.

[0057] like Figure 8 The following is a schematic diagram of Kleen closure and non-Kleen closure query execution in an embodiment of the present invention. Answers generated by Kleen closure should not be involved in the left join process. The results are primarily partitioned by planning the result tablespace, and different methods are applied to results with different iteration times. This primarily handles three different types of regular path queries, resolving the problem of joining results from mixed query types.

[0058] When a single predicate is modified by a Kleen closure, and both the left and right predicates are ordinary predicates: Given a query of worksfor / knows* / worksfor, and assuming the query sequence is executed from left to right, all triples of the three predicates are shown. When the first predicate is executed, all triples of that predicate are obtained as answers. The next predicate is in Kleen closure form. First, the first join is performed to obtain the result in the initial state. In this case, the result is the answer to the second predicate that does not have a Kleen closure form. Next, a recursive index tree is introduced, and the location of the first-generation result in the recursive index tree is found. The answer in Kleen closure form is then partitioned into a new space. When joining with the third predicate, both the first-generation result and the result generated by the Kleen closure are included to construct the new result. However, if a first-generation result is eliminated during this join, the entity corresponding to the first predicate is also eliminated during the left join. However, if the answer generated by the Kleen closure is eliminated, it does not need to participate in the left join of the leftmost result table. At the same time, when processing the result generated by the Kleene closure predicate, mark the number of iterations of the result to obtain the complete path.

[0059] The triples in the original result column are placed into the recursive index tree for matching. If a matching triple is found, it means that the path from the current position to the end of the recursive index tree is a result of the single-predicate Kleen closure. The current position of the triple in the recursive index tree is recorded. As the recursive index tree reaches its end, each result must indicate how many times it has been looped through, making it easier to retrieve the specific triple path leading to the current answer. The size of the original result column, Recover, is also recorded for use in subsequent join operations.

[0060] For KSfm, this involves executing multiple queries on a single path and updating the results. Preprocessing yields the Path_Step value, and the number of steps in the current path is directly calculated from this set. If it is zero, the query yields no results. If it is non-zero, the maximum number of recursions for the path is obtained. Each time a path is completed and the result table is updated, the size of the current result table (Recover) is recorded. Recover is used to connect the current expression result table with other expression result tables.

[0061] When the expression is modified by Kleen closure and both the left and right predicates are ordinary predicates: given a query knows / (knows / worksfor)* / worksfor, such as Figure 9As shown, it is a schematic diagram of some specific query executions of an embodiment of the present invention; for the result table generated by the expression, each time the path is completed and the result table is updated, the range of the current result table and the number of recursions are recorded. The size of the result table is used for the connection operation between the current expression result table and other expression result tables, and the number of recursions is used to obtain the specific path of the current answer. For KSfm, its two most important columns of results are the start column and the end column. The former guarantees the update of the left connection, and the latter guarantees the update of the right connection. After the result column in the expression is updated, its left connection can only use the first generation of results, and the entire result table is updated to the right immediately after the update. After that, the right connection uses the results of all generations to connect to the right.

[0062] When KSfm and KSone are mixed in the same query: Given a query of knows* / (knows / worksfor)* / worksfor, the join operation for this form must also conform to the two processing conditions described above. That is, a left join can only be performed on the first generation of results, while a right join is performed on all generations of results. The operation diagram is shown in Figure 9.

[0063] (3) Connect all result tables:

[0064] In the actual query process, the result table connection operation is from left to right, merging two result tables each time, and using the adjacent point attributes of the two tables as the boundary between the left and right connections. First, the two columns of adjacent point results are updated, and the left connection operation is performed on the left and right sides respectively through the updated two columns of results. At this time, it is only necessary to loop the connection of the two columns of results until the end point. The results discarded when updating the adjacent points need to be added back and then connected. After the two result tables are updated, the two can be spliced ​​together to form a new result table. At this time, each row in the table represents the specific result on a path.

[0065] The present invention can perform regular path query on the data graph of the inclusion relationship, optimize the query modified by Kleen closure, and return the correct result. A specific application scenario of the search system will be provided below.

[0066] For example, consider a query statement represented by the regular expression r = (father / son* / wife). Suppose Zhang San's son is Li Si, Li Si's son is Wang Wu, Li Si's wife is Zhao Liu, Wang Wu's son is Qian Qi, and Wang Wu's wife is Zhou Ba. Before executing the query, a recursive index tree is generated by traversing the dataset: Zhang San -> Li Si -> Wang Wu -> Qian Qi, where -> represents the predicate "son." When the query is executed, it is first determined that Zhang San is "father," satisfying the first predicate in the query path. The next predicate is "son." Normally, Zhang San's son, Li Si, should be found. However, "son" is modified by the Kleene closure, so Zhang San's location can be directly found in the recursive index tree. All nodes starting from Zhang San and ending at the end of the tree are retrieved, forming the answer to the Kleene closure for "son." The answer is now: "Father": Zhang San, "Son": Li Si, Wang Wu, Qian Qi. Continuing the query, the next predicate is "wife." By extending the entities in each "son" answer, we obtain the answer: "wife": Zhao Liu, Zhou Ba. The answer obtained for "wife" indicates that Qian Qi has no wife, so we remove it from the answer. The final answer path is: (Zhang San -> Li Si -> Zhao Liu) and (Zhang San -> Li Si -> Wang Wu -> Zhou Ba), a total of two paths.

[0067] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the embodiments of the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A Kleen closure regular path query optimization method based on recursive index tree, characterized in that: The method comprises the following steps: Step 1: The RDF data graph composed of triples is converted into key-value pairs, and the predicate As a key, subject and object Pair as value; Step 2: Convert the string into encoded data; Step 3: Perform key-value pair storage of the entire data graph, and then divide the key-value pair storage of the entire data graph into multiple data clusters, where each data cluster is used to store subject and object pairs with the same predicate; Step 4: Subdivide the query modified by Kleen closure into single predicate Kleen closure and expression Kleen closure; Step 5. First, traverse the entire key-value pair store and perform Kleen closure preprocessing on each data cluster, that is, traverse the entire key-value pair store and perform Kleen closure preprocessing on each data cluster. The most important one is the single-predicate Kleen closure, and generate a recursive index tree for it to facilitate subsequent query execution; for the expression Kleen closure, obtain its maximum recursive step number, and convert the infinite matching operation of the Kleen closure into bounded recursion, thereby reducing the complexity of the Kleen closure query; secondly, implement regular path query of the two Kleen closure forms of predicate and single-predicate Kleen closure and expression Kleen closure, distinguish the answers generated by the predicate and the answers generated by the Kleen closure to perform join operations, split the results by planning the result table space, and mark the answers generated in the same generation to distinguish them; including the following three processing situations: i. The recursive index tree construction process for optimizing single-predicate Kleen closure queries is as follows: predicates that may form Kleen closure queries are infinitely recursively queried in advance, the resulting answers are solidified and stored in the form of a tree, and when the query is executed, the branches of the tree are directly obtained based on the current results, and the recursive index tree is used to optimize the single-predicate Kleen closure query; ii. During the query process of the expression Kleen closure, the expression in the form of Kleen closure is expanded infinitely and the query is executed. The number of expansions is obtained through preprocessing, turning an infinite recursive query process into a finite number of query processes; iii. When single-predicate Kleen closure and expression Kleen closure are mixed in the same query: given a query, the join operation of its results must also comply with the above two processing conditions; In the actual query process, the connection operation is performed from left to right, merging two result tables each time, and using the adjacent point attributes of the two tables as the boundary between the left and right connections. First, the two columns of adjacent point results are updated, and the left connection operations are performed on the left and right sides respectively through the updated two columns of results. At this time, it is only necessary to loop the connection of the two columns of results until the end point. The results discarded when updating the adjacent points need to be added back and then connected. After the two result tables are updated, the two are spliced ​​together to form a new result table.

2. The Kleen closure regular path query optimization method based on recursive index tree according to claim 1, characterized in that: The RDF data graph includes a social relationship RDF data graph, a biological information RDF data graph or a transportation network RDF data graph.

Images