Batch query method and electronic device for graph database
By performing deep condition partitioning and batch processing on the query text of graph databases, the performance problem of graph databases when querying large amounts of data is solved, and the query efficiency and accuracy are improved.
Patent Information
- Application Number
- CN202411143543.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-08-20
AI Technical Summary
When the number of data records in a query range is too large, the existing graph database suffers from poor query performance, requiring multiple loop query operations, resulting in low efficiency.
By analyzing the query text, it is divided into multiple deep query conditions, the starting node or starting edge is determined, the target batch size is calculated based on the number of starting nodes or starting edges, and it is divided into multiple query batches. The second-level deep query conditions are used to filter the attribute graph stored in the graph database by edges or nodes, thereby improving query efficiency.
It enables flexible calculation of target batch size, improves query accuracy and efficiency, reduces the data processing pressure on the search engine, and makes full use of the underlying database's data processing capabilities.
Smart Images

Figure CN118656499B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of graph databases, and in particular to a batch query method and electronic device for graph databases. Background Art
[0002] A graph consists of two elements: nodes and edges. Each node represents an entity, and each edge represents the relationship between two connected nodes.
[0003] Current graph databases generally obtain data records from the underlying database one by one into the memory and complete the query by judging the data records in the memory. However, if the number of data records in the query range is too large, a single query may not be able to find data records that meet the conditions. In this case, the underlying database needs to be cyclically queried many times, resulting in poor query performance. Summary of the invention
[0004] In order to solve the above technical problems, the present application at least provides a batch query method and electronic device for a graph database.
[0005] In a first aspect, the present application provides a batch query method for a graph database, which is applied to a search engine. The graph database is used to store a property graph, which is composed of nodes and edges. The properties of the nodes and edges are indexed, and the indexes are stored in the search engine. The method includes: performing sentence analysis on a query text, dividing the query text into at least a first-level deep query condition and a second-level deep query condition; wherein the depth of the nodes or edges within the query range corresponding to the first-level deep query condition in the property graph is less than the depth of the nodes or edges within the query range corresponding to the second-level deep query condition in the property graph; determining the nodes or edges corresponding to the indexes matching the first-level deep query condition, and obtaining the starting node or starting edge; determining a batch quantity calculation strategy based on the number of starting nodes or starting edges, and calculating a target batch quantity using the batch quantity calculation strategy; dividing the starting node or starting edge into multiple query batches according to the target batch quantity; wherein the query batch contains a target batch number of starting nodes or starting edges; using the second-level deep query condition, edge filtering is performed on the edges corresponding to the starting nodes contained in the current query batch in the property graph stored in the graph database, or node filtering is performed on the nodes corresponding to the starting edges contained in the current query batch in the property graph stored in the graph database, to obtain a query result.
[0006] In one embodiment, a batch quantity calculation strategy is determined based on the size of the number of starting nodes or starting edges, and a target batch quantity is calculated using the batch quantity calculation strategy, including: detecting a quantity threshold range within which the number of starting nodes or starting edges is located, and querying a batch quantity calculation strategy that matches the quantity threshold range; and calculating the target batch quantity using the matching batch quantity calculation strategy.
[0007] In one embodiment, the quantity threshold range in which the number of starting nodes or starting edges is located is detected, and a batch quantity calculation strategy that matches the quantity threshold range is queried, including: obtaining a first quantity judgment threshold; if the number of starting nodes or starting edges is greater than the first quantity judgment threshold, the batch quantity calculation strategy that matches the quantity threshold range is queried as the first calculation strategy; the step of using the first calculation strategy to calculate the target batch quantity includes: obtaining a batch partitioning coefficient; and using the batch partitioning coefficient to divide the number of starting nodes or starting edges to obtain the target batch quantity.
[0008] In one embodiment, obtaining a batch partition coefficient includes: obtaining the total number of edges and the total number of nodes in the attribute graph; and calculating the ratio between the total number of edges and the total number of nodes to obtain the batch partition coefficient.
[0009] In one embodiment, the quantity threshold range of the number of starting nodes or starting edges is detected, and the batch quantity calculation strategy that matches the quantity threshold range is queried, including: obtaining a first quantity judgment threshold; if the number of starting nodes or starting edges is greater than the first quantity judgment threshold, the batch quantity calculation strategy that matches the quantity threshold range is queried as a second calculation strategy; the step of using the second calculation strategy to calculate the target batch quantity includes: taking the number of starting nodes or starting edges as the target batch quantity.
[0010] In one embodiment, edge filtering is performed on edges corresponding to starting nodes in a current query batch in a property graph stored in a graph database, or node filtering is performed on nodes corresponding to starting edges in a current query batch in a property graph stored in a graph database, to obtain query results, including: generating filtering conditions based on second-level deep query conditions; sending the filtering conditions and the current query batch to the graph database, so that the graph database performs edge filtering on edges corresponding to starting nodes in the current query batch in the property graph based on the filtering conditions, or node filtering on nodes corresponding to starting edges in the current query batch in the property graph, and returns the filtering results as query results to the search engine; receiving the query results returned by the graph database.
[0011] In one embodiment, a node corresponding to an index matching a first-level deep query condition is determined to obtain a starting node, and an edge filtering condition is generated based on a second-level deep query condition; the filtering condition and the current query batch are sent to a graph database so that the graph database performs edge filtering on edges corresponding to the starting nodes contained in the current query batch in the property graph based on the filtering condition, or performs node filtering on nodes corresponding to the starting edges contained in the current query batch in the property graph, and returns the filtering result as a query result to the search engine, including: sending the edge filtering condition and the current query batch to the graph database so that the graph database performs edge filtering on all edges connected to the starting nodes contained in the current query batch in the property graph based on the edge filtering condition, and returns the edge filtering result as a query result to the search engine.
[0012] In one embodiment, an edge filtering condition and a current query batch are sent to a graph database, so that the graph database performs edge filtering on all edges connected to the starting nodes contained in the current query batch in the property graph based on the edge filtering condition, and returns the edge filtering result as a query result to the search engine, including: obtaining a second quantity judgment threshold; if the number of starting nodes in the current query batch is greater than the second quantity judgment threshold, setting the edge query quantity threshold to a first preset value; if the number of starting nodes in the current query batch is not greater than the second quantity judgment threshold, setting the edge query quantity threshold to a second preset value; sending the edge query quantity threshold, the edge filtering condition, and the current query batch to the graph database, so that the graph database performs edge filtering on all edges connected to the starting nodes contained in the current query batch in the property graph based on the edge filtering condition, and limiting the number of edges returned by performing an edge filtering on the starting node based on the edge query quantity threshold, and returning the edge filtering result to the search engine.
[0013] In one embodiment, the edge corresponding to the index matching the first-level deep query condition is determined to obtain the starting edge, and a node filtering condition is generated based on the second-level deep query condition; the filtering condition and the current query batch are sent to the graph database so that the graph database performs edge filtering on the edges corresponding to the starting nodes contained in the current query batch in the property graph based on the filtering condition, or performs node filtering on the nodes corresponding to the starting edges contained in the current query batch in the property graph, and returns the filtering result as a query result to the search engine, including: sending the node filtering condition and the current query batch to the graph database so that the graph database performs node filtering on all nodes connected to the starting edges contained in the current query batch in the property graph based on the node filtering condition, and returns the node filtering result as a query result to the search engine.
[0014] The second aspect of the present application provides a batch query device for a graph database, wherein the graph database is used to store a property graph, the property graph is composed of nodes and edges, the properties of the nodes and edges are indexed, and the index is stored in a search engine, the device includes: a query text analysis module, used to perform sentence analysis on the query text, and divide the query text into at least a first-level deep query condition and a second-level deep query condition; wherein the depth of the nodes or edges within the query range corresponding to the first-level deep query condition in the property graph is less than the depth of the nodes or edges within the query range corresponding to the second-level deep query condition in the property graph; an index matching module, used to determine the nodes corresponding to the indexes that match the first-level deep query conditions; or edge, to obtain the starting node or the starting edge; a batch quantity calculation module, used to determine the batch quantity calculation strategy based on the size of the starting nodes or the starting edges, and use the batch quantity calculation strategy to calculate the target batch quantity; a batch division module, used to divide the starting node or the starting edge into multiple query batches according to the target batch quantity; wherein the query batch contains the target batch number of starting nodes or starting edges; a query module, used to use the second-level deep query condition to perform edge filtering on the edges corresponding to the starting nodes contained in the current query batch in the attribute graph stored in the graph database, or to perform node filtering on the nodes corresponding to the starting edges contained in the current query batch in the attribute graph stored in the graph database, to obtain the query result.
[0015] A third aspect of the present application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-mentioned batch query method of the graph database.
[0016] A fourth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, which implement the batch query method of the above-mentioned graph database when the program instructions are executed by a processor.
[0017] The above scheme divides the query text into at least the first-level deep query condition and the second-level deep query condition by performing sentence analysis on the query text; determines the node or edge corresponding to the index matching the first-level deep query condition to obtain the starting node or starting edge; determines the batch quantity calculation strategy based on the number of starting nodes or starting edges, and calculates the target batch quantity using the batch quantity calculation strategy; divides the starting node or starting edge into multiple query batches according to the target batch quantity; uses the second-level deep query condition to perform edge filtering on the edges corresponding to the starting nodes in the current query batch in the attribute graph stored in the graph database, or performs node filtering on the nodes corresponding to the starting edges in the current query batch in the attribute graph stored in the graph database to obtain the query results, and can flexibly calculate the target batch quantity according to the number of starting nodes or starting edges, thereby improving the accuracy of the target batch quantity and effectively improving the query efficiency.
[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.
[0020] Figure 1 is a schematic diagram of a property graph shown in an exemplary embodiment of the present application;
[0021] Figure 2 is a schematic diagram of a data structure of a property graph shown in an exemplary embodiment of the present application;
[0022] Figure 3 It is a schematic diagram of performing a graph database query in the prior art;
[0023] Figure 4 It is a schematic diagram of an implementation environment of a solution provided by an embodiment of the present application;
[0024] Figure 5 is a flow chart of a batch query method of a graph database shown in an exemplary embodiment of the present application;
[0025] Figure 6 is a schematic diagram of edge filtering in a graph database shown in an exemplary embodiment of the present application;
[0026] Figure 7 It is a schematic diagram comparing the processes of executing the same graph database query task between the prior art and the present application;
[0027] Figure 8 is a block diagram of a batch query device for a graph database shown in an exemplary embodiment of the present application;
[0028] Fig. 9 is a schematic structural diagram of an electronic device shown in an exemplary embodiment of the present application;
[0029] Fig.10 It is a schematic diagram of the structure of a computer-readable storage medium shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION
[0030] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.
[0031] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.
[0032] The term "and / or" in this article is only an association information describing the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the previous and next associated objects are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.
[0033] The efficiency of querying graph databases in the prior art is low, especially when the query conditions are ambiguous.
[0034] For example, see Figure 1 , Figure 1 A schematic diagram of a property graph is shown as an exemplary embodiment of the present application, such as Figure 1 As shown in the figure, nodes contain vehicles, events, etc., and relationships contain driving, stopping, traveling, etc. The data structure of the attribute graph in the graph database can be seen in Figure 2 , Figure 2 A schematic diagram of a data structure of a property graph is shown as an exemplary embodiment of the present application. Figure 2 As shown in the figure, a row of records in the attribute graph in the graph database contains all the properties (properties) of this vertex (vertex-id) and the edges (edges) connected to it. Taking the vehicle node as an example, it contains attributes such as color and model, as well as edges such as passing, stopping, and traveling together.
[0035] See also Figure 3 , Figure 3 FIG. 1 is a schematic diagram of performing a graph database query in the prior art according to an exemplary embodiment. Figure 3 As shown, if the query text is "query black vehicles passing through building A on June 1st", the following steps need to be executed repeatedly: Step 1, query vehicles with black color; Step 2, query events with the relationship of passing by; Step 3, query events with the time of June 1st and the location of building A.
[0036] For example, the first loop query is performed on the vehicle id1 with a black color, and the passing edge connected to the vehicle id1 is obtained by querying, and the passing id1 is queried, and the attributes of the event node corresponding to the passing id1 are further queried, and the time is February 1, and the location is Building B; the nth loop query obtains the vehicle id95 with a black color, and the passing id3651 is queried. The time of the event node corresponding to the passing id3651 is June 1, and the location is Building A. Therefore, the nth loop query can return the relevant attribute information of the vehicle id95.
[0037] It can be seen from this that if there are multiple vehicles with black color and there are multiple events of driving by, it is necessary to perform multiple loop queries, and the query effect is poor.
[0038] Based on this, the present application at least provides a batch query method and electronic device for a graph database.
[0039] The following is a description of the batch query method of the graph database provided in the embodiment of the present application.
[0040] Please refer to Figure 4 , which shows a schematic diagram of a solution implementation environment provided by an embodiment of the present application. The solution implementation environment may include a terminal 410, a search engine 420 and a graph database 430, and the terminal 410, the search engine 420 and the graph database 430 are connected to each other in communication.
[0041] The number of the terminal 410 may be one or more. The terminal 410 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto.
[0042] The search engine 420 is used to perform data search. The types of the search engine 420 include but are not limited to Elasticsearch, Manticore Search, etc., which are not limited in this application.
[0043] The graph database 430 is used to store property graphs. The types of the graph database 430 include but are not limited to JanusGraph, Neo4j, OrientDB, etc., which are not limited in this application.
[0044] In one example, terminal 410 generates a query text and transmits the query text to search engine 420. Search engine 420 performs sentence analysis on the query text, divides the query text into at least first-level deep query conditions and second-level deep query conditions, determines the nodes or edges corresponding to the indexes matching the first-level deep query conditions, obtains the starting node or starting edge, determines a batch quantity calculation strategy based on the number of starting nodes or starting edges, calculates a target batch quantity using the batch quantity calculation strategy, divides the starting node or starting edge into multiple query batches according to the target batch quantity, and then uses the second-level deep query conditions to perform edge filtering on the edges corresponding to the starting nodes in the current query batch in the property graph stored in graph database 430, or performs node filtering on the nodes corresponding to the starting edges in the current query batch in the property graph stored in graph database 430, to obtain query results. Search engine 420 returns the query results to terminal 410.
[0045] It should be noted that the search engine 420 and the graph database 430 may be integrated in the same device or separately arranged in different devices, and this application does not limit this.
[0046] See also Figure 5 , Figure 5 is a flowchart of a batch query method for a graph database shown in an exemplary embodiment of the present application. The batch query method for a graph database can be applied to Figure 4 The implementation environment shown is specifically executed by the search engine in the implementation environment.
[0047] like Figure 5 As shown, the batch query method of the graph database includes at least steps S510 to S550, which are described in detail as follows:
[0048] Step S510: Perform sentence analysis on the query text and divide the query text into at least a first-level deep query condition and a second-level deep query condition; wherein the depth of the nodes or edges within the query range corresponding to the first-level deep query condition in the property graph is less than the depth of the nodes or edges within the query range corresponding to the second-level deep query condition in the property graph.
[0049] Of course, the query text can also be divided into first-level deep query conditions, second-level deep query conditions, third-level deep query conditions, nth-level deep query conditions, etc.
[0050] The query text is analyzed to obtain multiple query conditions of different depth levels, and the query conditions of different depth levels are called step by step to realize the query of the graph database.
[0051] For example, if the query text is "Search for black vehicles that passed through building A on June 1st", the query text can be divided into the following by sentence analysis:
[0052] First-level deep query condition: query vehicles with black color;
[0053] Second-level deep query condition: the query relationship is the event of driving past;
[0054] The third-level deep query condition: query events whose time is June 1 and whose location is Building A.
[0055] Step S520: Determine the node or edge corresponding to the index matching the first-level deep query condition, and obtain the starting node or starting edge.
[0056] The graph database in this application is used to store attribute graphs, which are composed of nodes and edges. The attributes of nodes and edges are indexed, and the indexes are stored in the search engine. The indexes can be partial attribute values of nodes, partial attribute values of edges, labels of edges, etc.
[0057] For example, if the query text is "Query black vehicles that passed through building A on June 1", the first-level deep query condition is "Query vehicles that are black in color", and the nodes corresponding to the indexes that match the first-level deep query condition include vehicle id1, vehicle id2,..., vehicle idA, etc., and these A nodes are used as the starting nodes.
[0058] Optionally, the maximum number of results returned by a search engine query may be limited, such as limiting the maximum value of A to 1000. If the actual number of starting nodes or starting edges exceeds 1000, they are returned in multiple batches.
[0059] Step S530: Determine a batch quantity calculation strategy based on the number of starting nodes or starting edges, and use the batch quantity calculation strategy to calculate the target batch quantity.
[0060] According to the number of starting nodes or starting edges, a batch quantity calculation strategy is determined to calculate the target batch quantity using the batch quantity calculation strategy.
[0061] Exemplarily, the corresponding batch quantity calculation strategy can be determined according to the quantity threshold range of the number of starting nodes or starting edges. Different quantity threshold ranges are matched with different preset batch quantity calculation strategies. The calculation formulas used by different batch quantity calculation strategies can be different. The specific selection can be flexibly made according to the actual application situation. This application does not limit this.
[0062] Exemplarily, the corresponding batch quantity calculation strategy can also be determined in combination with the quantity threshold range of the number of starting nodes or starting edges, the node type or edge type to which the starting node or starting edge belongs, the number of preset type nodes or the number of preset type edges contained in the starting node or starting edge, etc.
[0063] According to the batch quantity calculation strategy determined above, the target batch quantity is calculated.
[0064] This application improves the accuracy of the target batch size by flexibly calculating the target batch size according to the size of the starting nodes or the starting edges.
[0065] Step S540: Divide the starting nodes or starting edges into multiple query batches according to the target batch quantity; wherein the query batch contains the target batch quantity of starting nodes or starting edges.
[0066] The target batch size is the number of starting nodes or starting edges contained in the query batch obtained by limiting the partitioning.
[0067] Divide the starting node or starting edge into multiple query batches according to the target batch number.
[0068] For example, if the number of starting nodes is 1000 and the calculated target batch size is 200, five query batches are obtained, each containing 200 starting nodes.
[0069] Compared with using a fixed quantity threshold to divide query batches, this application uses a flexibly calculated target batch quantity to query batches. The resulting query batches are more adapted to the current query scenarios, which improves the accuracy of query batch division and improves the query performance of graph data.
[0070] Step S550: Using the second-level deep query conditions, edge filtering is performed on the edges corresponding to the starting nodes in the current query batch in the attribute graph stored in the graph database, or node filtering is performed on the nodes corresponding to the starting edges in the current query batch in the attribute graph stored in the graph database to obtain query results.
[0071] After obtaining multiple query batches, batch query is performed on each query batch in turn.
[0072] Specifically, according to the second-level deep query condition, edge filtering is performed on the edges corresponding to the starting nodes contained in the current query batch, or node filtering is performed on the nodes corresponding to the starting edges contained in the current query batch.
[0073] In some embodiments, the edge filtering step or the node filtering step may be performed in a search engine, such as the search engine batch-acquires the edges corresponding to the starting nodes contained in the current query batch from the graph database, performs edge filtering on the acquired edges according to the second-level deep query conditions, and uses the filtering results as the query results.
[0074] In some embodiments, the edge filtering step or the node filtering step can be performed in a graph database, that is, the second-level deep query condition is pushed to the graph database, so that the graph database batch obtains the edges corresponding to the starting nodes contained in the current query batch, and performs edge filtering on the obtained edges according to the second-level deep query condition, and returns the filtering results as query results to the search engine.
[0075] It should be noted that if there are third-level deep query conditions, fourth-level deep query conditions, etc., edge filtering or point filtering will continue to be performed in sequence until all deep query conditions are executed to obtain the final filtering result.
[0076] Next, some embodiments of the present application are described in detail.
[0077] In some implementations, determining a batch quantity calculation strategy based on the number of starting nodes or starting edges in step S530, and calculating a target batch quantity using the batch quantity calculation strategy, includes:
[0078] Step S531: Detect the quantity threshold range of the number of starting nodes or starting edges, and query the batch quantity calculation strategy that matches the quantity threshold range.
[0079] For example, if the number of starting nodes or starting edges is within the first threshold range, the corresponding batch quantity calculation strategy is: use the first preset algorithm to calculate the number of starting nodes or starting edges to obtain the target batch quantity; if the number of starting nodes or starting edges is within the second threshold range, the corresponding batch quantity calculation strategy is: use the second preset algorithm to calculate the number of starting nodes or starting edges to obtain the target batch quantity.
[0080] Step S532: Calculate the target batch quantity using the matching batch quantity calculation strategy.
[0081] Exemplarily, a first quantity judgment threshold is obtained; if the number of starting nodes or starting edges is greater than the first quantity judgment threshold, the batch quantity calculation strategy that matches the quantity threshold range is queried as the first calculation strategy; the steps of using the first calculation strategy to calculate the target batch quantity include: obtaining a batch partitioning coefficient; using the batch partitioning coefficient to divide the number of starting nodes or starting edges to obtain the target batch quantity.
[0082] Among them, the first quantity judgment threshold can be pre-set, such as 50, 100; the batch division coefficient can be pre-set or flexibly calculated, and this application does not limit this.
[0083] For example, the step of obtaining the batch partition coefficient includes: obtaining the total number of edges and the total number of nodes in the attribute graph; calculating the ratio between the total number of edges and the total number of nodes to obtain the batch partition coefficient.
[0084] If the ratio between the total number of edges and the total number of nodes is 5, the batch partitioning coefficient is set to 5. At this time, if the maximum number returned by a search engine query is 1000, a maximum of 5 batch queries can be performed on the graph database for the start nodes or start edges returned by a search engine query, and each batch deep query is 200 start nodes or start edges.
[0085] Furthermore, if the number of starting nodes or starting edges is greater than the first quantity judgment threshold, the batch quantity calculation strategy that matches the quantity threshold range is queried as the second calculation strategy; the step of calculating the target batch quantity using the second calculation strategy includes: taking the number of starting nodes or starting edges as the target batch quantity.
[0086] Taking the first quantity judgment threshold of 50 as an example, the method of determining the target batch quantity is illustrated as follows:
[0087] Example 1: The query text is "Query cities that can sell target type vehicles". According to the first-level deep query condition "Query target type vehicles", the number of starting nodes obtained is A=10, which is less than the first quantity judgment threshold, so the target batch quantity N=A=10.
[0088] Example 2: The query text is "Query the black vehicles that passed through building A on June 1st". According to the first-level deep query condition "Query the vehicles with black color", the number of starting nodes A=1000 is obtained, which is greater than the first quantity judgment threshold. The batch division coefficient y=5 is obtained, and the target batch number N=A / y=200.
[0089] The starting nodes or starting edges are divided into multiple query batches according to the target batch quantity, and the second-level deep query conditions are used to perform edge filtering on the edges corresponding to the starting nodes in the current query batch in the property graph stored in the graph database, or to perform node filtering on the nodes corresponding to the starting edges in the current query batch in the property graph stored in the graph database to obtain the query results.
[0090] In some embodiments, edge filtering is performed on edges corresponding to starting nodes in a current query batch in a property graph stored in a graph database, or node filtering is performed on nodes corresponding to starting edges in a current query batch in a property graph stored in a graph database, to obtain query results, including: generating filtering conditions based on second-level deep query conditions; sending the filtering conditions and the current query batch to the graph database, so that the graph database performs edge filtering on edges corresponding to starting nodes in the current query batch in the property graph based on the filtering conditions, or node filtering on nodes corresponding to starting edges in the current query batch in the property graph, and returns the filtering results as query results to the search engine; receiving the query results returned by the graph database.
[0091] This application can fully utilize the data processing capabilities of the underlying database and further improve query efficiency by pushing subsequent query conditions to the graph database.
[0092] Next, the specific steps of edge filtering and node filtering are explained with examples.
[0093] In some embodiments, a node corresponding to an index matching the first-level deep query condition is determined to obtain a starting node, and an edge filtering condition is generated based on the second-level deep query condition; the edge filtering condition and the current query batch are sent to a graph database, so that the graph database performs edge filtering on all edges connected to the starting node contained in the current query batch in the property graph based on the edge filtering condition, and returns the edge filtering result as a query result to the search engine.
[0094] For example, see Figure 6 , Figure 6 A schematic diagram of edge filtering of a graph database shown as an exemplary embodiment of the present application is shown in FIG. Figure 6 As shown, the current query batch contains the starting node id1, and the edges connected to the starting node id1 include edge id1, edge id2, ..., edge idM, a total of M edges. The M edges are filtered according to the edge filtering condition, and the edges that meet the edge filtering condition include edge id2 and edge id3.
[0095] The edge filtering conditions and the current query batch are sent to the graph database, such as sending the edge filtering conditions and the identification ID of the starting node in the current query batch to the graph database, so that the graph database performs edge filtering on each starting node in the current query batch according to the edge filtering conditions, obtains the edges filtered by each starting node, and returns these filtered edges to the search engine.
[0096] It should be noted that, in addition to the start node obtained by querying according to the first-level deep query condition, edge filtering can also be performed on nodes obtained by querying according to other levels of deep query conditions.
[0097] Exemplarily, the edge filtering condition and the current query batch are sent to the graph database, so that the graph database performs edge filtering on all edges connected to the starting nodes contained in the current query batch in the property graph based on the edge filtering condition, and returns the edge filtering result as the query result to the search engine, including: obtaining a second quantity judgment threshold; if the number of starting nodes in the current query batch is greater than the second quantity judgment threshold, setting the edge query quantity threshold to a first preset value; if the number of starting nodes in the current query batch is not greater than the second quantity judgment threshold, setting the edge query quantity threshold to a second preset value; sending the edge query quantity threshold, the edge filtering condition and the current query batch to the graph database, so that the graph database performs edge filtering on all edges connected to the starting nodes contained in the current query batch in the property graph based on the edge filtering condition, and limiting the number of edges returned by performing an edge filtering on the starting node based on the edge query quantity threshold, and returning the edge filtering result to the search engine.
[0098] The values of the second quantity determination threshold and the first quantity determination threshold may be the same or different, and the first preset value and the second preset value may be flexibly set according to actual conditions.
[0099] Taking the second quantity judgment threshold of 50 as an example, if the number of starting nodes in the current query batch is greater than 50, the edge query quantity threshold x is set to 5; if the number of starting nodes in the current query batch is not greater than 50, the edge query quantity threshold x is set to 1.
[0100] For example, if the query text is "query black vehicles passing through building A", the starting node is "vehicles with black color", the number of starting nodes in the current query batch is 200, and the node identification ids are (1, 2..., 200), which is greater than the second quantity judgment threshold, then the edge query quantity threshold x is 5. In addition, the edge filtering condition corresponding to the starting node is "passing by", and the edges representing the "passing by" relationship are all stored together continuously starting with L1. Then all "passing by" edges can be obtained according to the prefix L1. The filtering rules used by the graph database for edge filtering are:
[0101] Rowkey=1,columnPrefix=L1,limit=5
[0102] Rowkey=2, columnPrefix=L1,limit=5
[0103] …
[0104] Rowkey=200,columPrefix=L1,limit=5
[0105] Among them, Rowkey is used to limit the node identification id of the query node, columnPrefix is used to limit the edge filtering condition, and limit is used to limit the edge query quantity threshold.
[0106] By executing the above filtering rules, the graph database can obtain up to 1,000 "traveled through" edges at a time, that is, a loop query can return up to 1,000 edges to the search engine.
[0107] For another example, the query text is "Query cities where target type vehicles can be sold", the starting node is "Color is target type vehicle", the number of starting nodes in the current query batch is 2, and the node IDs are (1, 2), which are less than the second quantity judgment threshold. The edge query quantity threshold x is 1. In addition, the edge filtering condition corresponding to the starting node is "sale", and the edges representing the "sale" relationship are all stored together starting with L2. All "sale" edges can be obtained according to the prefix L2. The filtering rules used by the graph database for edge filtering are:
[0108] Rowkey=1,columnPrefix=L2,limit=1
[0109] Rowkey=2, columnPrefix=L2,limit=1
[0110] By executing the above filtering rules, the graph database can obtain at most 2 "sale" edges at a time, that is, a loop query can return at most 2 edges to the search engine.
[0111] It should be noted that the above is only an example of one loop query. If there are edges that have not been queried, multiple loop queries need to be performed until all edges are filtered.
[0112] In some embodiments, the edge corresponding to the index matching the first-level deep query condition is determined to obtain the starting edge, and a node filtering condition is generated based on the second-level deep query condition; the node filtering condition and the current query batch are sent to a graph database, so that the graph database performs node filtering on all nodes connected to the starting edge contained in the current query batch in the property graph based on the node filtering condition, and returns the node filtering result as a query result to the search engine.
[0113] The node filtering conditions and the current query batch are sent to the graph database, such as sending the node filtering conditions and the identification id of the starting edge in the current query batch to the graph database, so that the graph database performs node filtering on each starting edge in the current query batch according to the node filtering conditions, obtains the nodes obtained by filtering each starting edge, and returns these filtered nodes to the search engine.
[0114] It should be noted that, in addition to the start edge obtained by querying the first-level deep query condition, node filtering may also be performed on the edges obtained by querying the other levels of deep query conditions.
[0115] For example, if the query text is "query black vehicles passing through building A", the starting node is "vehicles with black color", and it is assumed that through the edge filtering method in the above embodiment, 1000 "passing by" edges are filtered, and the edge identification ids are (1, 2..., 1000), each "passing by" edge is connected to an event node, and the node filtering condition is to filter event nodes whose location is building A. These event nodes are filtered according to the node filtering condition, and the filtering rules used by the graph database for node filtering are:
[0116] Rowkey=1,Value=Building A
[0117] Rowkey=2,Value=Building A
[0118] …
[0119] Rowkey=1000,Value=Building A
[0120] Return the filtered nodes to the search engine.
[0121] See also Figure 7 , Figure 7 It is a schematic diagram comparing the processes of executing the same graph database query task between the prior art and the present application, such as Figure 7As shown, assuming that the query text is "query black vehicles passing through building A", the attribute graph is stored in the hbase of Janusgraph, hbase is used as the graph database, and the search engine is Elasticsearch. In the prior art, Elasticsearch needs to first obtain the nodes of all vehicles with black color, and for one of the nodes of a vehicle with black color, query all the edges connected to the node from the graph database. Elasticsearch filters out the "passing by" edge based on the obtained edge, and then query the graph database to obtain the detailed information of the event node connected to the "passing by" edge, determine whether the location attribute of the event node is "building A", and loop query all the "passing by" edges connected to the node to achieve deep query of the node, and then repeat the above steps to query other nodes. In the present application, Elasticsearch obtains the nodes of all vehicles with black color as the starting nodes, calculates the target batch size as 200 according to the number of starting nodes, and pushes the edge filtering condition (querying the "passing by" edge) to the graph database for a maximum of 200 nodes of black color at a time, and obtains a maximum of 1,000 "passing by" edges connected by a maximum of 200 nodes from the graph database. Then Elasticsearch pushes the node filtering condition (querying the nodes with the location attribute of "Building A") to the graph database, and the graph database obtains the event nodes that meet the node filtering condition. Elasticsearch receives the event nodes that meet the node filtering condition from the nodes connected by the "passing by" edges returned by the graph database.
[0122] It can be seen that compared with the existing technology, the present application can effectively improve the query efficiency when the query conditions are ambiguous by flexibly dividing the starting nodes into batches and pushing the filtering conditions to the graph database. Simple and precise query conditions will not cause performance degradation due to calculations pushed down to the graph database. The data processing capabilities of the database can be fully utilized to reduce the data processing pressure on the search engine.
[0123] Figure 8 is a block diagram of a batch query device of a graph database shown in an exemplary embodiment of the present application. The graph database is used to store a property graph. The property graph is composed of nodes and edges. The properties of the nodes and edges are indexed. The index is stored in a search engine, such as Figure 8 As shown, the exemplary graph database batch query device 800 includes:
[0124] The query text analysis module 810 is used to perform sentence analysis on the query text, and divide the query text into at least a first-level deep query condition and a second-level deep query condition; wherein the depth of the nodes or edges within the query range corresponding to the first-level deep query condition in the property graph is less than the depth of the nodes or edges within the query range corresponding to the second-level deep query condition in the property graph;
[0125] An index matching module 820 is used to determine a node or edge corresponding to an index matching the first-level deep query condition, and obtain a starting node or a starting edge;
[0126] A batch quantity calculation module 830 is used to determine a batch quantity calculation strategy based on the number of starting nodes or starting edges, and calculate a target batch quantity using the batch quantity calculation strategy;
[0127] A batch division module 840 is used to divide the starting nodes or starting edges into multiple query batches according to the target batch number; wherein the query batch contains the target batch number of starting nodes or starting edges;
[0128] The query module 850 is used to use the second-level deep query conditions to perform edge filtering on the edges corresponding to the starting nodes contained in the current query batch in the property graph stored in the graph database, or to perform node filtering on the nodes corresponding to the starting edges contained in the current query batch in the property graph stored in the graph database to obtain query results.
[0129] It should be noted that the batch query device for the graph database provided in the above embodiment and the batch query method for the graph database provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment and will not be repeated here. In actual applications, the batch query device for the graph database provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.
[0130] See also Fig. 9 , Fig. 9 It is a structural diagram of an embodiment of an electronic device of the present application. The electronic device 900 includes a memory 901 and a processor 902, and the processor 902 is used to execute program instructions stored in the memory 901 to implement the steps in any of the above-mentioned batch query method embodiments of the graph database. In a specific implementation scenario, the electronic device 900 may include but is not limited to: a microcomputer, a server, and in addition, the electronic device 900 may also include a laptop computer, a tablet computer and other mobile devices, which are not limited here.
[0131] Specifically, the processor 902 is used to control itself and the memory 901 to implement the steps in any of the above-mentioned batch query method embodiments of the graph database. The processor 902 can also be called a central processing unit (CPU). The processor 902 may be an integrated circuit chip with signal processing capabilities. The processor 902 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 902 can be implemented by an integrated circuit chip.
[0132] See also Fig.10 , Fig.10 The computer-readable storage medium 1000 stores program instructions 1010 that can be executed by a processor, and the program instructions 1010 are used to implement the steps in any of the above-mentioned batch query method embodiments of the graph database.
[0133] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.
[0134] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.
[0135] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
[0136] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code.
Claims
1. A batch query method for a graph database, characterized in that: Applied to a search engine, the graph database is used to store a property graph, the property graph is composed of nodes and edges, the properties of the nodes and edges are indexed, and the index is stored in the search engine. The method includes: Perform sentence analysis on the query text, and divide the query text into at least a first-level deep query condition and a second-level deep query condition; wherein the depth of the nodes or edges within the query range corresponding to the first-level deep query condition in the property graph is less than the depth of the nodes or edges within the query range corresponding to the second-level deep query condition in the property graph; Determine a node or edge corresponding to an index matching the first-level deep query condition, and obtain a starting node or a starting edge; Determine a batch quantity calculation strategy based on the number of the starting nodes or the starting edges, and calculate a target batch quantity using the batch quantity calculation strategy; Dividing the starting node or the starting edge into a plurality of query batches according to the target batch quantity; wherein the query batch contains the target batch quantity of starting nodes or starting edges; Generate an edge filtering condition or a node filtering condition based on the second-level deep query condition, and send the edge filtering condition or the node filtering condition and the current query batch to the graph database, so that the graph database performs edge filtering on all edges connected to the starting node contained in the current query batch in the property graph based on the edge filtering condition, or performs node filtering on all nodes connected to the starting edge contained in the current query batch in the property graph based on the node filtering condition; return the edge filtering result or the node filtering result as a query result to the search engine; Receive the query result returned by the graph database.
2. The method according to claim 1, characterized in that The step of determining a batch quantity calculation strategy based on the number of the starting nodes or the starting edges, and calculating a target batch quantity using the batch quantity calculation strategy, includes: Detecting a quantity threshold range of the number of the starting node or the starting edge, and searching for a batch quantity calculation strategy that matches the quantity threshold range; The target batch quantity is calculated using the matching batch quantity calculation strategy.
3. The method according to claim 2, characterized in that The detecting the quantity threshold range of the quantity of the starting node or the starting edge, and searching for a batch quantity calculation strategy matching the quantity threshold range, includes: Obtaining a first quantity determination threshold; If the number of the starting nodes or the starting edges is greater than the first number judgment threshold, querying a batch quantity calculation strategy that matches the quantity threshold range as the first calculation strategy; The step of calculating the target batch size using the first calculation strategy includes: Get the batch partition coefficient; The batch partitioning coefficient is used to divide the number of the starting nodes or the starting edges to obtain a target batch number.
4. The method according to claim 3, characterized in that The step of obtaining the batch partition coefficient comprises: Obtain the total number of edges and the total number of nodes in the property graph; The ratio between the total number of edges and the total number of nodes is calculated to obtain a batch partition coefficient.
5. The method according to claim 2, characterized in that: The detecting the quantity threshold range of the quantity of the starting node or the starting edge, and searching for a batch quantity calculation strategy matching the quantity threshold range, includes: Obtaining a first quantity determination threshold; If the number of the starting nodes or the starting edges is greater than the first number judgment threshold, querying a batch number calculation strategy that matches the number threshold range as a second calculation strategy; The step of calculating the target batch size using the second calculation strategy includes: The number of the starting nodes or the starting edges is used as the target batch number.
6. The method according to claim 1, characterized in that Sending the edge filtering condition and the current query batch to the graph database, so that the graph database performs edge filtering on all edges connected to the start nodes contained in the current query batch in the property graph based on the edge filtering condition, and returning the edge filtering result as the query result to the search engine, including: Obtaining a second quantity determination threshold; If the number of starting nodes in the current query batch is greater than the second number judgment threshold, setting the edge query number threshold to a first preset value; If the number of starting nodes in the current query batch is not greater than the second number judgment threshold, setting the edge query number threshold to a second preset value; The edge query quantity threshold, the edge filtering condition and the current query batch are sent to the graph database, so that the graph database performs edge filtering on all edges connected to the starting node contained in the current query batch in the property graph based on the edge filtering condition, and limits the number of edges returned by performing an edge filtering on the starting node based on the edge query quantity threshold, and returns the edge filtering result to the search engine.
7. An electronic device, characterized in that: The electronic device comprises a memory and a processor, and the processor is used to execute program instructions stored in the memory to implement the steps in the method according to any one of claims 1 to 6.
