Data query method and device for distributed database

Abstract

The invention provides a data query method and device for a distributed database, and the method comprises the steps: obtaining a target query statement for the distributed database, and generating a query syntax tree corresponding to the target query statement; determining a sub-tree corresponding to a target sub-query statement in the target query statement in the query syntax tree, performing one-stage optimization on the determined sub-tree based on an optimizer, and performing one-stage calculation on an optimization result based on a calculation engine to obtain a query result of the target sub-query statement; and adjusting the query syntax tree according to the query result of the target sub-query statement, executing two-stage optimization on the adjusted query syntax tree based on the optimizer, and executing two-stage calculation on the optimization result based on the calculation engine to obtain the query result of the target query statement.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for querying data in a distributed database. Background Art

[0002] In database query scenarios, subqueries are a very common functional logic that can be used to nest a query block into the query expression of another query block to meet complex data query requirements. Among them, some subqueries may have an association with the original query outside the nesting, resulting in a significant increase in the calculation amount of the overall query and reduced query efficiency. Therefore, optimizing subqueries has always been one of the difficulties in data query optimization.

[0003] In related technologies, the corresponding subqueries are usually de-associated through the optimizer, and then rewritten into more efficient associative operators and calculated based on the computing engine to improve data query efficiency. However, in the scenario of distributed databases, since the calculation process of the above-mentioned associative operators is relatively complex, each distributed node needs to consume more resources for calculation, resulting in the subquery scenario with large computing requirements, which will inevitably reduce the overall query performance of the distributed database. Summary of the invention

[0004] In view of this, the present specification provides a data query method and device for a distributed database to solve the deficiencies existing in the related art.

[0005] Specifically, this specification is implemented through the following technical solutions:

[0006] According to a first aspect of an embodiment of this specification, a data query method for a distributed database is provided, the method comprising:

[0007] Obtaining a target query statement for the distributed database, and generating a query syntax tree corresponding to the target query statement;

[0008] Determine a subtree corresponding to a target subquery statement in the target query statement in the query syntax tree, perform a one-stage optimization on the determined subtree based on an optimizer, and perform a one-stage calculation on the optimization result based on a calculation engine to obtain a query result of the target subquery statement;

[0009] The query syntax tree is adjusted according to the query result of the target subquery statement, and two-stage optimization is performed on the adjusted query syntax tree based on the optimizer, and two-stage calculation is performed on the optimization result based on the calculation engine to obtain the query result of the target query statement.

[0010] According to a second aspect of an embodiment of this specification, a data query device for a distributed database is provided, the device comprising:

[0011] A query syntax tree generating unit, configured to obtain a target query statement for the distributed database and generate a query syntax tree corresponding to the target query statement;

[0012] a first query result acquisition unit, configured to determine a subtree corresponding to a target subquery statement in the target query statement in the query syntax tree, and to perform a one-stage optimization on the determined subtree based on an optimizer, and to perform a one-stage calculation on the optimization result based on a calculation engine to obtain a query result of the target subquery statement;

[0013] The second query result acquisition unit is used to adjust the query syntax tree according to the query result of the target sub-query statement, and perform two-stage optimization on the adjusted query syntax tree based on the optimizer, and perform two-stage calculation on the optimization result based on the calculation engine to obtain the query result of the target query statement.

[0014] According to a third aspect of the embodiments of this specification, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0015] According to a fourth aspect of the embodiments of this specification, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method described in the first aspect are implemented.

[0016] In the technical solution provided in this specification, after determining the subtree where the target subquery statement is located by the query syntax tree generated according to the target query statement, the process of calculating the query result of the target query statement can be divided into two sub-stages through a multi-stage concept, that is, firstly, performing a first-stage optimization on the above subtree based on the optimizer, and performing a first-stage calculation on the optimization result based on the computing engine to obtain the query result of the target subquery statement, and adjusting the above query syntax tree according to the query result of the target subquery statement, so as to perform a second-stage optimization on the adjusted query syntax tree based on the optimizer, and perform a second-stage calculation on the optimization result based on the computing engine, and then obtain the query result of the above target query statement. Among them, the two stages have different execution scopes for the query syntax tree and less calculation amount, so the computing resource requirements for each distributed node are reduced, and the original query performance can be improved while improving the subquery performance.

[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification, and for ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0019] Figure 1 It is a schematic diagram of the architecture of a data query system of a distributed database shown in an exemplary embodiment of this specification;

[0020] Figure 2 It is a flowchart of a data query method of a distributed database shown in an exemplary embodiment of this specification;

[0021] Figure 3 is a schematic diagram of a distributed database data query method shown in an exemplary embodiment of this specification;

[0022] Figure 4a is a schematic diagram based on a query syntax tree shown in an exemplary embodiment of this specification;

[0023] Figure 4b is another schematic diagram based on a query syntax tree shown in an exemplary embodiment of this specification;

[0024] Figure 5 is a schematic structural diagram of an electronic device shown in an exemplary embodiment of this specification;

[0025] Figure 6 It is a structural diagram of a data query device for a distributed database shown in an exemplary embodiment of this specification. DETAILED DESCRIPTION

[0026] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Instead, they are merely examples of devices and methods consistent with some aspects of this specification.

[0027] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, the single step described in this specification may be decomposed into multiple steps for description in other embodiments; and the multiple steps described in this specification may also be combined into a single step for description in other embodiments. It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0028] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0029] Figure 1 is a schematic diagram of the architecture of a distributed database data query system shown in an exemplary embodiment of this specification. Figure 1 As shown, the system includes a client 11, a server 12 and a distributed database 13.

[0030] The client 11 is an electronic device that has the ability to access the server 12. During the operation of the system, the client can send the target query statement input by the user or preset by the device to the server to obtain the corresponding query result. The client can be a mobile phone, a tablet device, a laptop, a PDA (Personal Digital Assistants), a wearable device (such as smart glasses, smart watches, etc.), and one or more embodiments of this specification are not limited to this.

[0031] The server 12 is a network device deployed with a distributed database management system. During the operation of the system, the server can determine the target sub-query statement from the target query statement from the client, and perform a first-stage optimization and calculation on the query syntax tree of the determined target sub-query statement according to the distributed database management system, thereby determining the corresponding query result, and adjusting the query syntax tree according to the query result of the target sub-query statement, and performing a second-stage optimization and calculation on the adjusted query syntax tree to obtain the query result of the target query statement. Specifically, the calculation process of each stage can involve one or more different database nodes of the distributed database. In addition, the server 12 can be built by a physical server of an independent host, or a virtual server hosted by a host cluster, and this specification does not limit this.

[0032] The distributed database 13 is a plurality of distributed nodes respectively deployed with different distributed databases. During the operation of the system, each distributed node can respectively perform two data query operations and return the query results to the server for aggregation, so that the client receives the query results aggregated by the server. Among them, the client does not actually and does not need to perceive the execution interaction process between the server and the distributed database, thereby achieving a senseless experience in data query operations.

[0033] In the related art, there is usually only one interaction between the server and each distributed node in the distributed database, that is, the server rewrites the query statement into a computable complete query statement and sends it to each distributed node, and aggregates multiple query results after each node completes the calculation and returns the corresponding query result, so as to return the complete query result to the client. As mentioned above, the defect of this method is that when faced with a complete query statement with a high computational amount, each distributed node needs to consume a large amount of computing resources for calculation, resulting in a decrease in the efficiency of determining the final aggregated query result.

[0034] This specification proposes the following technical solutions to solve the above problems, and the details of this technical solution are as follows.

[0035] Combine the following Figure 2 The illustrated embodiments illustrate the technical solution of this specification. Figure 2 is a flow chart of a distributed database data query method shown in an exemplary embodiment of this specification, such as Figure 2 As shown, the method may include the following steps:

[0036] S201, obtaining a target query statement for the distributed database, and generating a query syntax tree corresponding to the target query statement.

[0037] For the distributed database system of the above-mentioned service end, after obtaining the target query statement for the distributed database, the target query statement can be parsed to generate a corresponding expression tree (Expression Tree) as the above-mentioned query syntax tree. Specifically, the process of querying the target query statement is essentially equivalent to the distributed database executing each node of the query syntax tree in sequence according to the corresponding execution plan. Wherein, the above-mentioned target query statement can be a query language based on, for example, structured query language (Structured Query Language, SQL) implementation, or other arbitrary types of database languages, and those skilled in the art do not limit this. In addition, the above-mentioned execution plan can be understood as the specific steps when the database executes the target query statement, such as scanning the data in the corresponding table by means of index / full table, or the implementation method of connection query and the order of connection, etc., and its generation process can be detailed in the optimizer execution process below, which will not be repeated in this application. In addition, considering that the principle and specific execution method of generating a query syntax tree according to the target query statement have been basically disclosed in the relevant technology, it will not be described in detail in this application.

[0038] Of course, the above target query statement may also include the target subquery statement below, and the above target subquery statement may be further divided into a correlated subquery (Correlated Subquery) and a non-correlated subquery (Non-correlated Subquery), wherein the latter (non-correlated subquery) means that the data of the inner and outer layers of the nesting are not related during the query process, and the two can be calculated independently, so the amount of calculation involved is relatively small. Specifically, the statement corresponding to the non-correlated subquery can be executed first, and then the query statement of the outer layer can be executed after the query result is obtained. Therefore, the target subquery statement described in the following text of this specification also defaults to a correlated subquery of "the nested query block contains some parameters required to be provided by the query statement of the outer layer". This type of subquery cannot obtain query results as quickly as a non-correlated subquery, which will have a certain degree of negative impact on the data query efficiency of the node where the distributed database is located.

[0039] S202, determining a subtree corresponding to a target subquery statement in the target query statement in the query syntax tree, and performing a one-stage optimization on the determined subtree based on an optimizer, and performing a one-stage calculation on the optimization result based on a calculation engine to obtain a query result of the target subquery statement.

[0040] In the above query syntax tree, the nodes corresponding to the target sub-query statement (i.e., the nested inner query statement) and the query statements other than the target sub-query statement in the target query statement (i.e., the nested outer query statement) can be displayed through a tree structure. At the same time, the above-mentioned first-stage optimization can be performed on the subtree corresponding to the target sub-query statement in the query syntax tree based on the optimizer, and the above-mentioned first-stage calculation can be performed on the optimization result based on the calculation engine, so as to obtain the query result for the target sub-query statement.

[0041] The technical solution in this specification can determine the above subtree according to the nodes in the target query statement.

[0042] In one embodiment, each node in the query syntax tree can be traversed, and when any node contains an operator of the target type corresponding to the target subquery statement, the subtree formed by any of the above nodes and its descendant nodes is used as the subtree corresponding to the target subquery statement in the query syntax tree. It should be noted in advance that the above target subquery statement can be divided into three categories according to the generated data: scalar (Scalar-valued) subquery, existence test (Existential Test) subquery and set comparison (Quantified Comparison) subquery. Specifically, the output result of the scalar subquery is a result table with only one row and one column, and the scalar value is its result. The scalar subquery can appear in any place containing a scalar (for example, in SQL query statements such as SELECT and WHERE); the output result of the existence test subquery is a Boolean value, which can specifically refer to a subquery statement containing the keyword EXISTS; the output result of the set comparison subquery is also a Boolean value, but specifically refers to a subquery statement containing the keywords IN, SOME, and ANY. The existence test subquery and the set comparison subquery can appear in any position in the target query statement where a Boolean value can be placed. Among them, the above target types can be scalar subqueries and IN subquery statements. The reason is that: for the query results of the above two subqueries, the amount of data is generally small. Furthermore, the scalar subquery ensures that only one line is output at the grammatical level. The writing of IN is generally used in scenarios where the amount of data in the query results is small. Therefore, the two can be grammatically reflected as filtering operations on the nested outer target query statements, which is equivalent to generating a filter operator in the nested outer layer of the query syntax tree corresponding to the target query statement. Among them, the above filter operator is smaller than the complex operators such as Cross / Semi / Anti-Join that generally appear in existence detection subqueries, SOME, and ANY subquery statements. In other words, the present application can replace the complex operators originally contained in the query syntax tree with simple operators such as filter operators, thereby improving the efficiency of subsequent data queries. In addition, filtering has another major advantage. After the above-mentioned filter operator performs the data scanning operation, if the corresponding storage supports filter push-down or partition pruning, push-down or pruning can further reduce the data scanning amount and I / O (Input / Output), indirectly reducing the performance threshold of the corresponding distributed node.

[0043] After the above subtree is determined, the present specification can further optimize and calculate it.

[0044] In one embodiment, the subtree determined from the query syntax tree can be converted into a corresponding first execution plan by an optimizer, and the first execution plan can be sent to multiple nodes of a distributed database by a computing engine, so that the multiple nodes execute the execution plan and generate the query results of the corresponding target sub-query statements, and after collecting the query results of the target sub-query statements returned by the multiple nodes, the query results of all target sub-query statements can be aggregated into the query results of the target sub-query statements. Wherein, the so-called optimizer can include a rule-based optimizer (Rule-Based Optimization, RBO) and a cost-based optimizer (Cost-Based Optimization, CBO), the RBO can determine the execution plan of the executable target query statement according to the preset rules hard-coded in the database, the CBO can convert the relational expression in the target query statement according to the preset optimization rules, so as to generate multiple optimized execution plans based on the original execution plan, and then the CBO can calculate the cost (i.e., COST) of various possible "execution plans" according to statistics and cost models, and select the execution plan with the lowest COST as the actual operation plan. The so-called computing engine is a logical program used to manage the distribution of execution tasks and the collection and aggregation of query results between the distributed database management server and multiple distributed databases.

[0045] Among them, for the optimization process: CBO relies on the statistical information of database objects, and the accuracy of the statistical information will affect CBO's optimal choice. Combined with the previous embodiment, it can be seen that because the optimization object of the optimizer of this specification is the target subquery statement rather than the target query statement as a whole including the target subquery statement, the execution plan generated by RBO will no longer contain complex operators (whether it is the first-stage optimization or the second-stage optimization described below), and has higher execution efficiency than the traditional overall execution plan generated for the target query statement. At the same time, CBO can avoid the situation in which CBO cannot perform accurate cost estimation based on the basic statistical information of the corresponding database (such as the number of base table rows, the histogram between two columns) due to the complexity of the query in the traditional associated subquery scenario, thereby causing the efficiency of the final optimized execution plan to be not the optimal solution.

[0046] In addition, for the calculation process: the above-mentioned calculation engine can split each operator in the subtree corresponding to the target subquery statement in the above-mentioned query syntax tree into multiple parts, execute them in parallel on different nodes, and finally aggregate the result sets together to achieve the purpose of parallel acceleration. Among them, the query results generated by the above-mentioned multiple nodes can be collected based on a distributed information transmission framework (such as AnalyticDB for MySQL) or a distributed result set collection link (such as a query link commonly used in distributed databases). The former can be more efficient and have lower latency than the latter.

[0047] It can be understood by those skilled in the art that the principles and specific implementation methods of the optimizer and computing engine mentioned above have been basically disclosed in the relevant technologies. This specification focuses on achieving higher query efficiency and lower latency than single use through multi-stage reuse of the target query statements by the two, so this specification will not be described in detail.

[0048] Of course, if the target subquery statement does not exist in the above target query statement, it means that the query syntax tree does not contain a subtree corresponding to the target subquery statement. Therefore, based on the above optimizer and the above computing engine, the query syntax tree can be optimized and calculated in two stages to obtain the query result of the target query statement, thereby improving the overall data query efficiency.

[0049] S203, adjusting the query syntax tree according to the query result of the target subquery statement, and performing two-stage optimization on the adjusted query syntax tree based on the optimizer, and performing two-stage calculation on the optimization result based on the calculation engine to obtain the query result of the target query statement.

[0050] After the distributed database management server obtains the query result of the target subquery statement, the query syntax tree can be readjusted, and the adjusted query syntax tree can be optimized and calculated in two stages based on the above-mentioned optimizer and the above-mentioned computing engine to obtain the query result of the target query statement as a whole. Specifically: the adjusted query syntax tree can be converted into a corresponding second execution plan by the optimizer; the second execution plan is sent to multiple nodes of the distributed database by the computing engine, so that multiple nodes execute the plan and generate the query result of the corresponding target query statement, by collecting the query results of the target query statements of multiple nodes, and aggregating the query results of all target query statements into the query result of the target query statement. It can be understood by those skilled in the art that the so-called two-stage optimization and calculation is basically the same as the calculation process of the one-stage optimization and calculation, and the only difference is that the syntax tree objects calculated are different, so this specification will not be repeated here.

[0051] The above query syntax tree mentions the change in the relationship between the original query syntax tree and the subtree corresponding to the target subquery statement. At this time, the query result corresponding to the subtree has actually been determined in the first stage of optimization and calculation.

[0052] In one embodiment, the query result of the target subquery statement can be converted into a corresponding subtree to be replaced, wherein the query result is represented in the form of a screening operator, and then the determined subtree is replaced with the subtree to be replaced. The replaced query syntax tree can be regarded as no longer having the subquery statement (because the result of the subquery statement has been queried and determined), that is, it is equivalent to an ordinary query, so the required computing resources are less and the query efficiency is higher. Of course, other nodes of the query syntax tree outside the above subtree can also undergo adaptive structural modifications before and after the replacement according to the adjustment of the subtree to be replaced, and the result will not change substantially.

[0053] The following is an example of student performance statistics in a campus scenario. Figure 3 , 4a 4b describes in detail the data query method of the distributed database. The specific steps of the method are as follows:

[0054] First, assume that there are three tables t1, t2, and t3 in a distributed database. Among them, t1 is the student information table, which contains four columns: student ID student_id, student name student_name, student age age, and student gender gender. The whole table has 5,000 rows, which is used to record students' personal information; t2 is the student course table, which contains three columns: student ID student_id, course ID class_id, and course score score. The whole table has 20,000 rows, which is used to record the scores of each course; t3 is the course information table, which contains three columns: course ID class_id, course grade grade, and teacher ID teacher_id. The whole table has 50 rows, which is used to record the grade and teacher information of each course. If the school's education committee wants to query "the average score of fourth-grade female students in each course", they can enter the following SQL statement as the target query statement to query the above distributed database: "select t2.class_id,avg(t2.score)from t1,t2 where t1.student_id=t2.student_id and t1.gender='F'and t2.class_id in(select t3.class_id from t3 where t3.grade=4)group by t2.class_id".

[0055] By performing syntax tree analysis on the above target query statement, we can get Figure 4aThe overall query syntax tree is obtained, and the scalar / IN subquery operator is used to identify that the target query statement contains an IN subquery statement, so multi-stage optimization and calculation can be performed. At this time, the subtree of the target subquery statement "select t3.class_id from t3 where t3.grade=4" can be determined based on the / IN subquery operator "semi-join" (i.e., located at Figure 4a , which is surrounded by a dotted box and is located in the right tree part of the semi-join operator). Among them, for the query syntax tree, the grouping operator corresponds to the SQL statement "group by t2.class_id", the semi-join operator corresponds to the SQL statement "t2.class_id in t3.class", the filter operator A corresponds to the SQL statement "t3.grade=4 (assuming there are 10 records that meet the conditions)", the table scan operator A corresponds to the scan of "t3" (scan number 50 rows), the table join operator corresponds to the SQL statement "t1.student_id=t2.student_id", the filter operator B corresponds to the SQL statement "t1.gender='F' (assuming there are 2500 records that meet the conditions)", the table scan operator B corresponds to the scan of "t3" (scan number 5000 rows), and the table scan operator C corresponds to the scan of "t2" (scan number 20000 rows).

[0056] Then, a phase of optimization and calculation is performed, that is, the subtree corresponding to the determined target subquery statement (i.e., filter operator A and table scan operator A) is calculated by the optimizer and calculation engine in each distributed node with the same traditional execution process to obtain the query result of the target subquery statement "(class_2, class_5, class_7, class_8, class_9, class_20, class_21, class_22, class_23, class_24)" (assuming that the distributed database of distributed node 1 has data of "class_2, class_5, class_7, class_8, class_9, class_20", and the distributed database of distributed node 2 has data of "class_21, class_22, class_23, class_24"). In other words, the query result of the subquery "select t3.class_id from t3 where t3.grade=4" is 10 rows of records, and the content is as above. The query result of the target subquery statement can be returned to the distributed database management server through the distributed information transmission framework of AnalyticDB for MySQL for aggregation.

[0057] According to the query result of the aggregated target subquery statement, the above query syntax tree and the corresponding target query statement can be adjusted, that is, "select t2.class_id,avg(t2.score)from t1,t2 where t1.student_id=t2.student_id and t1.gender='F'and t2.class_id in(class_2,class_5,class_7,class_8,class_9,class_20,class_21,class_22,class_23,class_24)group by t2.class_id". The adjusted syntax tree can be seen in detail. Figure 4b , where the semi-join operator disappears, and the query results corresponding to the original filter operator A and the table scan operator A are used as the pre-filter conditions of the table scan operator C, and are replaced by the filter operator D before the node where the table scan operator C is located (it should be noted that the reason why the filter operator D is before the table scan operator C but not before other operators in the query syntax tree is that the query result of the filter operator D corresponds to "t2.student_id" and the table scan operator C is a scan operator for "t2", so there is a corresponding logical association between the two in the execution plan). At this time, the filter operator D corresponds to the SQL statement "t2.class_id in(class_2,class_5,class_7,class_8,class_9,class_20,class_21,class_22,class_23,class_24) (assuming that there are 4000 records that meet the conditions)". Compared with the query syntax tree before adjustment, the execution plan corresponding to the adjusted query syntax tree has lower calculation amount and more reasonable execution order, so the query efficiency is higher.

[0058] In the process of executing the two-stage optimization and calculation, the query result of the target query statement corresponding to the adjusted query syntax tree can be calculated by the optimizer and calculation engine in each distributed node in the same way as the traditional execution process, and returned to the client as the final execution result. In this process, since the real statistical information of the subquery is used as the input of the optimizer CBO in the optimizer to optimize the execution plan corresponding to the original target query statement, the accuracy of the cost estimation is improved. At the same time, for RBO, after rewriting the query syntax tree with the filter operator, the original Cross / Semi / Anti and other complex operators with poor performance are eliminated. These complex operators consume a lot of device processors when the amount of associated external data is large and will increase the overall query response time. The filter operator is relatively lightweight, so after eliminating the complex operators, the processor usage and query response time will be significantly reduced.

[0059] Figure 5 is a schematic structural diagram of an electronic device in an exemplary embodiment. Figure 5 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other required hardware. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a data query device for a distributed database at the logical level. Of course, in addition to the software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0060] Corresponding to an embodiment of a data query method for a distributed database, this specification also provides an embodiment of a data query device for a distributed database.

[0061] Please refer to Figure 6 , Figure 6 FIG. 1 is a schematic diagram of a data query device for a distributed database, showing an exemplary embodiment. Figure 6 As shown, in a software implementation, the device may include:

[0062] A query syntax tree generating unit 601 is used to obtain a target query statement for the distributed database and generate a query syntax tree corresponding to the target query statement;

[0063] A first query result acquisition unit 602 is used to determine a subtree corresponding to a target subquery statement in the target query statement in the query syntax tree, and acquire the determined subtree to acquire a query result of the target subquery statement;

[0064] The second query result obtaining unit 603 is configured to adjust the query syntax tree according to the query result of the target sub-query statement, and obtain the adjusted query syntax tree to obtain the query result of the target query statement.

[0065] Optionally, the first query result obtaining unit 602 is specifically configured to:

[0066] Traversing each node in the query syntax tree;

[0067] In the case where any node contains an operator of a target type corresponding to the target sub-query statement, a subtree formed by the any node and its descendant nodes is used as a subtree corresponding to the target sub-query statement in the query syntax tree.

[0068] Optionally, the operator of the target type includes any one of the following: a scalar subquery and an IN subquery.

[0069] Optionally, the second query result obtaining unit 603 is specifically configured to:

[0070] Converting the query result of the target subquery statement into a corresponding subtree to be replaced, wherein the query result is represented in a form based on a screening operator;

[0071] The determined subtree is replaced with the subtree to be replaced.

[0072] Optionally, the first query result obtaining unit 602 is specifically configured to:

[0073] Converting the determined subtree into a corresponding first execution plan by the optimizer;

[0074] Sending the first execution plan to multiple nodes of the distributed database through the first-stage computing engine, so that the multiple nodes execute the execution plan and generate query results of corresponding target sub-query statements;

[0075] The query results of the target sub-query statements of the multiple nodes are collected, and the query results of all the target sub-query statements are aggregated into the query result of the target sub-query statement.

[0076] Optionally, the second query result obtaining unit 603 is specifically configured to:

[0077] Converting the adjusted query syntax tree into a corresponding second execution plan by the optimizer;

[0078] Sending the second execution plan to multiple nodes of the distributed database through the computing engine, so that the multiple nodes execute the execution plan and generate query results of the corresponding target query statements;

[0079] Collect the query results of the target query statements of the multiple nodes, and aggregate the query results of all the target query statements into the query result of the target query statement. Optionally, the query results generated by the multiple nodes are collected based on a distributed information delivery framework or a distributed result set collection link. Optionally, the device also includes:

[0080] The query skipping unit 604 is configured to, when the target sub-query statement does not exist in the target query statement, perform the two-stage optimization and calculation on the query syntax tree to obtain a query result of the target query statement.

[0081] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0082] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. Ordinary technicians in this field can understand and implement it without paying creative work.

[0083] The embodiments of the subject matter and functional operations described in this specification may be implemented in the following: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or a combination of one or more of them. The embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules in computer program instructions encoded on a tangible non-temporary program carrier to be executed by a data processing device or to control the operation of the data processing device. Alternatively or additionally, the program instructions may be encoded on an artificially generated propagation signal, such as a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information and transmit it to a suitable receiver device for execution by a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

[0084] The processes and logic flows described in this specification may be performed by one or more programmable computers executing one or more computer programs to perform corresponding functions by operating on input data and generating outputs. The processes and logic flows may also be performed by a dedicated logic circuit, such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit), and the apparatus may also be implemented as a dedicated logic circuit.

[0085] Computers suitable for executing computer programs include, for example, general and / or special microprocessors, or any other type of processing unit. Typically, the processing unit will receive instructions and data from a read-only memory and / or a random access memory. The basic components of a computer include a processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, the computer will also include one or more large-capacity storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operably coupled to this large-capacity storage device to receive data from it or transmit data to it, or both. However, the computer does not necessarily have such a device. In addition, the computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA, Personal Digital Assistant), a mobile audio or video player, a game console, a global positioning system (GPS, Global Positioning System) receiver, or a portable storage device such as a universal serial bus (USB, Universal Serial Bus) flash drive, just to name a few.

[0086] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including, for example, semiconductor memory devices (e.g., EPROM (Electrically Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable read only memory) and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD ROM (Compact Disc Read Only Memory) and DVD-ROM (Digital Video Disc-Read Only Memory). The processor and memory may be supplemented by or incorporated in special purpose logic circuits.

[0087] Although this specification includes many specific implementation details, these should not be interpreted as limiting the scope of any invention or the scope of protection claimed, but are mainly used to describe the features of the specific embodiments of specific inventions. Certain features described in multiple embodiments in this specification may also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. In addition, although features may work in certain combinations as described above and even initially claim protection, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may point to a sub-combination or a variation of a sub-combination.

[0088] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that these operations be performed in the particular order shown or performed sequentially, or requiring that all illustrated operations be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

[0089] Thus, specific embodiments of the subject matter have been described. Furthermore, the processes depicted in the figures do not necessarily require the particular order shown or sequential order to achieve desired results. In some implementations, multitasking and parallel processing may be advantageous.

[0090] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A data query method for a distributed database, characterized in that: The method comprises: Obtaining a target query statement for the distributed database, and generating a query syntax tree corresponding to the target query statement; Determine a subtree corresponding to a target subquery statement in the target query statement in the query syntax tree, perform a one-stage optimization on the determined subtree based on an optimizer, and perform a one-stage calculation on the optimization result based on a calculation engine to obtain a query result of the target subquery statement; The query syntax tree is adjusted according to the query result of the target subquery statement, and two-stage optimization is performed on the adjusted query syntax tree based on the optimizer, and two-stage calculation is performed on the optimization result based on the calculation engine to obtain the query result of the target query statement.

2. The method according to claim 1, characterized in that The determining a subtree corresponding to a target subquery statement in the target query statement in the query syntax tree includes: Traversing each node in the query syntax tree; In the case where any node contains an operator of a target type corresponding to the target sub-query statement, a subtree formed by the any node and its descendant nodes is used as a subtree corresponding to the target sub-query statement in the query syntax tree.

3. The method according to claim 2, characterized in that The operator of the target type includes any of the following: a scalar subquery and an IN subquery.

4. The method according to claim 1, characterized in that: The step of adjusting the query syntax tree according to the query result of the target sub-query statement includes: Converting the query result of the target subquery statement into a corresponding subtree to be replaced, wherein the query result is represented in a form based on a screening operator; The determined subtree is replaced with the subtree to be replaced.

5. The method according to claim 1, characterized in that The step of performing one-stage optimization on the determined subtree based on the optimizer, and performing one-stage calculation on the optimization result based on the calculation engine to obtain the query result of the target subquery statement includes: Converting the determined subtree into a corresponding first execution plan by the optimizer; Sending the first execution plan to multiple nodes of the distributed database through the computing engine, so that the multiple nodes execute the execution plan and generate query results of corresponding target sub-query statements; The query results of the target sub-query statements of the multiple nodes are collected, and the query results of all the target sub-query statements are aggregated into the query result of the target sub-query statement.

6. The method according to claim 1, characterized in that The optimizing process based on the optimizer performs two-stage optimization on the adjusted query syntax tree, and the computing engine performs two-stage calculation on the optimization result to obtain the query result of the target query statement, including: Converting the adjusted query syntax tree into a corresponding second execution plan by the optimizer; Sending the second execution plan to multiple nodes of the distributed database through the computing engine, so that the multiple nodes execute the execution plan and generate query results of the corresponding target query statements; The query results of the target query statements of the multiple nodes are collected, and the query results of all the target query statements are aggregated into the query result of the target query statement.

7. The method according to claim 5 or 6, characterized in that: The query results generated by the multiple nodes are collected based on a distributed information delivery framework or a distributed result set collection link.

8. The method according to claim 1, characterized in that The method further comprises: In the case that the target sub-query statement does not exist in the target query statement, the two-stage optimization and calculation are performed on the query syntax tree to obtain a query result of the target query statement.

9. A data query device for a distributed database, characterized in that: The device comprises: A query syntax tree generating unit, configured to obtain a target query statement for the distributed database and generate a query syntax tree corresponding to the target query statement; A first query result acquisition unit, configured to determine a subtree corresponding to a target subquery statement in the target query statement in the query syntax tree, and perform a one-stage optimization on the determined subtree based on an optimizer, and perform a one-stage calculation on the optimization result based on a calculation engine to obtain a query result of the target subquery statement; The second query result acquisition unit is used to adjust the query syntax tree according to the query result of the target sub-query statement, and perform two-stage optimization on the adjusted query syntax tree based on the optimizer, and perform two-stage calculation on the optimization result based on the calculation engine to obtain the query result of the target query statement.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images