A query processing method, device, equipment and readable storage medium
By generating and calculating the feature signature of the candidate plan tree of the query statement, the cost details table is used to directly calculate the running cost, which solves the problem of inaccurate calculation of the plan tree in database queries, reduces computing resource consumption and improves efficiency.
Patent Information
- Application Number
- CN202411137179.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-08-19
AI Technical Summary
When the database executes SQL queries, the cost calculation of the plan tree is inaccurate due to missing statistical information or sampling errors, which increases the consumption of computing resources.
By generating a candidate plan tree for the received query statement, calculating its feature signature, and using the historical feature signature and operation cost relationship in the cost details table, the operation cost of the candidate plan tree is directly calculated, thereby selecting the optimal plan tree to execute.
It improves the accuracy of cost calculation of plan trees, reduces computing resource consumption, reduces additional computing tasks, and improves cost calculation efficiency.
Smart Images

Figure CN118656395B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a query processing method, apparatus, device, and readable storage medium. Background Art
[0002] When the database executes statements in Structured Query Language (SQL), the plan tree is a detailed description of the database's query execution. The plan tree is a tree structure consisting of multiple nodes, each of which represents an execution operation. The root node of the tree represents the execution plan for the entire query, while the leaf nodes represent the lowest-level operations, such as table scans, index scans, etc.
[0003] In actual applications, the plan tree of a database query will generate multiple equivalent plan trees. Each equivalent plan tree can rely on statistical information (the raw material information that provides cost calculation) and the cost model to calculate the running cost. The database query optimizer can select the plan tree with the lowest cost in the search space as the optimal plan tree, and then execute this optimal plan tree through the corresponding computing engine to implement the query. However, the above-mentioned method of relying on statistical information to calculate the running cost will reduce the accuracy of the collected statistical information once there is a lack of statistical information, statistical information sampling calculation errors, and automatic update mechanism, resulting in the calculated running cost being inaccurate; in addition, collecting corresponding statistical information when calculating the running cost requires triggering additional computing tasks, which will bring additional computing overhead to the database. Summary of the invention
[0004] The embodiments of the present application provide a query processing method, apparatus, device and readable storage medium, which can improve the accuracy of cost calculation of a plan tree and reduce computing resource consumption in a query business.
[0005] An embodiment of the present application provides a query processing method, including:
[0006] Parse the received query statement and generate N candidate plan trees for the query statement; N is a positive integer;
[0007] Perform feature signature calculation on each candidate plan tree to obtain a feature signature corresponding to each candidate plan tree; a feature signature corresponding to a candidate plan tree is used to reflect the query logic of the candidate plan tree;
[0008] Obtain a cost details table; the cost details table contains a correspondence between a historical feature signature set and a historical running cost set, there is a correspondence between a historical feature signature in the historical feature signature set and a historical running cost in the historical running cost set, and a historical feature signature refers to a feature signature corresponding to a historical plan tree of a historical query statement;
[0009] Calculate the running cost of each candidate plan tree through the corresponding relationship between the feature signature corresponding to each candidate plan tree, the historical feature signature set in the cost details table and the historical running cost set;
[0010] The optimal plan tree is selected from N candidate plan trees based on the running cost of each candidate plan tree, and the optimal plan tree is submitted to the calculation engine for execution.
[0011] An embodiment of the present application provides a query processing device, including:
[0012] The plan tree generation module is used to parse the received query statement and generate N candidate plan trees of the query statement; N is a positive integer;
[0013] The signature acquisition module is used to calculate and process the feature signature of each candidate plan tree to obtain the feature signature corresponding to each candidate plan tree; the feature signature corresponding to a candidate plan tree is used to reflect the query logic of the candidate plan tree;
[0014] A table acquisition module is used to obtain a cost details table; the cost details table contains a correspondence between a historical feature signature set and a historical running cost set, there is a correspondence between a historical feature signature in the historical feature signature set and a historical running cost in the historical running cost set, and a historical feature signature refers to a feature signature corresponding to a historical plan tree of a historical query statement;
[0015] The cost calculation module is used to calculate the running cost of each candidate plan tree through the feature signature corresponding to each candidate plan tree, the correspondence between the historical feature signature set and the historical running cost set in the cost details table;
[0016] The plan tree selection module is used to select the best plan tree from N candidate plan trees based on the running cost of each candidate plan tree, and submit the best plan tree to the calculation engine for execution.
[0017] In one embodiment, each candidate plan tree is composed of one or more operators;
[0018] The signature acquisition module performs feature signature calculation processing on each candidate plan tree to obtain the specific implementation method of the feature signature corresponding to each candidate plan tree, including:
[0019] Determine any one of the N candidate plan trees as a target candidate plan tree;
[0020] Obtaining a tree level of each operator in the target candidate plan tree, and sorting one or more operators in the target candidate plan tree according to the tree level of each operator to obtain a first operator sequence;
[0021] Perform operator signature calculation on each operator in the first operator sequence to obtain an operator signature corresponding to each operator;
[0022] According to the arrangement order of the one or more operators in the first operator sequence, the operator signatures corresponding to the one or more operators are sorted to obtain an operator signature sequence;
[0023] The operator signature sequence is determined as the feature signature corresponding to the target candidate plan tree.
[0024] In one embodiment, the signature acquisition module performs operator signature calculation on each operator in the first operator sequence to obtain the specific implementation of the operator signature corresponding to each operator, including:
[0025] Determine any one operator in the first operator sequence as a target operator;
[0026] Perform feature interpretation processing on the target operator to obtain the interpretation feature field of the target operator;
[0027] The interpretation feature field of the target operator is processed to obtain the operator signature of the target operator.
[0028] In one embodiment, the signature acquisition module performs feature processing on the interpretation feature field of the target operator to obtain the operator feature signature of the target operator, and the specific implementation method includes:
[0029] According to the operator logic of the target operator, the interpretation feature field of the target operator is logically classified to obtain the logical classification field of the target operator;
[0030] Based on the standard format of the field, the logical classification fields of the target operator are standardized to obtain the standard interpretation fields of the target operator;
[0031] The standard interpretation field of the target operator is determined as the operator signature of the target operator.
[0032] In one embodiment, the signature acquisition module performs logical classification processing on the interpretation feature field of the target operator according to the operator logic of the target operator to obtain a specific implementation method of the logical classification field of the target operator, including:
[0033] Classify the target operator according to the operator logic of the target operator to obtain the target logic category to which the target operator belongs;
[0034] The first method field in the interpretation feature field of the target operator is replaced with the second method field used to characterize the logical method indicated by the target logical category, thereby obtaining the logical classification field of the target operator.
[0035] In one embodiment, the signature acquisition module performs standardization processing on the logical classification interpretation field of the target operator based on the field standard format to obtain the specific implementation of the standard interpretation field of the target operator, including:
[0036] Obtain non-standard fields whose field formats do not conform to the field standard format from the logical classification fields of the target operator;
[0037] According to the field standard format, the non-standard field is converted into a standard field; the field format of the standard field conforms to the field standard format;
[0038] The field composed of the standard field and the remaining fields is determined as the standard interpretation field of the target operator; the remaining fields refer to the fields other than the standard fields in the logical classification fields of the target operator.
[0039] In one embodiment, the specific implementation of the plan tree generation module parsing the received query statement and generating N candidate plan trees for the query statement includes:
[0040] Receive query statements to be processed;
[0041] Performing syntax parsing on the query statement to obtain a syntax parsing tree of the query statement;
[0042] Perform logical transformation on the syntax parse tree to generate N candidate plan trees for the query statement.
[0043] In one embodiment, the cost calculation module calculates the operation cost of each candidate plan tree by using the feature signature corresponding to each candidate plan tree, the correspondence between the historical feature signature set and the historical operation cost set in the cost details table, including:
[0044] Determine any one of the N candidate plan trees as a target candidate plan tree;
[0045] Based on the feature signature corresponding to the target candidate plan tree, traverse the historical feature signature set in the cost details table;
[0046] If there is a target historical feature signature in the historical feature signature set that is the same as the feature signature corresponding to the target candidate plan tree, the historical running cost corresponding to the target historical feature signature is determined as the running cost of the target candidate plan tree;
[0047] If there is no target historical feature signature in the historical feature signature set that is identical to the feature signature corresponding to the target candidate plan tree, the target candidate plan tree is split into Q plan subtrees, and the running cost of the target candidate plan tree is calculated based on the correspondence between the Q plan subtrees, the historical feature signature set in the cost details table and the historical running cost set; Q is a positive integer.
[0048] In one embodiment, the cost calculation module calculates the operation cost of the target candidate plan tree according to the correspondence between the Q plan subtrees, the historical feature signature set and the historical operation cost set in the cost details table, and the specific implementation method includes:
[0049] Get the sub-feature signature corresponding to each plan subtree;
[0050] Traverse the historical feature signature set based on the sub-feature signature corresponding to each plan subtree;
[0051] According to the traversal results of the historical feature signature set, the running cost of the target candidate plan tree is determined.
[0052] In one embodiment, the target candidate plan tree is composed of one or more operators; the traversal result of the historical feature signature set is the result of including the target sub-feature signature in the historical feature signature set, and the target sub-feature signature refers to any sub-feature signature among the Q sub-feature signatures;
[0053] The cost calculation module determines the specific implementation method of the running cost of the target candidate plan tree according to the traversal results of the historical feature signature set, including:
[0054] The plan subtree corresponding to the target sub-feature signature is determined as the target plan subtree, and the operators included in the target plan subtree are determined as the cost-known operators;
[0055] Determine the operators other than the operators with known costs in the target candidate plan tree as the remaining operators, and obtain the cost raw material information corresponding to the remaining operators; the cost raw material information includes the historical running cost corresponding to the target sub-feature signature;
[0056] Calculate the operator running costs corresponding to the remaining operators based on the cost raw material information corresponding to the remaining operators;
[0057] The running cost of the target candidate plan tree is determined according to the operator running costs corresponding to the remaining operators.
[0058] In one embodiment, the target candidate plan tree is composed of one or more operators; the traversal result of the historical feature signature set is the result that the historical feature signature set does not contain any sub-feature signature;
[0059] The cost calculation module determines the specific implementation method of the running cost of the target candidate plan tree according to the traversal results of the historical feature signature set, including:
[0060] Obtaining the tree level of each operator in the target candidate plan tree, and sorting one or more operators in the target candidate plan tree according to the tree level of each operator to obtain a second operator sequence corresponding to the target candidate plan tree;
[0061] According to the arrangement order of one or more operators in the second operator sequence, the operator running cost corresponding to each operator is calculated in sequence;
[0062] The operator running cost corresponding to the sequence end operator is determined as the running cost of the target candidate plan tree; the sequence end operator refers to the operator at the sequence end position in the second operator sequence.
[0063] In one embodiment, the specific implementation of the plan tree selection module selecting the optimal plan tree from N candidate plan trees based on the running cost of each candidate plan tree includes:
[0064] Get the minimum running cost from the running costs of N candidate plan trees;
[0065] The candidate plan tree indicated by the minimum running cost is determined as the optimal plan tree of the query statement.
[0066] An embodiment of the present application provides a computer device, including: a processor and a memory;
[0067] The memory stores a computer program, and when the computer program is executed by the processor, the processor executes the method in the embodiment of the present application.
[0068] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, the method in the embodiment of the present application is executed.
[0069] In one aspect of the present application, a computer program product is provided, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the method provided in one aspect of the embodiments of the present application.
[0070] In an embodiment of the present application, a method for calculating the running cost of a plan tree is provided, which can improve the accuracy of the calculated running cost and reduce the resource consumption of calculating the running cost. Specifically, after receiving a query statement, the query statement can be parsed to obtain N candidate plan trees for the query statement; for any candidate plan tree, a feature signature calculation process can be performed to obtain a feature signature that can be used to reflect the query logic of the candidate plan tree, and then, through the correspondence between each historical feature signature and the historical running cost in the cost details table, the running cost of the feature signature corresponding to each candidate plan tree can be calculated and matched, thereby obtaining the running cost of each candidate plan tree; according to the running cost of each candidate plan tree, the optimal plan tree can be found and submitted to the computing engine for execution. It should be understood that the present application determines the historical feature signature for each historical plan tree, and calculates and records the historical running cost corresponding to each historical feature signature, so that when you want to calculate the running cost of the candidate plan tree of a query statement, you can directly calculate and match the running cost of the candidate plan tree based on the feature signature corresponding to the candidate plan tree, through the historical running cost of each historical feature signature. Compared with the traditional cost model-based estimation, there is no need to obtain additional statistical information, and the running cost obtained is more complete and accurate; in addition, since different query statements can correspond to the same plan tree or plan subtree in actual applications, the method of using feature signatures as the basis for running cost calculation can realize the sharing of historical running costs. In this way, when calculating the running cost of a plan tree, it can be based on the shared existing historical running costs to achieve fast and efficient cost calculation, which can not only improve the cost calculation efficiency, but also reduce the additional computing consumption paid during cost calculation, thereby reducing resource consumption during calculation. In summary, the present application can improve the cost calculation accuracy of the plan tree of the query statement in the query business and reduce computing resource consumption. BRIEF DESCRIPTION OF THE DRAWINGS
[0071] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0072] Figure 1 It is a schematic diagram of the architecture of a solution system provided by an exemplary embodiment of the present application;
[0073] Figure 2 is a flowchart of a query processing method provided by an exemplary embodiment of the present application;
[0074] Figure 3 It is a schematic diagram of a plan tree conversion of a query statement provided in an embodiment of the present application;
[0075] Figure 4 It is a schematic diagram of a logical architecture for generating a cost details table provided in an embodiment of the present application;
[0076] Figure 5 It is a schematic diagram of plan tree splitting provided in an embodiment of the present application;
[0077] Figure 6 is a schematic diagram of a traditional cost calculation method provided in an embodiment of the present application;
[0078] Figure 7 It is a schematic diagram of an operation cost of a plan tree based on a feature signature calculation provided in an embodiment of the present application;
[0079] Figure 8 It is a flow chart of generating a feature signature for a candidate plan tree provided by an embodiment of the present application;
[0080] Fig. 9 It is a schematic diagram of performing feature interpretation processing on an operator provided in an embodiment of the present application;
[0081] Fig.10 It is a logical flow diagram of generating a feature signature corresponding to a plan tree provided by an embodiment of the present application;
[0082] Fig.11 is a structural diagram of a query processing device provided in an embodiment of the present application;
[0083] Fig.12 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0084] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0085] The embodiments of the present application involve database and other related technologies. For ease of understanding, the database and related technical terms and concepts will be briefly explained below.
[0086] 1. Database
[0087] A database is a warehouse that organizes, stores, and manages data according to data structures. It is a collection of large amounts of data that is stored in a computer for a long time, organized, shareable, and uniformly managed. The data in a database is organized, described, and stored according to a certain data model, has low redundancy, high data independence, and easy scalability, and can be shared by various users.
[0088] 2. Structured Query Language (SQL)
[0089] Structured Query Language is a standard computer language for accessing and processing databases. It provides statements for various tasks, including: querying data; inserting, updating, and deleting a row of data in a table; creating, replacing, changing, or deleting an object; controlling access to the database and its objects; and ensuring the consistency and integrity of the database.
[0090] 3. Abstract Syntax Tree (AST)
[0091] The syntax tree is also called a parse tree or an abstract syntax tree. When the database system executes SQL, it will convert the SQL into an AST syntax tree based on lexical analysis and syntax analysis. The conversion enables the database to understand and execute the query. In this application, the syntax tree is called a syntax parse tree.
[0092] 4. Plan Tree
[0093] The plan tree is also called the query plan, which includes the logical plan tree and the physical plan tree. Unlike the syntax tree mentioned above, the plan tree is a detailed description of query execution in the database, and query optimization is achieved based on relational algebra transformation. The plan tree is a tree structure composed of multiple nodes, each node represents an execution operation. The root node of the tree represents the execution plan of the entire query, while the leaf nodes represent the lowest-level operations, such as table scan operations, index scan operations, etc.
[0094] 5. Query Optimizer
[0095] The query optimizer is one of the key components of the database query processor and can be used to optimize the plan tree. The query optimizer can be divided into two categories: 1. Rule-based Optimizer (RBO); 2. Cost-based Optimizer. Among them, the RBO optimizer can sequentially execute a set of preset fixed optimization rules to perform equivalent transformation of the plan tree to obtain multiple equivalent plan trees; while the CBO optimizer is a cost / cost-based optimizer, which relies on statistical information (the statistical information in this application may refer to the raw material information that provides cost calculation for the CBO optimizer within the scope of the CBO optimizer. The statistical information may include but is not limited to: related information at the table level, partition level and field level, for example: table storage size and number of table rows at the table level, number of partition rows at the partition level, field cardinality value at the field level, etc. can all be used as statistical information) and cost model (cost model is a data model that can calculate the cost of the plan tree. Combining statistical information and node information of the plan tree, the cost model can calculate the query execution cost of the plan tree from the bottom up from the plan tree. The query execution cost is the resources consumed by executing the plan tree, which can be called the cost / running cost of the plan tree) to calculate the running cost of each equivalent plan tree, and then select the equivalent plan tree with the lowest cost in the search space as the optimal plan tree.
[0096] In practical applications, the SQL processing flow in the database includes at least the following five steps: 1. Parse the SQL statement to obtain the AST abstract syntax tree (i.e., syntax parsing tree); 2. Based on the AST abstract syntax tree, verify the legality and bind related metadata, and convert it into a logical plan tree; 3. The logical plan tree implements rule-based query rewriting based on the RBO optimizer (i.e., sequentially execute a set of preset fixed optimization rules to perform equivalent conversion of the plan tree), and different equivalent plan trees can be obtained through the RBO optimizer; 4. The logical plan tree implements cost-based query optimization based on CBO, binds physical attributes, and generates the optimal physical plan tree (i.e., combining statistical information and node information of the plan tree, calculating the running cost of each equivalent plan tree from bottom to top based on the cost model, and then selecting the plan tree with the lowest cost as the optimal plan tree in the search space); 5. Submit the executable plan tree (i.e., the optimal plan tree, which can be called the optimal plan tree or the optimal physical plan tree) to the computing engine for execution. Among them, the computing engine generally refers to the system and components used to perform computing tasks in the field of big data, such as Spark / Presto / Hive, etc.
[0097] In traditional technology, when the CBO optimizer calculates the running cost of each plan tree (such as an equivalent plan tree), the corresponding statistical information (such as table statistics) can be obtained based on the leaf node (such as table scan), and the execution / running cost of the plan tree can be calculated from the leaf node to the bottom up through the cost model defined by each node. However, due to the characteristics of fast data changes and huge data volume, statistical information is prone to incorrect collection and omission during sampling and collection, and the collected statistical information is very likely to be incomplete, which will reduce the accuracy of the cost calculated based on the statistical information. Moreover, in some scenarios with large statistical granularity, the cost of collecting statistical information is very high, which not only consumes more computing resources, but also increases the maintenance cost of statistical information.
[0098] Based on this, in order to improve the accuracy of the calculated running cost and reduce the resource consumption when calculating the running cost, the present application provides a plan tree running cost calculation scheme. Specifically, the plan tree running cost calculation scheme provided in the embodiment of the present application, its process can roughly include the following four steps: 1. After receiving a query statement to be processed, the query statement to be processed (a SQL statement) can be parsed, wherein the statement parsing here can include the above-mentioned syntax parsing processing, the processing of checking the legality and binding related metadata, and the processing of rule-based query rewriting based on the RBO optimizer. In this way, different equivalent plan trees corresponding to the query statement can be obtained through statement parsing. For each equivalent plan tree, it is necessary to calculate the running cost through the CBO optimizer calculator, and then select an optimal plan tree from it as the plan tree to be executed by the computing engine. Then, the present application can refer to each equivalent plan tree as a candidate plan tree for the query statement. After obtaining each candidate plan tree that needs to be cost calculated (here it is assumed that there are N candidate plan trees, N is a positive integer), feature signature calculation processing can be performed on each candidate plan tree to obtain a feature signature corresponding to each candidate plan tree, wherein the feature signature in this application refers to the information obtained after interpreting and feature processing the candidate plan tree, which can be used to characterize the query plan (query logic) of the candidate plan tree, that is, this application can determine a feature signature for each candidate plan tree that can be used to reflect its query logic; 2. After obtaining the feature signature of the candidate plan tree, a cost details table can be obtained, wherein the cost details table records the correspondence between different historical feature signatures and historical operating costs. Specifically, the present application can obtain a batch of SQL statements that have been executed (each SQL statement can be referred to as a historical query statement), and then, the optimal plan tree of each historical query statement can be obtained (that is, the plan tree executed by the computing engine when processing the historical query statement). The optimal plan tree of the historical query statement can be referred to as the historical plan tree. Furthermore, the indicator information of different computing engines when executing the historical plan tree can be identified (for example, data scan size, number of data scan rows, execution time, etc.). By executing the indicator information of the historical plan tree, the resources consumed during execution can be analyzed (this resource may include resources of different indicators, for example, CPU resources, GPU resources, memory resources, etc.). Based on the resources consumed during execution, the running cost of executing the historical plan tree can be analyzed. The running cost of any historical plan tree can be referred to as the historical running cost.After obtaining each historical plan tree and its corresponding historical running cost, the present application can use feature signatures to organize and store the historical plan tree and its corresponding historical running cost. Specifically, the present application can perform feature signature calculation processing on each historical plan tree to determine the feature signature corresponding to each historical plan tree (the feature signature of a historical plan tree can be called the historical feature signature corresponding to the historical plan tree). Then, the historical feature signature of the historical plan tree can be bound to the historical running cost of the historical plan tree. Specifically, a corresponding relationship can be established between the historical feature signature of the historical plan tree and the historical running cost of the historical plan tree. Based on the corresponding relationship between each historical feature signature and the historical running cost, a cost details table can be constructed. In other words, the cost details table in the present application is pre-constructed based on different historical execution behaviors of the computing engine, which contains the corresponding relationship between different historical feature signatures and different historical running costs. For this cost details table, it can be persistently stored, so that it can be used by the subsequent CBO to calculate the running cost of the candidate plan tree of a query statement. 3. After obtaining the cost details table, the running cost of each candidate plan tree can be calculated through the correspondence between the feature signature corresponding to the candidate plan tree, the historical feature signature set in the cost details table, and the historical running cost set. For example, taking a candidate plan tree as an example, you can first traverse the historical feature signature set. If the historical feature signature set contains a historical feature signature that is identical to the feature signature corresponding to the candidate plan tree, then it can be shown that the computing engine has executed a historical plan tree with the same query plan as the candidate plan tree. There is no need to calculate the running cost of the candidate plan tree again. The historical running cost corresponding to the same historical feature signature can be directly obtained, and the historical running cost can be used as the running cost of the candidate plan tree. If the historical feature signature set does not contain a historical feature signature that is identical to the feature signature corresponding to the candidate plan tree, then the candidate plan tree can be deleted. The plan tree is split to obtain different plan subtrees (the plan subtree is used to describe a part of the overall query plan of the candidate plan tree, and can represent a part of the execution strategy in the query. The entire candidate plan tree can be decomposed into multiple smaller parts, namely plan subtrees, each of which can handle a part of the query task, such as filtering, connecting, sorting, etc.), and each plan subtree can generate a feature signature (for easy distinction, the feature signature of the plan subtree can be called a sub-feature signature). The running cost of the candidate plan tree can be calculated based on the sub-feature signatures of each plan subtree, the correspondence between each historical feature signature and the historical running cost in the cost details table.4. After obtaining the running costs corresponding to each candidate plan tree, the minimum running cost can be found. The candidate plan tree corresponding to the minimum running cost can be used as the optimal plan tree. The optimal plan tree can be submitted to the corresponding computing engine, and the computing engine executes the optimal plan tree to complete the query task indicated by the query statement.
[0099] It should be understood that the present application determines a historical feature signature for each historical plan tree, and calculates and records the historical running cost corresponding to each historical feature signature. This allows, when one wants to calculate the running cost of a candidate plan tree for a query statement, to directly calculate and match the running cost of the candidate plan tree based on the feature signature corresponding to the candidate plan tree and the historical running costs of each historical feature signature. Compared with estimation based on a traditional cost model, there is no need to obtain additional statistical information, and the obtained running cost is more complete and accurate. In addition, in actual applications, different query statements can correspond to the same plan tree or plan subtree. Therefore, by using feature signatures as the basis for running cost calculation, the sharing of historical running costs can be achieved. In this way, when calculating the running cost of a plan tree, fast and efficient cost calculation can be achieved based on the shared existing historical running costs. This not only improves the cost calculation efficiency, but also reduces the additional computing consumption incurred during cost calculation, thereby reducing resource consumption during calculation.
[0100] The solution provided in the embodiment of the present application can be applied to any application scenario that requires a database to store business data, including but not limited to short video push scenarios, game scenarios, etc.
[0101] The short video push scenario may refer to a scenario where video data is continuously pushed to the user, and the user can request to update and display the next video data by performing an operation of pulling video data (such as sliding the video display interface of the terminal device). In the short video push scenario, the user can continuously refresh and browse different video data by continuously performing the operation of pulling video data.
[0102] The game scene may refer to a scene in which a user plays an immersive game through a terminal device.
[0103] To sum up, the solution provided by the embodiment of the present application can improve the cost calculation accuracy of the plan tree of the query task in various application scenarios where the database is used to perform query tasks, so as to improve the accuracy of the database query, and at the same time reduce the consumption of computing resources, and to a certain extent can effectively improve the business coverage (such as expanding the applicable scenarios).
[0104] It should be noted that the several application scenarios given above are only examples and do not limit the application scenarios to which the solutions provided in the embodiments of the present application are applicable.
[0105] Furthermore, the solution provided by the embodiment of the present application can be executed by a computer device, which can refer to a device for processing query statements in a database. For example, the computer device can refer to a server that provides query statement processing services. For another example, the computer device can refer to an application node corresponding to any application. The application node can be used to provide corresponding computing services for the application, and can specifically include a terminal or a server. The application node can also include a terminal and a server. To facilitate understanding of the solution provided by the embodiment of the present application, the following is combined with Figure 1 The system architecture diagram shown introduces the application scenarios involved in the embodiments of the present application; wherein, Figure 1 is a schematic diagram of the architecture of a solution system provided by an exemplary embodiment of the present application, such as Figure 1 As shown, the system includes an application layer and a database, and the application layer includes one or more application nodes (for example, Figure 1 The application node 1001, the application node 1002, ..., the application node 100m shown in the figure); the database includes one or more data nodes (for example, Figure 1 There is a connection relationship between the application layer and the database. Among them: any application node can refer to a background server corresponding to an application or a terminal device deploying the application. For ease of understanding, the terminal device or server will be described below:
[0106] 1) The terminal device may include the terminal device used by the user. Depending on the application scenarios and fields to which the solution is applied, the terminals provided in the embodiments of the present application are different. The terminal device may include, but is not limited to: smartphones (such as smartphones with Android systems or smartphones with Internetworking Operating System (IOS)), tablet computers, portable personal computers, mobile Internet devices (MID), vehicle-mounted devices, head-mounted devices, smart homes, and smart voice interaction devices. The embodiments of the present application do not limit the types of terminal devices, which is explained here.
[0107] For example, in the short video push scenario, the terminal device can be a smart phone; that is, in this implementation, the solution provided by the embodiment of the present application can be deployed on the smart phone; when the user uses the smart phone to use the short video push application, the smart phone can establish a connection relationship with the database, so that the smart phone can write the business data in the short video push application (for example: the user's account data, communication data, historical video data watched by the user, historical video data liked, historical video data collected, etc.) into the database, and the data node in the database saves this business data; of course, if the smart phone wants to read the relevant business data of a user in the short video push application from the database, then the smart phone can directly send a data read request about these data to the database, and the database can respond to this request and connect the smart phone to the data node storing the business data requested by the smart phone, so that the data node can return the data requested to be read or queried by the smart phone to the smart phone. Correspondingly, the smart phone can return the business data queried from the database to the user, and can also perform data recommendation and other processing to the user based on the queried business data. In addition, in the short video push scenario, the smartphone can also use the cost calculation scheme of the plan tree provided by this application to efficiently and accurately calculate the running cost of the candidate plan tree of a query statement, thereby improving the efficiency and accuracy of this query task. For another example, in the smart car scenario, the application deployed with the scheme provided by the embodiment of this application is a car application; the types of the car application may include but are not limited to: music, video or games, etc.
[0108] Among them, the application may refer to a computer program for completing one or more specific tasks; by classifying the application according to different dimensions (such as the operation mode and function of the application), the types of the same application under different dimensions can be obtained. For example: according to the operation mode of the application, the application may include but is not limited to: a client installed in the terminal, a small program (as a subprogram of the client) that can be used without downloading and installing, a World Wide Web (Web) application opened through a browser, etc. For another example: according to the functional type of the application, the application may include but is not limited to: an instant messaging (IM) application, a content interaction application, an audio application or a video application, etc. Among them, an instant messaging application refers to an application for instant communication of messages and social interaction based on the Internet. The instant messaging application may include but is not limited to: an application with communication functions, a map application with interactive functions, a game application, etc. Content interaction applications refer to applications that can realize content interaction, such as sharing platforms, personal spaces, and news applications. Audio applications refer to applications that realize audio functions based on the Internet. Audio applications may include but are not limited to: music applications with music playback and editing capabilities, radio applications with radio playback capabilities, or live broadcast applications with live broadcast capabilities, etc. Video applications refer to applications that can play images. Video applications may include but are not limited to: applications with short videos (videos are often short, such as a few seconds or minutes, etc.) (such as short video push applications), applications with long videos (such as videos with longer playback time such as movies or TV series), etc.
[0109] Of course, the solution provided in the embodiment of the present application can be directly deployed on a device (such as a smart phone), a server (such as a backend server) or an application as described above, and can also be deployed in a device or application in the form of a plug-in. The embodiment of the present application does not limit the carrier of the deployment solution.
[0110] 2) The server can be a server corresponding to the terminal, which is used to interact with the terminal to provide computing and application service support for the terminal. Specifically, the server is a background server corresponding to the application deployed in the terminal, which is used to interact with the terminal to provide computing and application servers for the application. Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.
[0111] The terminal and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in this application. In addition, the embodiment of this application does not limit the number of terminals and servers.
[0112] Based on the solution and system architecture described above, the following points need to be explained:
[0113] ① The above-mentioned embodiments of the present application Figure 1 The system shown is for more clearly illustrating the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided in the embodiment of the present application. It is known to those skilled in the art that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems. For example, in an application scenario, the execution subject "application node" of the embodiment of the present application includes a terminal and a server, that is, the terminal and the server jointly execute the solution provided in the embodiment of the present application; it should be understood that in actual applications, the application node can also be a terminal or a server, that is, it supports the terminal or the server to execute the solution provided in the embodiment of the present application alone.
[0114] ② The collection and processing of relevant data in the embodiments of this application should be strictly in accordance with the requirements of relevant laws and regulations. The acquisition of personal information requires the knowledge or consent of the individual subject (or the legal basis for obtaining the information), and the subsequent use and processing of data shall be carried out within the scope of authorization of laws and regulations and the subject of personal information. For example, when the embodiments of this application are applied to specific products or technologies, such as obtaining relevant information or data of users, the user's permission or consent is required, and the collection (such as the historical video data liked by the above-mentioned user), use and processing of relevant data shall comply with the relevant laws, regulations and standards of the relevant regions.
[0115] Based on the above-described solutions and application scenarios, the embodiments of the present application propose a more detailed query processing method. The query processing method proposed in the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings.
[0116] See also Figure 2 , Figure 2 1 is a flowchart of a query processing method provided by an exemplary embodiment of the present application, and the flowchart may refer to the flowchart of the calculation scheme of the running cost of the plan tree provided by the embodiment of the present application. The query processing method may be executed by a computer device in the aforementioned system, such as a terminal and / or a server; the query processing method may at least include the following steps S201-S205:
[0117] Step S201, parse the received query statement to generate N candidate plan trees for the query statement; N is a positive integer.
[0118] In this application, a query statement may refer to a certain SQL statement to be processed, which may be input by a certain object (such as a user). Based on the above description, after obtaining the query statement to be processed, it may be first subjected to syntax parsing to obtain the syntax parsing tree (AST abstract syntax tree) of the query statement. After the syntax parsing tree is subjected to a validity check, the relevant metadata may be bound and converted into a logical plan tree, and the logical plan tree may be subjected to an equivalent conversion of the plan tree through the RBO optimizer to obtain multiple equivalent plan trees. For each equivalent plan tree, it is necessary to calculate the cost of running the equivalent plan tree to find the optimal plan tree based on the cost. Based on this, each equivalent plan tree may be understood as a candidate plan tree, so the candidate plan tree in this application may be understood as an equivalent plan tree obtained after processing by the RBO optimizer. In other words, for the syntax parsing tree, it may be subjected to a logical conversion (including a validity check, binding relevant metadata to convert it into a logical plan tree, and performing an equivalent conversion of the plan tree on the logical plan tree through the RBO optimizer, etc.), thereby obtaining a candidate plan tree for the query statement. To facilitate understanding of the process from SQL statement generation to candidate plan tree, the following will briefly describe it with the help of the attached figure. Figure 3 , Figure 3 Schematic diagram of a plan tree conversion of a query statement provided in an embodiment of the present application. Figure 3 As shown, for a query statement, it can be parsed to obtain a grammar parse tree (AST abstract syntax tree), which is represented by SqlNode (where SqlNode can be used to represent the AST abstract syntax tree, which can represent various parts of the SQL statement, for example, it can represent the SELECT clause, FROM clause, WHERE clause, etc.). The grammar parse tree (SqlNode) is as follows Figure 3 As shown, the syntax parse tree may contain multiple nodes (such as the root node "query", node 301, node 303, etc.); after determining the syntax parse tree, it can be converted into a plan tree, which is represented by RelNode (where RelNode can be used to represent the node of the logical query plan, which is the expression object of the plan tree) for complete maintenance of execution details. Among the multiple nodes contained in the plan tree, each node is a node corresponding to an operator, for example, Figure 3 As shown in the figure, the plan tree contains a Project node corresponding to the Project operator, a Filter node corresponding to the Filter operator, and a TableScan node corresponding to the TableScan operator. Figure 3 The plan tree shown may refer to a candidate plan tree in this application.
[0119] From the above, it can be seen that the specific implementation process of the present application for parsing the received query statement and generating N candidate plan trees for the query statement may include at least but not limited to: first, after the user enters a query statement, the computer device may receive the query statement to be processed; further, the query statement may be parsed to obtain the grammatical parse tree of the query statement; then, the grammatical parse tree may be logically converted to generate N candidate plan trees for the query statement. Based on the above, it can be seen that after obtaining the grammatical parse tree, the grammatical parse tree may be checked for legality, and the relevant metadata may be bound to convert it into a logical plan tree, and the logical plan tree may be converted into an equivalent plan tree through the RBO optimizer to obtain multiple equivalent plan trees. It can be seen that the logical conversion of the grammatical parse tree in the present application may include at least the following two steps: 1. Check the legality of the grammatical parse tree and bind the relevant metadata to obtain the logical plan tree; 2. Perform an equivalent conversion of the logical plan tree through the RBO optimizer to obtain multiple equivalent plan trees (i.e., multiple candidate plan trees, and this application is explained by taking N candidate plan trees as an example).
[0120] Step S202, perform feature signature calculation processing on each candidate plan tree to obtain a feature signature corresponding to each candidate plan tree; the feature signature corresponding to a candidate plan tree is used to reflect the query logic of the candidate plan tree.
[0121] After obtaining each candidate plan tree of the query statement, the present application can perform feature signature calculation processing on each candidate plan tree to obtain the feature signature corresponding to each candidate plan tree. Among them, in actual applications, RelNode is processed as a Java object, which cannot be directly transferred and identified across systems. The CBO optimizer cannot share information when performing cross-system cost calculations. Based on this, in order to achieve information sharing of cost calculations of cross-system CBO optimizers, the present application designs a new operator-level feature signature for the plan tree, by combining the feature signatures of each operator (wherein, combining the feature signatures of each operator here can actually be understood as splicing the feature signatures of each operator. The specific method of combining the feature signatures of each operator can be found in the subsequent Figure 8 According to the relevant description in the corresponding embodiment), the feature signature of the entire plan tree can be obtained, and the feature signature of each operator in the present application can describe the operator feature information of the operator (including operator category, library name, field name, etc.), so the feature signature of the entire plan tree can describe the key feature information of the plan tree (that is, the feature information of the query statement that characterizes the plan tree), and the query logic of the plan tree can be reflected through the feature information of the query statement of the plan tree (for example, the logic of the sequence between each query operation), then the feature signature in the present application can be used to reflect the query logic of the corresponding plan tree. In other words, the feature signature corresponding to the candidate plan tree in the present application can characterize and maintain the feature information of the candidate plan tree (query statement) to reflect the query logic of the candidate plan tree. For the method of generating the feature signature of the candidate plan tree, please refer to the subsequent Figure 8 Related description in the corresponding embodiment.
[0122] Step S203, obtain the cost details table; the cost details table contains the correspondence between the historical feature signature set and the historical running cost set, there is a correspondence between a historical feature signature in the historical feature signature set and a historical running cost in the historical running cost set, a historical feature signature refers to a feature signature corresponding to a historical plan tree of a historical query statement.
[0123] In the present application, after obtaining each candidate plan tree of the query statement and the feature signature corresponding to each candidate plan tree, a cost details table can be obtained. Among them, the cost details table of the present application records the correspondence between different historical feature signatures and different historical running costs. Specifically, the present application can obtain a batch of SQL statements that have been executed by different computing engines (each SQL statement can be called a historical query statement), and then, the plan tree executed by the computing engine when executing the query statement can be obtained. This plan tree is also the optimal plan tree of the historical query statement, and the optimal plan tree for each historical query statement can be called a historical plan tree; further, the indicator information of different computing engines when executing the historical plan tree (for example, data scan size, number of data scan rows, execution time, etc.) can be identified. By executing the indicator information of the historical plan tree, the resources consumed during execution can be analyzed (this resource can include resources of different indicators, such as CPU resources, GPU resources, memory resources, etc.), and the running cost of executing the historical plan tree can be analyzed based on the resources consumed during execution. The running cost of any historical plan tree can be called the historical running cost. After obtaining each historical plan tree and its corresponding historical operating cost, the present application can generate a corresponding feature signature for each historical plan tree (the feature signature of a historical plan tree can be called the historical feature signature corresponding to the historical plan tree). Then, the historical feature signature of the historical plan tree can be bound to the historical operating cost of the historical plan tree. Specifically, a corresponding relationship can be established between the historical feature signature of the historical plan tree and the historical operating cost of the historical plan tree. Based on the corresponding relationship between each historical feature signature and the historical operating cost, a cost details table can be constructed.
[0124] For ease of understanding, please refer to Table 1, which is an exemplary cost details table provided in an embodiment of the present application, as shown in Table 1:
[0125] Table 1
[0126]
[0127] Among them, the cost details table shown in Table 1 includes historical feature signature 1 (feature signature corresponding to historical plan tree 1 of historical query statement 1), historical feature signature 1 (feature signature corresponding to historical plan tree 2 of historical query statement 2), ..., historical feature signature n (feature signature corresponding to historical plan tree 1 of historical query statement n), and also includes historical running cost 1, historical running cost 2, ..., historical running cost n, and in the cost details table, there is a corresponding relationship between historical feature signature 1 and historical running cost 1, a corresponding relationship between historical feature signature 2 and historical running cost 2, ..., a corresponding relationship between historical feature signature n and historical running cost n. Through the cost details table shown in Table 1, the historical running cost of each historical query statement / historical plan tree can be clearly identified.
[0128] That is to say, the cost details table in this application is determined based on the historical execution behavior of different historical query statements by different computing engines. Please refer to Figure 4 , Figure 4 is a schematic diagram of a logical architecture for generating a cost details table provided in an embodiment of the present application. Figure 4 As shown, the logical architecture may include at least the following components: a computing engine component, a historical execution analysis component, a cost processing component, and a cost storage component. For ease of understanding, the functions implemented by each component are briefly described below:
[0129] Calculation engine component: The calculation engine can be used to execute the optimal plan tree for any query statement.
[0130] Historical execution analysis component: The historical execution analysis component mainly performs the following steps: 1. Identify the indicator information (such as data scan size, number of data scan rows, execution time, etc.) generated by different computing engines when executing a plan tree; 2. Based on the indicator information of the computing engine at runtime, analyze the computing resources consumed by the computing engine, and then obtain the running cost generated by the computing engine when executing the plan tree.
[0131] Cost processing component: For different plan trees executed by the computing engine, the cost processing component can be unified and regularized based on feature signatures. Specifically, the cost processing component needs to determine the historical feature signatures of the historical plan tree and bind the running cost generated by the computing engine when executing the historical plan tree. In other words, the cost processing component needs to bind the historical feature signature corresponding to each historical plan tree to its historical running cost (which can refer to establishing a corresponding relationship between the historical feature signature and the historical running cost).
[0132] Cost storage component: The cost storage component can add the correspondence between each historical feature signature and the historical running cost to the cost details table for persistent storage, and provide a calling interface for the subsequent CBO optimizer to calculate the running cost of a plan tree.
[0133] Step S204, calculating the running cost of each candidate plan tree through the feature signature corresponding to each candidate plan tree, the correspondence between the historical feature signature set and the historical running cost set in the cost details table.
[0134] In the present application, after obtaining the cost details table, the running cost of each candidate plan tree can be calculated through the correspondence between the feature signature corresponding to each candidate plan tree, the historical feature signature set in the cost details table and the historical running cost set. Here, taking any candidate plan tree as an example, the specific implementation process for calculating the running cost of the candidate plan tree may at least include but is not limited to: first, for the sake of distinction, the arbitrary candidate plan tree can be called the target candidate plan tree; based on the feature signature corresponding to the target candidate plan tree, the historical feature signature set in the cost details table can be traversed; if there is a historical feature signature set corresponding to the target candidate If the feature signature corresponding to the plan tree is the same as the historical feature signature, then the same historical feature signature can be called the target historical feature signature, and the historical running cost corresponding to this target historical feature signature in the cost details table can be directly determined as the running cost of the target candidate plan tree; if there is no target historical feature signature that is the same as the feature signature corresponding to the target candidate plan tree in the historical feature signature set, then the target candidate plan tree can be split into Q (Q is a positive integer) plan subtrees, and the running cost of the target candidate plan tree can be calculated based on the correspondence between the Q plan subtrees, the historical feature signature set in the cost details table and the historical running cost set.
[0135] Among them, based on the previous description, it can be known that the plan tree in this application can be composed of multiple nodes (each node corresponds to an operator), so it can be considered that the plan tree is composed of multiple operators. Then, the candidate plan tree in this application is also composed of one or more operators. When splitting the target candidate plan tree, the plan subtree can be generated from the bottom up, thereby obtaining different plan subtrees. For ease of understanding, please refer to Figure 5 , Figure 5 Schematic diagram of a plan tree splitting provided in an embodiment of the present application. Figure 5As shown, assuming that the plan tree contains the Project node corresponding to the Project operator, the Filter node corresponding to the Filter operator, and the TableScan node corresponding to the TableScan operator, by splitting the plan subtree from bottom to top, we can get the following Figure 5 The plan subtree 1 shown (the plan subtree 1 only contains the bottom-most table scan (TableScan) node) and the plan subtree 2 (the plan subtree 2 contains the table scan (TableScan) node and the filter (Filter) node).
[0136] Furthermore, after splitting the target candidate plan tree into Q plan subtrees, the method of determining the feature signature for the candidate plan tree provided in the present application can be used to determine the feature signature corresponding to each plan subtree (for ease of distinction, the feature signature corresponding to the plan subtree can be referred to as a sub-feature signature). In fact, after obtaining the feature signature of the target candidate plan tree, the present application, since the feature signature of the target candidate plan tree is obtained by combining the feature signatures of each operator, for the plan subtree of the target candidate plan tree, the feature signature of the operator contained in the plan subtree can be directly obtained, and then, the feature signatures of the operators contained in the plan subtree can be combined to obtain the sub-feature signature corresponding to the plan subtree. Similarly, after obtaining the sub-feature signature corresponding to each plan subtree, the historical feature signature set in the cost details table can be traversed again according to the sub-feature signature corresponding to each plan subtree, and the running cost of the candidate plan tree can be determined based on the traversal results of the historical feature signature set. Specifically, if the traversal result of the historical feature signature set is a result that the historical feature signature set contains a sub-feature signature of any plan subtree (the sub-feature signature can be called a target sub-feature signature), then the plan subtree corresponding to the target sub-feature signature can be determined as the target plan subtree. Since the target sub-feature signature corresponds to a historical running cost in the cost details table, the operators contained in the target plan subtree can be considered to be operators with known costs; for the remaining operators with unknown costs in the target candidate plan tree, the present application can calculate the operator running cost of each operator based on the traditional method of calculating the running cost.
[0137] In order to understand the traditional cost calculation method, please refer to Figure 6 , Figure 6 Schematic diagram of a traditional cost calculation method provided by the embodiment of the present application. Figure 6As shown in the figure, taking the plan tree consisting of the Project node corresponding to the Project operator, the Filter node corresponding to the Filter operator, and the TableScan node corresponding to the TableScan operator as an example, the traditional cost calculation method needs to aggregate the cost models of each operator from bottom to top in the process of calculating the running cost of the plan tree. For example, for Figure 6 As for the plan tree shown, according to the bottom-up tree level determination method, the tree level of the Project operator in the plan tree is 3, the tree level of the Filter operator in the plan tree is 2, and the tree level of the Table Scan operator in the plan tree is 1. The running cost of the Project operator depends on the running cost of the Filter operator, and the running cost of the Filter operator depends on the running cost of the Table Scan operator. In this way, the running cost of the table scan operator at the bottom leaf node can be determined first. The running cost of the table scan operator at the leaf node depends on the statistical metadata (i.e., statistical information, such as table size, number of table rows, etc.) provided by the external system. Based on the statistical metadata, the cost model of the table scan operator can calculate the running cost of the table scan operator; then, the cost model of the filter operator can calculate the running cost of the filter operator based on the running cost of the table scan operator; the cost model of the project operator can calculate the running cost of the project operator based on the running cost of the filter operator. The running cost of the project operator can be used as the final running cost of the plan tree.
[0138] It should be understood that the plan subtree in the present application is obtained by splitting the target candidate plan tree from bottom to top. Therefore, if the historical feature signature contains the target subfeature signature of a target plan subtree, the operators contained in the target plan subtree should be one or more operators in the target candidate plan tree from the bottom to the top layer by layer. These operators can be used as operators with known costs in the target candidate plan tree. Operators with unknown costs in the target candidate plan tree should be one or more operators at the top position in the target candidate plan tree. For example, Figure 5Taking the plan subtree shown as an example, assuming that plan subtree 1 is the target plan subtree, then the table scan (TableScan) operator can be used as the operator with a known cost in the plan tree, and the remaining filter (Filter) operator and projection (Project) operator at the top position in the plan tree can be used as operators with unknown costs.
[0139] For operators with unknown costs in the target candidate plan tree, this application may refer to them as the remaining operators in the target candidate plan tree, and the operating costs of these remaining operators (which may be referred to as operator operating costs) may be determined according to the traditional cost calculation method. Specifically, the cost raw material information corresponding to the remaining operators may be obtained, wherein the cost raw material information here may refer to the raw material information required to calculate the operating costs of the remaining operators. Since the traditional cost calculation method needs to obtain the operating costs of the operators under a non-leaf node in the process of calculating the operating cost of the operator of the operator, based on this, the cost raw material information corresponding to the remaining operators here should include the operating cost of the operator under the remaining operator (that is, the operator in the candidate plan tree with a tree level lower than the remaining operator, which may be referred to as the cost-dependent operator of the remaining operator) (the operating cost is actually the operating cost of the target plan subtree), that is, the cost raw material information corresponding to the remaining operators will include the historical operating cost corresponding to the target sub-feature signature. Furthermore, based on the cost raw material information corresponding to the remaining operators, the operator running costs corresponding to the remaining operators can be calculated, and the running costs of the target candidate plan tree can be determined according to the operator running costs corresponding to the remaining operators: if there are at least two remaining operators, then according to the traditional cost calculation method, aggregation can be performed from bottom to top until the operator running cost of the top remaining operator in the target candidate plan tree is calculated, and the running cost of the top remaining operator can be used as the running cost of the target candidate plan tree; if there is only one remaining operator, then the remaining operator is the top remaining operator in the target candidate plan tree, and the running cost of the remaining operator can be directly used as the running cost of the target candidate plan tree.
[0140] The above describes the method for calculating the running cost of the target candidate plan tree when the traversal result of the historical feature signature set is the result that the historical feature signature set contains a sub-feature signature of a certain plan subtree. When the traversal result of the historical feature signature set is the result that the historical feature signature set does not contain any sub-feature signature, the running cost of the target candidate plan tree can be calculated in another way. Specifically, if the traversal result of the historical feature signature set is the result that the historical feature signature set does not contain any sub-feature signature, it can be indicated that the historical running cost of the candidate plan tree and the related plan subtree is not recorded in the cost details table. In this case, it is necessary to calculate the operator running cost by bottom-up aggregation of each operator in the target candidate plan tree according to the traditional cost calculation method. The operator running cost of the top operator is the running cost of the target candidate plan tree. The specific implementation process may include but is not limited to: first, the tree level of each operator in the target candidate plan tree in the candidate plan tree can be obtained, and one or more operators in the target candidate plan tree are sorted according to the tree level of each operator (here, they can be sorted in order from low to high tree levels), and an operator sequence corresponding to the target candidate plan tree is obtained (the operator sequence can be called the second operator sequence); then, according to the arrangement order of one or more operators in the second operator sequence, the operator running cost corresponding to each operator can be calculated in turn; the operator at the end position of the sequence in the second operator sequence can be called the sequence end operator, and the operator running cost corresponding to the sequence end operator can be used as the running cost of the target candidate plan tree.
[0141] It is worth noting that after calculating the running cost of the candidate plan tree of the query statement, if the running cost of the candidate plan tree is calculated by bottom-up aggregation according to the traditional cost calculation method (that is, the traversal result of the above-mentioned historical feature signature set is the result that no sub-feature signature is included in the historical feature signature set), then after obtaining the running cost of the candidate plan tree, a corresponding relationship can be established between the feature signature corresponding to the candidate plan tree and the running cost of the candidate plan tree, and the corresponding relationship can be added to the cost details table. In this way, the query statement can be used as a new historical query statement, the candidate plan tree will also be used as a new historical plan tree, the running cost of the candidate plan tree will be used as a new historical running cost, and the feature signature corresponding to the candidate plan tree will be used as a new historical feature signature. In this way, the historical feature signatures and historical running costs in the cost details table can be continuously enriched. When a new query statement to be processed is received subsequently, the running cost of the candidate plan tree of the new query statement can be efficiently and accurately calculated based on the rich and complete cost details table.
[0142] To better illustrate the calculation method of the consideration provided in this application, please see Figure 7 , Figure 7 Schematic diagram of a running cost of a plan tree based on a feature signature calculation provided by an embodiment of the present application. Figure 7 Taking the plan tree shown as an example, for the running cost of this plan tree, we can first traverse the cost details table based on its feature signature to see if there is the same historical feature signature. If so, the historical running cost corresponding to the historical feature signature can be directly determined as the running cost of the plan tree; if not, the sub-feature signatures of each plan subtree can be obtained. If there is a historical feature signature that is the same as a sub-feature signature in the cost details table, then the historical running cost corresponding to the sub-feature signature can be used according to the traditional cost calculation method. The running cost of the plan tree can be obtained by bottom-up aggregation based on the historical running cost corresponding to the plan subtree; if there is no sub-feature signature in the cost details table, then the running cost of the plan tree can be obtained by bottom-up aggregation starting from the leaf node according to the traditional cost calculation method.
[0143] Step S205 , selecting an optimal plan tree from the N candidate plan trees based on the running cost of each candidate plan tree, and submitting the optimal plan tree to the computing engine for execution.
[0144] In this application, in actual applications, there may be multiple (such as N) candidate plan trees for a query statement. By using the cost calculation scheme provided in this application, the running cost of each candidate plan tree can be calculated. After determining the running costs of each candidate plan tree, the minimum running cost (i.e., the minimum running cost) can be selected from the running costs of the N candidate plan trees. Then, the candidate plan tree indicated by the minimum running cost can be determined as the optimal plan tree for the query statement. After that, the optimal plan tree can be submitted to the corresponding computing engine, and the computing engine executes the optimal plan tree to complete the query task indicated by the query statement and obtain the query result.
[0145] In an embodiment of the present application, a method for calculating the running cost of a plan tree is provided, which can improve the accuracy of the calculated running cost and reduce the resource consumption of calculating the running cost. Specifically, after obtaining a candidate plan tree of a query statement, the feature signature corresponding to the candidate plan tree can be obtained, wherein the feature signature in the present application can fully describe the feature information of the query statement, and the corresponding plan tree can be associated and identified based on the feature signature; through the correspondence between each historical feature signature and the historical running cost in the cost details table, the running cost of the feature signature corresponding to the candidate plan tree can be calculated and matched, thereby obtaining the running cost of the candidate plan tree. It should be understood that the present application determines a historical feature signature for each historical plan tree, and calculates and records the historical running cost corresponding to each historical feature signature. This allows, when one wants to calculate the running cost of a candidate plan tree for a query statement, to directly calculate and match the running cost of the candidate plan tree based on the feature signature corresponding to the candidate plan tree and the historical running costs of each historical feature signature. Compared to cost estimation based on the traditional bottom-up aggregation based on the cost model, the present application can directly generate a feature signature based on the current candidate plan tree, and accurately match the accurate cost information from the cost details table, without having to calculate the cost of each operator layer by layer, and without having to obtain additional statistics. The calculation information is more efficient, and the obtained running cost is more complete and accurate; in addition, because different query statements can correspond to the same plan tree or plan subtree in actual applications, the differentiated information of different computing engines can be processed into a unified feature signature by using feature signatures as the basis for running cost calculation, which can realize cross-engine information sharing (such as sharing historical running costs). In this way, when calculating the running cost of a plan tree, fast and efficient cost calculation can be achieved based on the shared existing historical running costs. This can not only improve the cost calculation efficiency, but also reduce the additional computing consumption incurred during cost calculation, thereby reducing resource consumption during calculation. In summary, the present application can expand the cost model in the calculation business of the running cost of the plan tree, support the cost acquisition of the plan subtree based on the plan tree, improve the calculation accuracy, and reduce the consumption of computing resources.
[0146] Further, in order to facilitate understanding of the process of generating a feature signature for a plan tree proposed in this application, the process will be described below in conjunction with the accompanying drawings. Figure 8 , Figure 8 is a schematic diagram of a process for generating a feature signature for a candidate plan tree provided by an embodiment of the present application, wherein the process may correspond to the above Figure 2 In the corresponding embodiment, the characteristic signature calculation process is performed on each candidate plan tree to obtain the characteristic signature corresponding to each candidate plan tree, and the process is described by taking each candidate plan tree as composed of one or more operators. Figure 8 As shown, the process may include at least the following steps S601 to S605:
[0147] Step S601: determine any one of the N candidate plan trees as a target candidate plan tree.
[0148] Specifically, the present application can take any candidate plan tree as an example to illustrate the specific process of calculating and processing the feature signature of the candidate plan tree to obtain the feature signature corresponding to the candidate plan tree. For the convenience of distinction, the any candidate plan tree can be referred to as the target candidate plan tree.
[0149] Step S602: obtaining the tree level of each operator in the target candidate plan tree, and sorting one or more operators in the target candidate plan tree according to the tree level of each operator to obtain a first operator sequence.
[0150] Specifically, a plan tree contains multiple layers, one layer can correspond to one level, and the level corresponding to the bottom layer can be the first level. The tree level of the operator in the candidate plan tree can be calculated and determined from the bottom up. For example, if an operator is at the bottom of the candidate plan tree, at the bottom layer, and is a leaf node of the candidate plan tree, then the tree level of the operator in the candidate plan tree can be 1 (i.e., the first level); for another example, if an operator is the parent node of a leaf node in the candidate plan tree and is at the upper layer of the bottom layer, then the tree level of the operator in the candidate plan tree can be 2 (i.e., the second level). Here, the tree level of each operator in the target candidate plan tree can be obtained, and then one or more operators can be sorted according to the tree level of each operator. Here, they can be sorted in order from high to low in tree level, thereby obtaining an operator sequence, which can be called the first operator sequence.
[0151] Step S603: perform operator signature calculation on each operator in the first operator sequence to obtain an operator signature corresponding to each operator.
[0152] Specifically, for the candidate plan tree, the present application can recursively calculate the characteristic signature (also called operator signature) of each operator from top to bottom. Therefore, after obtaining the first operator sequence, the operator signature of each operator can be calculated in sequence according to the arrangement order of each operator in the first operator sequence. Figure 3 Taking the plan tree in the corresponding embodiment as an example, the operator signature is recursively calculated from top to bottom, and the operator signature corresponding to the Project operator is generated first, followed by the operator signature corresponding to the Filter operator, and finally the operator signature corresponding to the TableScan operator.
[0153] Here, taking any operator as an example, the specific implementation process of performing operator signature calculation on each operator in the first operator sequence to obtain the operator signature corresponding to each operator may at least include but is not limited to: first, any operator in the first operator sequence may be determined as the target operator; then, feature interpretation processing may be performed on the target operator to obtain the interpretation feature field of the target operator, wherein the feature interpretation processing here may refer to explaining the functions and effects implemented by the operator, and each operator has its specific functions and effects, for example: the scanning operator is responsible for reading data from the data source; the filtering operator filters the data according to conditions; the aggregation operator is responsible for aggregating and designing the data; and the connection operator is used to connect the data of different tables or views according to certain conditions, etc. Through the feature interpretation processing, the data operations performed by each operator and the functions and effects implemented can be clarified, and the interpretation field of the operator can be obtained through the feature interpretation processing, and this application may refer to this field as the interpretation feature field of the operator.
[0154] For ease of understanding, the following will be described in conjunction with the accompanying drawings. Fig. 9 Schematic diagram of a feature interpretation process for an operator provided by an embodiment of the present application. Taking the query statement "select s.name, s.age fromstudent s where s.name='Bob'" as an example, after AST analysis, logical conversion and other processing, a candidate plan tree of the query statement can be obtained as follows: Fig. 9As shown, the candidate plan tree may include a database table scan (JdbcTableScan) operator, a database projection (JdbcProject) operator, a database filter (JdbcFliter) operator, and a database projection (JdbcProject) operator from bottom to top. According to the bottom-up tree level calculation method, the tree level of each operator in the candidate plan tree may be 1, 2, 3, and 4. After sorting from high to low, the first operator sequence obtained is {JdbcProject operator, JdbcFliter operator, JdbcProject operator, JdbcTableScan operator}. According to the arrangement order of the first operator sequence, each operator may be processed with feature interpretation in turn to obtain the feature interpretation field of each operator. Specifically, first, based on the database projection (JdbcProject) operator at the starting position of the sequence, the interpretation feature field 1 can be interpreted as: "JdbcProject(name=[name#0],age=[age#1])", based on the database filtering (JdbcFliter) operator, the interpretation feature field 2 can be interpreted as: "JdbcFilter(condition=[=(name#0,'$LITERAL')])"; based on the second database projection (JdbcProject) operator in the sequence, the interpretation feature field 3 can be interpreted as: "JdbcProject(name=[name#1],age=[age#2])"; based on the database table scanning (JdbcTableScan) operator in the sequence, the interpretation feature field 4 can be interpreted as: "JdbcTableScan(table=[[oms,student]])".
[0155] Further, after obtaining the interpretation feature fields of each operator in the candidate plan tree, the interpretation feature fields of each operator can be processed in sequence according to the arrangement order of the first operator sequence to obtain the operator signature of the operator. Taking the target operator as an example, the interpretation feature fields of the target operator can be processed to obtain the operator signature of the target operator. The specific implementation process can at least include but is not limited to: First, according to the operator logic of the target operator, the interpretation feature fields of the target operator can be logically classified to obtain the logical classification fields of the target operator. It should be understood that in the database, there are various operators. Although some operators have different physical properties, their logic is the same. For example, the physical properties of the logical join (LogicalJoin) operator, the database join (JdbcJoin) operator and the enumerated hash join (EnumerableHashJoin) operator are different, but the logic of these three operators is Join logic. Based on this, the present application can pre-set multiple operator categories based on the logic of different operators. In this way, after obtaining the candidate plan tree of the query statement to be processed, it can be classified according to the logic of each operator. Fig. 9 For example, for the bottom-level database table scan (JdbcTableScan) operator in the candidate plan tree, the logic it belongs to is TableScan logic, so the JdbcTableScan operator can be classified into the TableScan operator category. In this application, the interpretation feature fields of any operator can be logically classified according to the operator logic to which the operator belongs. For example, Fig. 9Taking the topmost JdbcProject operator in the candidate plan tree as an example, the operator logic of the JdbcProject operator is Project (projection) logic, then the interpretation feature field 1 (JdbcProject(name=[name#0],age=[age#1])) of the JdbcProject operator can be logically classified, and the resulting logical classification field is: Project(name=[name#0],age=[age#1]). It can be seen that in the present application, the interpretation feature field of the target operator is logically classified to obtain the logical classification field of the target operator, which actually means first classifying the target operator according to the operator logic of the target operator to obtain the target logical category (such as the above-mentioned Project category) to which the target operator belongs; then, the first method field (that is, the field used to characterize the logic (such as the JdbcProject field)) in the interpretation feature field of the target operator needs to be replaced with the second method field (such as the Project field) used to characterize the logical method indicated by the target logical category, thereby obtaining the logical classification field of the target operator.
[0156] After logically classifying the interpretation feature fields to obtain logical classification fields, the logical classification fields of the target operator can be standardized based on the field standard format to obtain the standard interpretation fields of the target operator. It should be understood that in different systems, the fields used to represent the same content in the query statement may be different. For example, in a certain system, the referenced field name is represented by the label "$1" by default, while in another system, the referenced field name is represented by other labels. In order to facilitate cross-system information sharing, the present application can set a field standard format, and convert the fields that do not conform to the field standard format (which can be called non-standard fields) into fields that conform to the field standard format. For example, for the referenced field name, the present application can set the field standard format to the specific content parameter referenced by the application field name (for example, the content parameter referenced by the label "$1" is "name", then the referenced field name can be converted to the content parameter "name"). In this way, even if the referenced field name has a different label, the specific content parameter referenced by it can be used to clarify the specific content applied by the referenced field name. For example, a certain system uses the "age" field to represent age, while another system uses "s.age" to represent age. The two fields are different, one without an alias and the other with an alias "s." Based on this, this application can set the field standard format to "alias ignore format", so that the field with an alias can be converted to a field without an alias. In other words, the field standard format in this application can include standard formats corresponding to different categories of fields, and the field standard format can be specifically set according to actual business needs. This application is only used as an example for illustration. In summary, the specific implementation process of the present application for standardizing the logical classification interpretation field of the target operator based on the field standard format to obtain the standard interpretation field of the target operator may at least include but is not limited to: obtaining non-standard fields (such as the above-mentioned fields with aliases) whose field formats do not conform to the field standard format from the logical classification field of the target operator; then, the non-standard fields may be subjected to field conversion processing according to the field standard format to obtain standard fields; wherein the field format of the standard field conforms to the field standard format; finally, the field composed of the standard field and the remaining fields (the remaining fields refer to the fields in the logical classification field of the target operator other than the standard fields) may be determined as the standard interpretation field of the target operator. Here, when combining the standard field with the remaining fields, the standard field may be specifically placed back to the position where the non-standard field is located, that is, the non-standard field may be replaced with the standard field.
[0157] It can be seen that the feature processing in this application mainly includes two steps: 1. Logically classify the interpretation feature fields to obtain logical classification fields; 2. Standardize the logical classification fields to obtain standard interpretation fields. Feature processing is actually to convert the interpretation feature fields of the operator to fields that conform to the field standard format and have universality. Through feature processing, the final standard interpretation field can be more concise and universal to support cross-system use.
[0158] After the standard interpretation field of the target operator is determined in the above manner, the standard interpretation field of the target operator can be determined as the operator signature of the target operator.
[0159] Step S604: sort the operator signatures corresponding to the one or more operators respectively according to the arrangement order of the one or more operators in the first operator sequence to obtain an operator signature sequence.
[0160] Specifically, according to the above method of determining the operator signature of the target operator, the operator signature of each operator in the first operator sequence can be calculated. After obtaining the operator signature of each operator in the first operator sequence, the operator signatures corresponding to the N operators can be sorted according to the arrangement order of the N operators in the first operator sequence, thereby obtaining an operator signature sequence. Figure 5 Taking the plan tree shown as an example, assuming that the operator signature of the projection (Project) operator is "Project(name=[name#1],age=[age#2])", the operator signature of the filter (Fliter) operator is "Filter(condition=[=(name#1,'Bob')])", and the operator signature of the table scan (TableScan) operator is "TableScan(table=[[oms,student]])", then the operator signature sequence can be {Project(name=[name#1],age=[age#2]), Filter(condition=[=(name#1,'Bob')]), TableScan(table=[[oms,student]])}.
[0161] Step S605: determine the operator signature sequence as the feature signature corresponding to the target candidate plan tree.
[0162] Specifically, the operator signature sequence obtained above can be determined as the feature signature corresponding to the target candidate plan tree. Based on the operator signature of each operator, the sub-feature signature of each plan subtree can also be obtained. Figure 5Taking the plan tree shown as an example, assuming that the operator signature of the Project operator is "Project(name=[name#1],age=[age#2])", the operator signature of the Filter operator is "Filter(condition=[=(name#1,'Bob')])", and the operator signature of the TableScan operator is "TableScan(table=[[oms,student]])", then the sub-feature signature of plan subtree 1 can be {TableScan(table=[[oms,student]])}; the sub-feature signature of plan subtree 2 can be {Filter(condition=[=(name#1,'Bob')])", TableScan(table=[[oms,student]])}.
[0163] In order to more clearly understand the feature signature corresponding to the calculation plan tree (such as the candidate plan tree), the following will briefly explain the logical process of generating the feature signature corresponding to the plan tree with the help of the attached figure. Fig.10 , Fig.10 This is a logical flow diagram of a feature signature corresponding to a plan tree provided by an embodiment of the present application. The logical flow can be implemented by at least the following components: an operator interpretation component, an interpretation feature output component, a feature processing component, and a feature assembly component. The functions implemented by each component are described below:
[0164] Operator interpretation component: After receiving the plan tree, the operator interpretation component can perform feature interpretation processing on each operator in a top-down order to obtain the feature interpretation field of each operator.
[0165] Interpretation feature output component: The interpretation feature output component is used to output the feature interpretation fields of each operator.
[0166] Feature processing component: The feature processing component is used to perform feature processing on the feature interpretation field of each operator to obtain the operator signature of each operator.
[0167] Feature assembly component: The feature assembly component can assemble the operator signatures of each operator in a top-down order (for example, concatenate the operator signatures of each operator in sequence), thereby obtaining the feature signature corresponding to the plan tree.
[0168] In an embodiment of the present application, by determining a historical feature signature for each historical plan tree, and calculating and recording the historical running cost corresponding to each historical feature signature, it is possible to calculate the running cost of a candidate plan tree for a query statement based on the feature signature corresponding to the candidate plan tree and the historical running costs of each historical feature signature to directly calculate and match the running cost of the candidate plan tree. Compared with the cost estimation based on the traditional bottom-up aggregation based on the cost model, the present application can directly generate a feature signature based on the current candidate plan tree, and accurately match the accurate cost information from the cost details table, without calculating the cost of each operator layer by layer, and without additionally obtaining Statistical information is more efficient, and the resulting running cost is more complete and accurate. In addition, in actual applications, different query statements can correspond to the same plan tree or plan subtree. Therefore, by using feature signatures as the basis for running cost calculation, the differentiated information of different computing engines can be processed into a unified feature signature, which can realize cross-engine information sharing (such as sharing historical running costs). In this way, when calculating the running cost of a plan tree, fast and efficient cost calculation can be achieved based on the shared existing historical running costs. This not only improves the cost calculation efficiency, but also reduces the additional computing consumption incurred during cost calculation, thereby reducing resource consumption during calculation. In summary, compared with the traditional method of calculating the cost of each operator one by one from the bottom up based on the cost model, the present application can directly obtain the running cost of the plan tree / plan subtree based on the feature signature, and the obtained cost is more accurate and complete; at the same time, the feature signature of the present application is processed through feature processing, and the feature signature is used as the cost query identifier to support SQL sharing across systems and improve universality; in addition, the unified cost processing based on historical execution behavior can adapt to the running cost information of different engines, and can provide cost optimization based on execution behavior for different engines. Moreover, the present application does not need to trigger additional computing tasks, which can reduce the calculation cost of the cost.
[0169] For further information, see Fig.11 , Fig.11 1 is a schematic diagram of the structure of a query processing device provided in an embodiment of the present application. The query processing device may be a computer program (including program code) running in a computer device, for example, the query processing device is an application software; the query processing device may be used to execute Figure 3 As shown in the method. Fig.11 As shown, the query processing device 1 may include: a plan tree generating module 11, a signature acquiring module 12, a table acquiring module 13, a cost calculating module 14 and a plan tree selecting module 15.
[0170] The plan tree generation module 11 is used to parse the received query statement and generate N candidate plan trees of the query statement; N is a positive integer;
[0171] The signature acquisition module 12 is used to perform feature signature calculation processing on each candidate plan tree to obtain a feature signature corresponding to each candidate plan tree; the feature signature corresponding to a candidate plan tree is used to reflect the query logic of the candidate plan tree;
[0172] Table acquisition module 13, used to obtain a cost details table; the cost details table contains the correspondence between the historical feature signature set and the historical running cost set, there is a correspondence between a historical feature signature in the historical feature signature set and a historical running cost in the historical running cost set, and a historical feature signature refers to a feature signature corresponding to a historical plan tree of a historical query statement;
[0173] The cost calculation module 14 is used to calculate the running cost of each candidate plan tree through the feature signature corresponding to each candidate plan tree, the correspondence between the historical feature signature set and the historical running cost set in the cost details table;
[0174] The plan tree selection module 15 is used to select the best plan tree from N candidate plan trees based on the running cost of each candidate plan tree, and submit the best plan tree to the calculation engine for execution.
[0175] The specific implementation of the plan tree generation module 11, the signature acquisition module 12, the table acquisition module 13, the cost calculation module 14 and the plan tree selection module 15 can be found in the above Figure 2 The description of steps S201 to S205 in the corresponding embodiment will not be repeated here.
[0176] In one embodiment, each candidate plan tree is composed of one or more operators;
[0177] The signature acquisition module 12 performs feature signature calculation processing on each candidate plan tree to obtain the specific implementation method of the feature signature corresponding to each candidate plan tree, including:
[0178] Determine any one of the N candidate plan trees as a target candidate plan tree;
[0179] Obtaining a tree level of each operator in the target candidate plan tree, and sorting one or more operators in the target candidate plan tree according to the tree level of each operator to obtain a first operator sequence;
[0180] Perform operator signature calculation on each operator in the first operator sequence to obtain an operator signature corresponding to each operator;
[0181] According to the arrangement order of the one or more operators in the first operator sequence, the operator signatures corresponding to the one or more operators are sorted to obtain an operator signature sequence;
[0182] The operator signature sequence is determined as the feature signature corresponding to the target candidate plan tree.
[0183] In one embodiment, the signature acquisition module 12 performs operator signature calculation on each operator in the first operator sequence to obtain the specific implementation of the operator signature corresponding to each operator, including:
[0184] Determine any one operator in the first operator sequence as a target operator;
[0185] Perform feature interpretation processing on the target operator to obtain the interpretation feature field of the target operator;
[0186] The interpretation feature field of the target operator is processed to obtain the operator signature of the target operator.
[0187] In one embodiment, the signature acquisition module 12 performs feature processing on the interpretation feature field of the target operator to obtain the operator feature signature of the target operator, and the specific implementation method includes:
[0188] According to the operator logic of the target operator, the interpretation feature field of the target operator is logically classified to obtain the logical classification field of the target operator;
[0189] Based on the standard format of the field, the logical classification fields of the target operator are standardized to obtain the standard interpretation fields of the target operator;
[0190] The standard interpretation field of the target operator is determined as the operator signature of the target operator.
[0191] In one embodiment, the signature acquisition module 12 performs logical classification processing on the interpretation feature field of the target operator according to the operator logic of the target operator to obtain a specific implementation method of the logical classification field of the target operator, including:
[0192] Classify the target operator according to the operator logic of the target operator to obtain the target logic category to which the target operator belongs;
[0193] The first method field in the interpretation feature field of the target operator is replaced with the second method field used to characterize the logical method indicated by the target logical category, thereby obtaining the logical classification field of the target operator.
[0194] In one embodiment, the signature acquisition module 12 performs standardization processing on the logical classification interpretation field of the target operator based on the field standard format to obtain a specific implementation method of the standard interpretation field of the target operator, including:
[0195] Obtain non-standard fields whose field formats do not conform to the field standard format from the logical classification fields of the target operator;
[0196] According to the field standard format, the non-standard field is converted into a standard field; the field format of the standard field conforms to the field standard format;
[0197] The field composed of the standard field and the remaining fields is determined as the standard interpretation field of the target operator; the remaining fields refer to the fields other than the standard fields in the logical classification fields of the target operator.
[0198] In one embodiment, the specific implementation of the plan tree generation module 11 parsing the received query statement and generating N candidate plan trees for the query statement includes:
[0199] Receive query statements to be processed;
[0200] Performing syntax parsing on the query statement to obtain a syntax parsing tree of the query statement;
[0201] Perform logical transformation on the syntax parse tree to generate N candidate plan trees for the query statement.
[0202] In one embodiment, the cost calculation module 14 calculates the operation cost of each candidate plan tree by using the feature signature corresponding to each candidate plan tree, the correspondence between the historical feature signature set and the historical operation cost set in the cost details table, including:
[0203] Determine any one of the N candidate plan trees as a target candidate plan tree;
[0204] Based on the feature signature corresponding to the target candidate plan tree, traverse the historical feature signature set in the cost details table;
[0205] If there is a target historical feature signature in the historical feature signature set that is the same as the feature signature corresponding to the target candidate plan tree, the historical running cost corresponding to the target historical feature signature is determined as the running cost of the target candidate plan tree;
[0206] If there is no target historical feature signature in the historical feature signature set that is identical to the feature signature corresponding to the target candidate plan tree, the target candidate plan tree is split into Q plan subtrees, and the running cost of the target candidate plan tree is calculated based on the correspondence between the Q plan subtrees, the historical feature signature set in the cost details table and the historical running cost set; Q is a positive integer.
[0207] In one embodiment, the cost calculation module 14 calculates the operation cost of the target candidate plan tree according to the correspondence between the Q plan subtrees, the historical feature signature set and the historical operation cost set in the cost details table, including:
[0208] Get the sub-feature signature corresponding to each plan subtree;
[0209] Traverse the historical feature signature set based on the sub-feature signature corresponding to each plan subtree;
[0210] According to the traversal results of the historical feature signature set, the running cost of the target candidate plan tree is determined.
[0211] In one embodiment, the target candidate plan tree is composed of one or more operators; the traversal result of the historical feature signature set is the result of including the target sub-feature signature in the historical feature signature set, and the target sub-feature signature refers to any sub-feature signature among the Q sub-feature signatures;
[0212] The cost calculation module 14 determines the specific implementation method of the operation cost of the target candidate plan tree according to the traversal result of the historical feature signature set, including:
[0213] The plan subtree corresponding to the target sub-feature signature is determined as the target plan subtree, and the operators included in the target plan subtree are determined as the cost-known operators;
[0214] Determine the operators other than the operators with known costs in the target candidate plan tree as the remaining operators, and obtain the cost raw material information corresponding to the remaining operators; the cost raw material information includes the historical running cost corresponding to the target sub-feature signature;
[0215] Calculate the operator running costs corresponding to the remaining operators based on the cost raw material information corresponding to the remaining operators;
[0216] The running cost of the target candidate plan tree is determined according to the operator running costs corresponding to the remaining operators.
[0217] In one embodiment, the target candidate plan tree is composed of one or more operators; the traversal result of the historical feature signature set is the result that the historical feature signature set does not contain any sub-feature signature;
[0218] The cost calculation module 14 determines the specific implementation method of the operation cost of the target candidate plan tree according to the traversal result of the historical feature signature set, including:
[0219] Obtaining the tree level of each operator in the target candidate plan tree, and sorting one or more operators in the target candidate plan tree according to the tree level of each operator to obtain a second operator sequence corresponding to the target candidate plan tree;
[0220] According to the arrangement order of one or more operators in the second operator sequence, the operator running cost corresponding to each operator is calculated in sequence;
[0221] The operator running cost corresponding to the sequence end operator is determined as the running cost of the target candidate plan tree; the sequence end operator refers to the operator at the sequence end position in the second operator sequence.
[0222] In one embodiment, the specific implementation of the plan tree selection module 15 selecting the optimal plan tree from N candidate plan trees based on the running cost of each candidate plan tree includes:
[0223] Get the minimum running cost from the running costs of N candidate plan trees;
[0224] The candidate plan tree indicated by the minimum running cost is determined as the optimal plan tree of the query statement.
[0225] In an embodiment of the present application, by determining a historical feature signature for each historical plan tree, and calculating and recording the historical running cost corresponding to each historical feature signature, it is possible to calculate the running cost of a candidate plan tree for a query statement based on the feature signature corresponding to the candidate plan tree and the historical running costs of each historical feature signature to directly calculate and match the running cost of the candidate plan tree. Compared with the cost estimation based on the traditional bottom-up aggregation based on the cost model, the present application can directly generate a feature signature based on the current candidate plan tree, and accurately match the accurate cost information from the cost details table, without calculating the cost of each operator layer by layer, and without additionally obtaining Statistical information is more efficient, and the resulting running cost is more complete and accurate. In addition, in actual applications, different query statements can correspond to the same plan tree or plan subtree. Therefore, by using feature signatures as the basis for running cost calculation, the differentiated information of different computing engines can be processed into a unified feature signature, which can realize cross-engine information sharing (such as sharing historical running costs). In this way, when calculating the running cost of a plan tree, fast and efficient cost calculation can be achieved based on the shared existing historical running costs. This not only improves the cost calculation efficiency, but also reduces the additional computing consumption incurred during cost calculation, thereby reducing resource consumption during calculation.
[0226] For further information, see Fig.12 , Fig.12 Schematic diagram of the structure of a computer device provided in an embodiment of the present application. Fig.12As shown, the above-mentioned computer device 8000 may include: a processor 8001, a network interface 8004 and a memory 8005. In addition, the above-mentioned computer device 8000 also includes: a user interface 8003, and at least one communication bus 8002. Among them, the communication bus 8002 is used to realize the connection and communication between these components. Among them, the user interface 8003 may include a display screen (Display), a keyboard (Keyboard), and the user interface 8003 may optionally include a standard wired interface and a wireless interface. The network interface 8004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 8005 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 8005 may optionally also be at least one storage device located away from the aforementioned processor 8001. As Fig.12 As shown, the memory 8005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application.
[0227] exist Fig.12 In the computer device 8000 shown, the network interface 8004 can provide a network communication function; the user interface 8003 is mainly used to provide an input interface for the user; and the processor 8001 can be used to call the device control application stored in the memory 8005 to achieve:
[0228] Parse the received query statement and generate N candidate plan trees for the query statement; N is a positive integer;
[0229] Perform feature signature calculation on each candidate plan tree to obtain a feature signature corresponding to each candidate plan tree; a feature signature corresponding to a candidate plan tree is used to reflect the query logic of the candidate plan tree;
[0230] Obtain a cost details table; the cost details table contains a correspondence between a historical feature signature set and a historical running cost set, there is a correspondence between a historical feature signature in the historical feature signature set and a historical running cost in the historical running cost set, and a historical feature signature refers to a feature signature corresponding to a historical plan tree of a historical query statement;
[0231] Calculate the running cost of each candidate plan tree through the corresponding relationship between the feature signature corresponding to each candidate plan tree, the historical feature signature set in the cost details table and the historical running cost set;
[0232] The optimal plan tree is selected from N candidate plan trees based on the running cost of each candidate plan tree, and the optimal plan tree is submitted to the calculation engine for execution.
[0233] In one embodiment, each candidate plan tree is composed of one or more operators;
[0234] When the processor 8001 performs feature signature calculation processing on each candidate plan tree to obtain the feature signature corresponding to each candidate plan tree, the processor 8001 specifically performs the following steps:
[0235] Determine any one of the N candidate plan trees as a target candidate plan tree;
[0236] Obtaining a tree level of each operator in the target candidate plan tree, and sorting one or more operators in the target candidate plan tree according to the tree level of each operator to obtain a first operator sequence;
[0237] Perform operator signature calculation on each operator in the first operator sequence to obtain an operator signature corresponding to each operator;
[0238] According to the arrangement order of the one or more operators in the first operator sequence, the operator signatures corresponding to the one or more operators are sorted to obtain an operator signature sequence;
[0239] The operator signature sequence is determined as the feature signature corresponding to the target candidate plan tree.
[0240] In one embodiment, when the processor 8001 performs operator signature calculation on each operator in the first operator sequence to obtain the operator signature corresponding to each operator, the processor 8001 specifically performs the following steps:
[0241] Determine any one operator in the first operator sequence as a target operator;
[0242] Perform feature interpretation processing on the target operator to obtain the interpretation feature field of the target operator;
[0243] The interpretation feature field of the target operator is processed to obtain the operator signature of the target operator.
[0244] In one embodiment, when the processor 8001 performs feature processing on the interpretation feature field of the target operator to obtain the operator feature signature of the target operator, the processor 8001 specifically performs the following steps:
[0245] According to the operator logic of the target operator, the interpretation feature field of the target operator is logically classified to obtain the logical classification field of the target operator;
[0246] Based on the standard format of the field, the logical classification fields of the target operator are standardized to obtain the standard interpretation fields of the target operator;
[0247] The standard interpretation field of the target operator is determined as the operator signature of the target operator.
[0248] In one embodiment, when the processor 8001 performs logical classification processing on the interpretation feature field of the target operator according to the operator logic of the target operator to obtain the logical classification field of the target operator, the processor 8001 specifically performs the following steps:
[0249] Classify the target operator according to the operator logic of the target operator to obtain the target logic category to which the target operator belongs;
[0250] The first method field in the interpretation feature field of the target operator is replaced with the second method field used to characterize the logical method indicated by the target logical category, thereby obtaining the logical classification field of the target operator.
[0251] In one embodiment, when the processor 8001 performs standardization processing on the logical classification interpretation field of the target operator based on the field standard format to obtain the standard interpretation field of the target operator, the processor 8001 specifically performs the following steps:
[0252] Obtain non-standard fields whose field formats do not conform to the field standard format from the logical classification fields of the target operator;
[0253] According to the field standard format, the non-standard field is converted into a standard field; the field format of the standard field conforms to the field standard format;
[0254] The field composed of the standard field and the remaining fields is determined as the standard interpretation field of the target operator; the remaining fields refer to the fields other than the standard fields in the logical classification fields of the target operator.
[0255] In one embodiment, when the processor 8001 executes the plan tree generation module to parse the received query statement and generate N candidate plan trees for the query statement, the processor 8001 specifically performs the following steps:
[0256] Receive query statements to be processed;
[0257] Performing syntax parsing on the query statement to obtain a syntax parsing tree of the query statement;
[0258] Perform logical transformation on the syntax parse tree to generate N candidate plan trees for the query statement.
[0259] In one embodiment, when the processor 8001 calculates the running cost of each candidate plan tree by executing the correspondence between the feature signature corresponding to each candidate plan tree, the historical feature signature set in the cost details table and the historical running cost set, it specifically performs the following steps:
[0260] Determine any one of the N candidate plan trees as a target candidate plan tree;
[0261] Based on the feature signature corresponding to the target candidate plan tree, traverse the historical feature signature set in the cost details table;
[0262] If there is a target historical feature signature in the historical feature signature set that is the same as the feature signature corresponding to the target candidate plan tree, the historical running cost corresponding to the target historical feature signature is determined as the running cost of the target candidate plan tree;
[0263] If there is no target historical feature signature in the historical feature signature set that is identical to the feature signature corresponding to the target candidate plan tree, the target candidate plan tree is split into Q plan subtrees, and the running cost of the target candidate plan tree is calculated based on the correspondence between the Q plan subtrees, the historical feature signature set in the cost details table and the historical running cost set; Q is a positive integer.
[0264] In one embodiment, when the processor 8001 calculates the running cost of the target candidate plan tree according to the correspondence between the historical feature signature set and the historical running cost set in the cost details table, the processor 8001 specifically performs the following steps:
[0265] Get the sub-feature signature corresponding to each plan subtree;
[0266] Traverse the historical feature signature set based on the sub-feature signature corresponding to each plan subtree;
[0267] According to the traversal results of the historical feature signature set, the running cost of the target candidate plan tree is determined.
[0268] In one embodiment, the target candidate plan tree is composed of one or more operators; the traversal result of the historical feature signature set is the result of including the target sub-feature signature in the historical feature signature set, and the target sub-feature signature refers to any sub-feature signature among the Q sub-feature signatures;
[0269] When the processor 8001 determines the running cost of the target candidate plan tree according to the traversal result of the historical feature signature set, it specifically performs the following steps:
[0270] The plan subtree corresponding to the target sub-feature signature is determined as the target plan subtree, and the operators included in the target plan subtree are determined as the cost-known operators;
[0271] Determine the operators other than the operators with known costs in the target candidate plan tree as the remaining operators, and obtain the cost raw material information corresponding to the remaining operators; the cost raw material information includes the historical running cost corresponding to the target sub-feature signature;
[0272] Calculate the operator running costs corresponding to the remaining operators based on the cost raw material information corresponding to the remaining operators;
[0273] The running cost of the target candidate plan tree is determined according to the operator running costs corresponding to the remaining operators.
[0274] In one embodiment, the target candidate plan tree is composed of one or more operators; the traversal result of the historical feature signature set is the result that the historical feature signature set does not contain any sub-feature signature;
[0275] When the processor 8001 determines the running cost of the target candidate plan tree according to the traversal result of the historical feature signature set, it specifically performs the following steps:
[0276] Obtaining the tree level of each operator in the target candidate plan tree, and sorting one or more operators in the target candidate plan tree according to the tree level of each operator to obtain a second operator sequence corresponding to the target candidate plan tree;
[0277] According to the arrangement order of one or more operators in the second operator sequence, the operator running cost corresponding to each operator is calculated in sequence;
[0278] The operator running cost corresponding to the sequence end operator is determined as the running cost of the target candidate plan tree; the sequence end operator refers to the operator at the sequence end position in the second operator sequence.
[0279] In one embodiment, when the processor 8001 selects the optimal plan tree from N candidate plan trees based on the running cost of each candidate plan tree, the processor 8001 specifically performs the following steps:
[0280] Get the minimum running cost from the running costs of N candidate plan trees;
[0281] The candidate plan tree indicated by the minimum running cost is determined as the optimal plan tree of the query statement.
[0282] It should be understood that the computer device 8000 described in the embodiment of the present application can execute the above Figures 2 to 8 The description of the query processing method in the corresponding embodiment can also be performed as described above. Fig.11 The description of the query processing device 1 in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated here either.
[0283] In addition, it should be pointed out here that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the data processing computer device 8000 mentioned above, and the computer program includes program instructions. When the processor executes the program instructions, the computer program can execute the above-mentioned data processing computer device 8000. Figures 2 to 8The description of the query processing method in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.
[0284] The computer-readable storage medium may be the query processing device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Further, the computer-readable storage medium may also include both the internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.
[0285] In one aspect of the present application, a computer program product is provided, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the method provided in one aspect of the embodiments of the present application.
[0286] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, device, product, or equipment that includes a series of steps or units is not limited to the listed steps or modules, but optionally includes steps or modules that are not listed, or optionally includes other step units inherent to these processes, methods, devices, products, or equipment.
[0287] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.
[0288] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0289] The method and related apparatus provided by the embodiment of the present application are described with reference to the method flow chart and / or structural diagram provided by the embodiment of the present application. Specifically, each process and / or box in the method flow chart and / or structural diagram, as well as the combination of the processes and / or boxes in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A flow or multiple flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.
[0290] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.
Claims
1. A query processing method, characterized in that: The method comprises: Parse the received query statement to generate N candidate plan trees for the query statement; N is a positive integer; each candidate plan tree is composed of one or more operators; Perform feature signature calculation processing on each of the candidate plan trees to obtain a feature signature corresponding to each of the candidate plan trees; the feature signature corresponding to a candidate plan tree is used to reflect the query logic of the candidate plan tree; wherein, determining the feature signature corresponding to any candidate plan tree includes: determining any one of the N candidate plan trees as the target candidate plan tree; obtaining the tree level of each of the operators in the target candidate plan tree in the target candidate plan tree, and sorting one or more operators in the target candidate plan tree according to the tree level of each operator to obtain a first operator sequence; performing operator signature calculation on each of the operators in the first operator sequence to obtain an operator signature corresponding to each operator; sorting the operator signatures corresponding to the one or more operators respectively according to the arrangement order of the one or more operators in the first operator sequence to obtain an operator signature sequence; determining the operator signature sequence as the feature signature corresponding to the target candidate plan tree; Obtain a cost details table; the cost details table contains a correspondence between a historical feature signature set and a historical running cost set, there is a correspondence between a historical feature signature in the historical feature signature set and a historical running cost in the historical running cost set, and a historical feature signature refers to a feature signature corresponding to a historical plan tree of a historical query statement; Calculate the running cost of each candidate plan tree by using the feature signature corresponding to each candidate plan tree, the correspondence between the historical feature signature set in the cost details table and the historical running cost set; An optimal plan tree is selected from the N candidate plan trees based on the running cost of each of the candidate plan trees, and the optimal plan tree is submitted to the computing engine for execution.
2. The method according to claim 1, characterized in that: The performing operator signature calculation on each operator in the first operator sequence to obtain an operator signature corresponding to each operator includes: Determine any one operator in the first operator sequence as a target operator; Performing feature interpretation processing on the target operator to obtain an interpretation feature field of the target operator; The interpretation feature field of the target operator is processed to obtain the operator signature of the target operator.
3. The method according to claim 2, characterized in that The performing feature processing on the interpretation feature field of the target operator to obtain the operator feature signature of the target operator includes: According to the operator logic of the target operator, the interpretation feature field of the target operator is logically classified to obtain the logical classification field of the target operator; Based on the field standard format, the logical classification field of the target operator is standardized to obtain the standard interpretation field of the target operator; The standard interpretation field of the target operator is determined as the operator signature of the target operator.
4. The method according to claim 3, characterized in that The step of performing logical classification processing on the interpretation feature field of the target operator according to the operator logic of the target operator to obtain the logical classification field of the target operator includes: Classifying the target operator according to the operator logic of the target operator to obtain the target logic category to which the target operator belongs; The first method field in the interpretation feature field of the target operator is replaced with the second method field used to characterize the logical method indicated by the target logical category, so as to obtain the logical classification field of the target operator.
5. The method according to claim 3, characterized in that: The step of performing standard processing on the logical classification interpretation field of the target operator based on the field standard format to obtain the standard interpretation field of the target operator includes: Acquire a non-standard field whose field format does not conform to the field standard format from the logical classification field of the target operator; The non-standard field is subjected to field conversion processing according to the field standard format to obtain a standard field; the field format of the standard field conforms to the field standard format; A field composed of the standard field and the remaining fields is determined as a standard interpretation field of the target operator; the remaining fields refer to fields in the logical classification field of the target operator except the non-standard fields.
6. The method according to claim 1, characterized in that The step of parsing the received query statement to generate N candidate plan trees for the query statement includes: Receive query statements to be processed; Performing syntax parsing on the query statement to obtain a syntax parsing tree of the query statement; The syntax parse tree is logically transformed to generate N candidate plan trees for the query statement.
7. The method according to claim 1, characterized in that The calculating the running cost of each candidate plan tree by using the feature signature corresponding to each candidate plan tree, the correspondence between the historical feature signature set in the cost details table and the historical running cost set, comprises: Determine any one of the N candidate plan trees as a target candidate plan tree; Based on the feature signature corresponding to the target candidate plan tree, traverse the historical feature signature set in the cost details table; If there is a target historical feature signature in the historical feature signature set that is the same as the feature signature corresponding to the target candidate plan tree, then the historical running cost corresponding to the target historical feature signature is determined as the running cost of the target candidate plan tree; If there is no target historical feature signature in the historical feature signature set that is identical to the feature signature corresponding to the target candidate plan tree, the target candidate plan tree is split into Q plan subtrees, and the operating cost of the target candidate plan tree is calculated based on the correspondence between the Q plan subtrees, the historical feature signature set in the cost details table and the historical operating cost set; Q is a positive integer.
8. The method according to claim 7, characterized in that The calculating the running cost of the target candidate plan tree according to the correspondence between the Q plan subtrees, the historical feature signature set in the cost details table and the historical running cost set includes: Obtaining a sub-feature signature corresponding to each of the plan subtrees; Traversing the historical feature signature set based on the sub-feature signature corresponding to each of the planned subtrees; The running cost of the target candidate plan tree is determined according to the traversal result of the historical feature signature set.
9. The method according to claim 8, characterized in that The target candidate plan tree is composed of one or more operators; the traversal result of the historical feature signature set is the result of the target sub-feature signature included in the historical feature signature set, and the target sub-feature signature refers to any sub-feature signature among the Q sub-feature signatures; The step of determining the running cost of the target candidate plan tree according to the traversal result of the historical feature signature set includes: Determine the plan subtree corresponding to the target sub-feature signature as the target plan subtree, and determine the operator included in the target plan subtree as the cost known operator; Determine the operators other than the operator with known cost in the target candidate plan tree as remaining operators, and obtain cost raw material information corresponding to the remaining operators; the cost raw material information includes the historical running cost corresponding to the target sub-feature signature; Calculating the operator running costs corresponding to the remaining operators based on the cost raw material information corresponding to the remaining operators; The running cost of the target candidate plan tree is determined according to the operator running costs corresponding to the remaining operators.
10. The method according to claim 8, characterized in that The target candidate plan tree is composed of one or more operators; the traversal result of the historical feature signature set is the result that the historical feature signature set does not contain any sub-feature signature; The step of determining the running cost of the target candidate plan tree according to the traversal result of the historical feature signature set includes: Obtaining a tree level of each operator in the target candidate plan tree, and sorting one or more operators in the target candidate plan tree according to the tree level of each operator to obtain a second operator sequence corresponding to the target candidate plan tree; According to the arrangement order of the one or more operators in the second operator sequence, the operator running cost corresponding to each of the operators is calculated in sequence; The operator running cost corresponding to the sequence end operator is determined as the running cost of the target candidate plan tree; the sequence end operator refers to the operator at the sequence end position in the second operator sequence.
11. The method according to claim 1, characterized in that: Selecting an optimal plan tree from the N candidate plan trees based on the running cost of each of the candidate plan trees includes: Obtaining the minimum running cost from the running costs of the N candidate plan trees; The candidate plan tree indicated by the minimum running cost is determined as the optimal plan tree for the query statement.
12. A query processing device, characterized in that: The device comprises: A plan tree generation module, used to parse the received query statement and generate N candidate plan trees of the query statement; N is a positive integer; each candidate plan tree is composed of one or more operators; A signature acquisition module is used to perform feature signature calculation processing on each of the candidate plan trees to obtain a feature signature corresponding to each of the candidate plan trees; a feature signature corresponding to a candidate plan tree is used to reflect the query logic of the candidate plan tree; wherein determining the feature signature corresponding to any candidate plan tree includes: determining any one of the N candidate plan trees as a target candidate plan tree; obtaining the tree level of each operator in the target candidate plan tree in the target candidate plan tree, and sorting one or more operators in the target candidate plan tree according to the tree level of each operator to obtain a first operator sequence; performing operator signature calculation on each of the operators in the first operator sequence to obtain an operator signature corresponding to each operator; sorting the operator signatures corresponding to the one or more operators respectively according to the arrangement order of the one or more operators in the first operator sequence to obtain an operator signature sequence; determining the operator signature sequence as the feature signature corresponding to the target candidate plan tree; A table acquisition module is used to obtain a cost details table; the cost details table contains a correspondence between a historical feature signature set and a historical running cost set, there is a correspondence between a historical feature signature in the historical feature signature set and a historical running cost in the historical running cost set, and a historical feature signature refers to a feature signature corresponding to a historical plan tree of a historical query statement; A cost calculation module, used to calculate the running cost of each candidate plan tree through the feature signature corresponding to each candidate plan tree, the corresponding relationship between the historical feature signature set in the cost details table and the historical running cost set; The plan tree selection module is used to select the optimal plan tree from the N candidate plan trees based on the running cost of each candidate plan tree, and submit the optimal plan tree to the calculation engine for execution.
13. A computer device, characterized in that: include: Processor, memory, and network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide a network communication function, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method described in any one of claims 1-11.
14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the method according to any one of claims 1 to 11.
15. A computer program product, characterized in that The computer program product comprises a computer program, which is stored in a computer-readable storage medium. The computer program is suitable for being read and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 11.
