Query processing method and device and electronic equipment

Abstract

The invention discloses a query processing method and device and electronic equipment, and relates to the technical field of data processing. The method comprises the steps that a target query statement is received, an original physical execution plan used for query processing is generated according to the target query statement, and a plurality of first physical execution plans are obtained; obtaining first runtime index data of each first physical operator in each first physical execution plan; calculating execution cost corresponding to each first physical execution plan according to the first runtime index data, and determining a target physical execution plan from the plurality of first physical execution plans according to the execution cost; and executing the target physical execution plan to obtain a query result corresponding to the target query statement. The technical problem that the target physical execution plan cannot be determined due to the fact that cost calculation cannot be performed according to the runtime index of the optimized physical execution plan in the prior art is solved.

Description

Technical Field

[0001] This application relates to the technical field of data processing. Specifically, it relates to a query processing method, device, and electronic device. Background Art

[0002] The computing engine Apache Spark provides a Structured Query Language (SQL) development mode, which can convert SQL text into multiple physical execution plans. Cost-based optimization is a query optimization technique that can select a better execution plan from multiple physical execution plans and obtain a query result based on this plan.

[0003] Adaptive Query Execution (AQE) is an important feature introduced by Apache Spark starting from version 3.0. AQE can dynamically optimize physical execution plans according to the runtime metrics of the application and adjust the physical operators in the physical execution plans. However, in the related art, when using AQE to optimize the original physical execution plan, it is impossible to calculate the cost based on the runtime metrics of the optimized physical execution plan, resulting in the inability to determine the target physical execution plan.

[0004] In view of the above problems, no effective solution has been proposed yet. Summary of the Invention

[0005] Embodiments of this application provide a query processing method, device, and electronic device to at least solve the technical problem that in the related art, when using the adaptive query execution method to optimize the original physical execution plan, it is impossible to calculate the cost based on the runtime metrics of the optimized physical execution plan, resulting in the inability to determine the target physical execution plan.

[0006] According to one aspect of the embodiments of the present application, a query processing method is provided, including: receiving a target query statement, and generating an original physical execution plan for query processing based on the target query statement to obtain a plurality of first physical execution plans, wherein each first physical execution plan includes a plurality of first physical operators, and each first physical operator is used to implement query processing; obtaining first runtime metric data of each first physical operator in each first physical execution plan, wherein the first runtime metric data is determined after matching the operator relationship between the first physical operator and a second physical operator, and the second physical operator is a physical operator of a second physical execution plan obtained by optimizing the first physical execution plan according to the adaptive query execution method; calculating the execution cost corresponding to each first physical execution plan based on the first runtime metric data, and determining a target physical execution plan from the plurality of first physical execution plans according to the execution cost; executing the target physical execution plan to obtain a query result corresponding to the target query statement.

[0007] Further, obtaining the first runtime metric data of each first physical operator in each first physical execution plan includes: traversing each first physical execution plan and each second physical execution plan respectively according to the target traversal algorithm, and matching the operator relationship between the first physical operator and the second physical operator according to the target rule during the traversal process to obtain a matching result; determining the first runtime metric data of each first physical operator according to the matching result.

[0008] Further, traversing each first physical execution plan and each second physical execution plan respectively according to the target traversal algorithm includes: for each second physical execution plan, traversing the second physical execution plan according to the target traversal algorithm to determine a second target physical operator from a plurality of second physical operators; for each first physical execution plan, traversing the first physical execution plan based on the second target physical operator according to the target traversal algorithm to determine a first target physical operator from a plurality of first physical operators; using the next physical operator of the first target physical operator in the first physical execution plan as the next physical operator to be matched in the second physical execution plan; traversing the first physical execution plan based on the next physical operator to be matched in the second physical execution plan according to the target traversal algorithm to determine a sub-physical operator of the first target physical operator from the first physical operators other than the first target physical operator.

[0009] Further, during the traversal process, operator relationship matching is performed on the first physical operator and the second physical operator according to the target rule to obtain a matching result, including: obtaining the operator information of the first physical operator and the operator information of the second physical operator; judging the operator information of the first physical operator and the operator information of the second physical operator according to the target rule to obtain a judgment result; if the judgment result indicates that the operator information of the first physical operator and the operator information of the second physical operator meet the target rule, it is determined that the matching result is that the first physical operator and the second physical operator are successfully matched.

[0010] Further, the first runtime index data of each first physical operator is determined according to the matching result, including: for each first physical operator, if the matching result is that the first physical operator and the second physical operator are successfully matched, the operator correspondence between the first physical operator and the second physical operator is determined; the second runtime index data of the second physical operator is obtained, and the second runtime index data is converted according to the target mapping relationship to obtain target index information, where the target mapping relationship is used to represent the correspondence between semantic representation information and runtime indexes; the target index information is assigned to the first physical operator according to the operator correspondence to obtain the first runtime index data of each first physical operator.

[0011] Further, the target index information is assigned to the first physical operator according to the operator correspondence to obtain the first runtime index data of each first physical operator, including: for each first physical operator, the semantic representation information in the target index information is assigned to the key field of the first runtime index of the first physical operator according to the operator correspondence, and the numerical information in the target index information is assigned to the value field of the first runtime index of the first physical operator; the key field and the value field of the first runtime index are combined to form the first runtime index data of the first physical operator.

[0012] Further, the execution cost corresponding to each first physical execution plan is calculated according to the first runtime index data, including: for each first physical execution plan, the runtime index data of multiple sub-physical operators of the first target physical operator in the first physical execution plan is summed to obtain the sum of the initial index data; the sum of the initial index data and the runtime index data of the first target physical operator are summed to obtain the sum of the target index data; the sum of the target index data is used as the execution cost corresponding to the first physical execution plan.

[0013] Further, the target physical execution plan is determined from multiple first physical execution plans according to the execution cost, including: sorting the sums of the target index data corresponding to multiple first physical execution plans to obtain a sorting result; determining the target physical execution plan from multiple first physical execution plans according to the sorting result.

[0014] Further, before receiving the target query statement, the method further includes: obtaining a plurality of runtime metrics, where the plurality of runtime metrics include a first runtime metric and a second runtime metric; processing the plurality of runtime metrics according to the semantic representation information of the runtime metrics to obtain a target mapping relationship.

[0015] According to another aspect of the embodiments of the present application, there is also provided a query processing method, including: obtaining a target query statement uploaded by a client; generating an original physical execution plan for query processing in a cloud server to obtain a plurality of first physical execution plans, where each first physical execution plan includes a plurality of first physical operators, and each first physical operator is used to implement query processing; obtaining first runtime metric data of each first physical operator in each first physical execution plan, where the first runtime metric data is determined after operator relationship matching between the first physical operator and a second physical operator, and the second physical operator is a physical operator of a second physical execution plan obtained by optimizing the first physical execution plan according to an adaptive query execution method; calculating the execution cost corresponding to each first physical execution plan according to the first runtime metric data, and determining a target physical execution plan from the plurality of first physical execution plans according to the execution cost; executing the target physical execution plan to obtain a query result corresponding to the target query statement; and feeding back the query result corresponding to the target query statement to the client.

[0016] According to another aspect of the embodiments of the present application, there is also provided a query processing device, including: a first receiving unit, configured to receive a target query statement and generate an original physical execution plan for query processing according to the target query statement to obtain a plurality of first physical execution plans, where each first physical execution plan includes a plurality of first physical operators, and each first physical operator is used to implement query processing; a first obtaining unit, configured to obtain first runtime metric data of each first physical operator in each first physical execution plan, where the first runtime metric data is determined after operator relationship matching between the first physical operator and a second physical operator, and the second physical operator is a physical operator of a second physical execution plan obtained by optimizing the first physical execution plan according to an adaptive query execution method; a first determining unit, configured to calculate the execution cost corresponding to each first physical execution plan according to the first runtime metric data, and determine a target physical execution plan from the plurality of first physical execution plans according to the execution cost; and a first processing unit, configured to execute the target physical execution plan to obtain a query result corresponding to the target query statement.

[0017] Further, the first acquisition unit includes: a first processing subunit, configured to traverse each first physical execution plan and each second physical execution plan respectively according to a target traversal algorithm, and perform operator relationship matching between the first physical operator and the second physical operator according to a target rule during the traversal process to obtain a matching result; a first determination subunit, configured to determine the first runtime metric data of each first physical operator according to the matching result.

[0018] Further, the first processing subunit includes: a first determination module, configured to traverse each second physical execution plan according to the target traversal algorithm to determine a second target physical operator from multiple second physical operators for each second physical execution plan; a second determination module, configured to traverse each first physical execution plan according to the target traversal algorithm based on the second target physical operator to determine a first target physical operator from multiple first physical operators for each first physical execution plan; a third determination module, configured to use the next physical operator of the first target physical operator in the first physical execution plan as the next physical operator to be matched in the second physical execution plan; a fourth determination module, configured to traverse the first physical execution plan according to the target traversal algorithm based on the next physical operator to be matched in the second physical execution plan to determine a sub-physical operator of the first target physical operator from the first physical operators other than the first target physical operator.

[0019] Further, the first processing subunit further includes: a first acquisition module, configured to acquire the operator information of the first physical operator and the operator information of the second physical operator; a first judgment module, configured to judge the operator information of the first physical operator and the operator information of the second physical operator according to the target rule to obtain a judgment result; a fifth determination module, configured to determine that the matching result is that the first physical operator and the second physical operator are successfully matched if the judgment result indicates that the operator information of the first physical operator and the operator information of the second physical operator meet the target rule.

[0020] Further, the first determination subunit includes: a sixth determination module, configured to determine the operator correspondence between the first physical operator and the second physical operator for each first physical operator if the matching result is that the first physical operator and the second physical operator are successfully matched; a first processing module, configured to acquire the second runtime metric data of the second physical operator and convert the second runtime metric data according to the target mapping relationship to obtain target metric information, where the target mapping relationship is used to represent the correspondence between semantic representation information and runtime metrics; a second processing module, configured to assign the target metric information to the first physical operator according to the operator correspondence to obtain the first runtime metric data of each first physical operator.

[0021] Further, the second processing module includes: a first processing sub-module, configured to, for each first physical operator, assign the semantic representation information in the target metric information to the key field of the first runtime metric of the first physical operator according to the operator correspondence, and assign the numerical information in the target metric information to the value field of the first runtime metric of the first physical operator; a second processing sub-module, configured to form the first runtime metric data of the first physical operator by combining the key field of the first runtime metric and the value field of the first runtime metric.

[0022] Further, the first determination unit includes: a first calculation sub-unit, configured to, for each first physical execution plan, perform a summation calculation on the runtime metric data of multiple sub-physical operators of the first target physical operator in the first physical execution plan to obtain an initial metric data sum; a second calculation sub-unit, configured to perform a summation calculation on the initial metric data sum and the runtime metric data of the first target physical operator to obtain a target metric data sum; a second determination sub-unit, configured to use the target metric data sum as the execution cost corresponding to the first physical execution plan.

[0023] Further, the first determination unit further includes: a sorting sub-unit, configured to sort the target metric data sums corresponding to multiple first physical execution plans to obtain a sorting result; a third determination sub-unit, configured to determine a target physical execution plan from multiple first physical execution plans according to the sorting result.

[0024] Further, the query processing device further includes: a second acquisition unit, configured to acquire multiple runtime metrics before receiving a target query statement, where the multiple runtime metrics include a first runtime metric and a second runtime metric; a second processing unit, configured to process the multiple runtime metrics according to the semantic representation information of the runtime metrics to obtain a target mapping relationship.

[0025] According to another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, where the storage medium stores a program, and when the program runs, it controls the device where the storage medium is located to execute the query processing method described in any one of the above.

[0026] According to another aspect of the embodiments of the present invention, there is also provided an electronic device, including: a memory storing an executable program; a processor configured to run the program, and when the program runs, it executes the query processing method described in any one of the above.

[0027] In an embodiment of the present application, by receiving a target query statement and generating an original physical execution plan for query processing based on the target query statement, a plurality of first physical execution plans are obtained. Each first physical execution plan includes a plurality of first physical operators, and each first physical operator is used to implement query processing. Obtain the first runtime metric data of each first physical operator in each first physical execution plan, where the first runtime metric data is determined after the first physical operator matches the operator relationship with the second physical operator, and the second physical operator is the physical operator of the second physical execution plan obtained by optimizing the first physical execution plan according to the adaptive query execution method. Calculate the execution cost corresponding to each first physical execution plan based on the first runtime metric data, and determine the target physical execution plan from the plurality of first physical execution plans according to the execution cost. Execute the target physical execution plan to obtain the query result corresponding to the target query statement. This solves the technical problem in the related art that when using the adaptive query execution method to optimize the original physical execution plan, it is impossible to calculate the cost based on the runtime metrics of the optimized physical execution plan, resulting in the inability to determine the target physical execution plan. Through the matching of physical operator relationships, the operator relationship between the physical execution plan adjusted by AQE (i.e., the optimized second physical execution plan) and the original physical execution plan (i.e., the first physical execution plan) can be identified, intuitively understanding the operator optimization result, so as to better evaluate the optimization effect and running cost of the original execution plan. By extracting the runtime metric data of the physical operators of the physical execution plan adjusted by AQE and providing it to the original physical execution plan, the problem that the runtime metrics cannot match the original physical execution plan after the physical execution plan is adjusted and optimized by AQE is solved, providing a reliable cost calculation method for the implementation of cost optimization in Apache Spark 3, achieving the purpose of effectively identifying the operator relationship between the physical execution plan adjusted by AQE and the original physical execution plan, and thus realizing the technical effect of being able to calculate the cost based on the runtime metrics of the optimized physical execution plan, and thus being able to determine a better physical execution plan. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The illustrative embodiments and descriptions thereof of the present application are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0029] Figure 1 is a schematic diagram of a computer terminal provided in the first embodiment of the present application;

[0030] Figure 2 is a flowchart of a query processing method provided in the first embodiment of the present application;

[0031] Figure 3It is a schematic flowchart of an optional process for screening a physical execution plan provided in Embodiment 1 of the present application;

[0032] Figure 4 It is a schematic flowchart of an optional cost calculation provided in Embodiment 1 of the present application;

[0033] Figure 5 It is a flowchart of a query processing method provided in Embodiment 2 of the present application;

[0034] Figure 6 It is a schematic diagram of a query processing device provided in Embodiment 3 of the present application;

[0035] Figure 7 It is a schematic diagram of a computing terminal provided in Embodiment 4 of the present application. Detailed implementation manners

[0036] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily need to be limited to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0038] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards in the relevant regions, and corresponding operation entrances are provided for the user to choose to authorize or refuse.

[0039] First, some nouns or terms that appear during the description of the embodiments of the present application are applicable to the following explanations:

[0040] Runtime metrics: Metrics used to measure the performance and running status of an application during program execution, which can provide information about the real-time performance and efficiency of the application.

[0041] Apache Spark physical execution plan: An execution plan defined in an Apache Spark application, which describes the actual execution method of a query or transformation operation written in Spark SQL. The physical execution plan defines the process of converting the logical execution plan into specific physical operations and tasks.

[0042] Embodiment 1

[0043] According to the embodiments of the present application, a query processing method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0044] The method embodiments provided by the first embodiment of the present application can be executed on a mobile terminal, a computer terminal, or a similar computing device. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing the query processing method is shown. As Figure 1 shown, the computer terminal (or mobile device) 10 may include a processor set 102 (the processor set 102 may include, but is not limited to, a processing device such as a microprocessor MCU (Microcontroller Unit) or a field programmable gate array FPGA (Field Programmable Gate Array), and the processor set 102 may include a processor set, Figure 1 which is shown as 102a, 102b,..., 102n in it), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may further include: a display, an input / output interface (I / O interface), a universal serial bus (USB, Universal Serial Bus) port (which can be included as one of the ports of the BUS bus), a network interface, a power supply, and / or a camera. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 may further include more or fewer components than Figure 1 shown in it, or have a different configuration from Figure 1 shown in it.

[0045] It should be noted that one or more of the above-mentioned processors 102 and / or other data processing circuits can generally be referred to as "data processing circuits" herein. The data processing circuit can be embodied in whole or in part as software, hardware, firmware or any combination thereof. In addition, the data processing circuit can be a single independent processing module, or be incorporated in whole or in part into any one of other elements in the computer terminal 10 (or mobile device).

[0046] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage devices corresponding to the query processing method in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the above-mentioned query processing method. The memory 104 can include high-speed random access memory, and can also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory 104 can further include a memory remotely disposed relative to the processor 102, and these remote memories can be connected to the computer terminal 10 through a network. Examples of the above-mentioned network include but are not limited to the Internet, enterprise intranet, local area network, mobile communication network and their combinations.

[0047] The transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network can include the wireless network provided by the communication provider of the computer terminal 10. In one instance, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0048] The display can be, for example, a touch-screen liquid crystal display, which enables the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0049] The computing engine Apache Spark provides an SQL development mode, which can convert SQL text into multiple physical execution plans. Cost-based optimization is a query optimization technique that can select a better execution plan from multiple physical execution plans and obtain the query result according to this plan.

[0050] Adaptive Query Execution (AQE) is an important feature introduced in Apache Spark starting from version 3.0. AQE can dynamically optimize the physical execution plan according to the runtime metrics of the application and adjust the physical operators in the physical execution plan. However, cost optimization has faced challenges after Apache Spark version 3.0. The difficulty lies in that since AQE adjusts and optimizes the physical execution plan, the runtime metrics no longer match the original physical execution plan. As a result, it is impossible to calculate the cost based on the runtime metrics of the optimized physical execution plan, making it impossible to determine a better physical execution plan.

[0051] Against the above technical background, this application provides a query processing method as Figure 2 shown. Figure 2 is a flowchart of the query processing method provided in the first embodiment of this application. The method includes:

[0052] Step S201: Receive a target query statement, and generate an original physical execution plan for query processing based on the target query statement, obtaining multiple first physical execution plans. Each first physical execution plan includes multiple first physical operators, and each first physical operator is used to implement query processing.

[0053] Optionally, the target query statement can be an SQL statement written based on Spark SQL. Based on the target query statement, an original physical execution plan for query processing can be generated, that is, multiple first physical execution plans are obtained. That is, the first physical execution plan is the original physical execution plan, and the first physical operator is the physical operator in the original physical execution plan, simply referred to as the original physical operator. For example, the Apache Spark physical execution plan consists of multiple physical operators, and the physical operators are responsible for executing computing tasks on the cluster.

[0054] Figure 3 is a schematic flowchart of an optional process for screening physical execution plans provided in the first embodiment of this application. As Figure 3 shown, first, the Apache Spark SQL text is converted into an unparsed logical execution plan. Then, Apache Spark uses an analyzer to parse it to obtain a parsed logical execution plan, and uses an optimizer to optimize it to obtain an optimized logical execution plan. Then, the planner is used to convert the optimized logical execution plan into multiple physical execution plans, and a better physical execution plan is selected based on cost optimization. For example, AQE dynamically optimizes the physical execution plan according to runtime metrics and adjusts the physical operators in the physical execution plan. By collecting and using statistical information and runtime metric data, the costs of different physical execution plans can be estimated and compared (i.e., cost calculation is performed), and the execution plan with the lower cost is selected as the target physical execution plan to obtain the screened physical execution plan.

[0055] Among them, AQE can provide the following functions and optimizations:

[0056] (1) Change the Join strategy. AQE can obtain the sizes of the tables participating in the Join calculation based on runtime metrics and re-plan the Join strategy. For example, if the data volume of one of the tables is less than the specified threshold, AQE can adjust the Join operator in the physical execution plan to a broadcast hash Join operator.

[0057] (2) Remove unnecessary sorting. Runtime metrics can collect the number of data rows. For example, if the number of sorted data rows in the physical execution plan is less than or equal to 1, AQE will remove the Sort operator in the physical execution plan to avoid performing unnecessary sorting calculations.

[0058] Step S202: Obtain the first runtime metric data of each first physical operator in each first physical execution plan, where the first runtime metric data is determined after the first physical operator matches the operator relationship with the second physical operator, and the second physical operator is the physical operator of the second physical execution plan obtained by optimizing the first physical execution plan according to the adaptive query execution method.

[0059] The first runtime metric data is the metric data generated during the execution of the physical operator. For example, the physical operator (i.e., the second physical operator) in the physical execution plan adjusted by AQE (i.e., the second physical execution plan) can be matched with the physical operator (i.e., the first physical operator) in the original physical execution plan (i.e., the first physical execution plan). After successful matching, the runtime metric data of the physical operator in the physical execution plan adjusted by AQE can be synchronized to the corresponding physical operator in the original physical execution plan, that is, the first runtime metric data of the first physical operator is obtained.

[0060] The second physical execution plan can be the physical execution plan adjusted by AQE, and the second physical operator is the physical operator in the physical execution plan adjusted by AQE, simply referred to as the adjusted physical operator.

[0061] Step S203: Calculate the execution cost corresponding to each first physical execution plan based on the first runtime metric data, and determine the target physical execution plan from multiple first physical execution plans according to the execution cost.

[0062] Step S204: Execute the target physical execution plan to obtain the query result corresponding to the target query statement.

[0063] Based on the runtime metric data of the original physical operators (i.e., the first runtime metric data), the execution cost corresponding to the original physical execution plan can be calculated, so that a better physical execution plan (i.e., the target physical execution plan) can be determined from multiple original physical execution plans according to the execution cost, and then the selected better physical execution plan is executed to obtain the query result corresponding to the SQL statement.

[0064] For example, after performing operator relationship matching and synchronizing runtime metric data, the runtime metric data of all physical operators of the original physical execution plan can be obtained, which is used as the running cost of the physical operators, and by summarizing the running costs of all physical operators, the total running cost (i.e., the execution cost) of the original physical execution plan can be obtained. Taking the running duration as an example, by summarizing the running duration data in the runtime metric data of the physical operators, the running duration cost of the original physical execution plan can be calculated.

[0065] For example, the running duration costs of multiple original physical execution plans can be sorted, and the original physical execution plan corresponding to the lower running duration cost in the sorting result is used as the target physical execution plan.

[0066] In this solution, through physical operator relationship matching, the operator relationship between the physically executed plan adjusted by AQE (i.e., the second physically executed plan obtained after optimization) and the original physically executed plan (i.e., the first physically executed plan) can be identified, and the operator optimization result can be intuitively understood, so that the optimization effect and running cost of the original execution plan can be better evaluated; by extracting the runtime metric data of the physical operators of the physically executed plan adjusted by AQE and providing it to the original physically executed plan, the problem that the runtime metrics cannot be matched with the original physically executed plan due to the adjustment and optimization of the physically executed plan by AQE is solved, providing a reliable cost calculation method for the implementation of cost optimization in Apache Spark 3.

[0067] In an alternative embodiment, the Figure 4 schematic diagram shown can be used to implement cost calculation. As Figure 4 shown, first, the runtime metrics are abstractly represented, and the startup time, end time, running duration, parameters, data volume, number of output data rows, and peak memory occupancy are selected as the runtime metrics to be statistically calculated for cost calculation; then, the physical execution plan is dynamically adjusted by AQE, as Figure 4As shown, the tree structure on the left represents the original physical execution plan, and the tree structure on the right represents the physical execution plan after AQE adjustment; then relevant operators are identified and runtime metrics are associated. For example, through operator relationship matching, it is determined that the Broadcast hash join operator corresponds to the Sort merge join operator, and the runtime metric data of the Broadcast hash join operator is synchronized to the Sort merge join operator, so as to obtain the runtime metric data of the original physical execution plan. For example, the start time is "2023-06-13 17:27:31", the end time is "2023-06-13 17:28:00", the running duration is "29", the parameter is "{"leftKeys":xxx}", the data volume is "800000000", the number of output data rows is "529035", and the peak memory occupancy is "262144"; then the running cost of the original physical execution plan can be calculated based on the runtime metric data of all original physical operators.

[0068] To solve the problem that the runtime metrics cannot be matched with the original physical execution plan after the physical execution plan is adjusted and optimized by AQE, this solution provides a method for abstracting the runtime metrics of physical operators, matching the physical execution plan after AQE adjustment with the original physical execution plan, and calculating the cost based on the runtime metrics of the original physical execution plan, providing a new cost estimation ability for Apache Spark SQL optimized by AQE. First, combined with the runtime metrics of all physical operators in Apache Spark, a unified metric representation is abstracted, providing a basis for subsequent recovery of the metric mismatch caused by AQE-adjusted physical operators. Then, using the depth-first algorithm, the physical execution plan after AQE adjustment and the original physical execution plan are traversed simultaneously, and the operators in the two plans are matched based on specific rules. After successful matching, the runtime metric data of the physical operators in the physical execution plan after AQE adjustment is converted into a unified metric representation and synchronized to the physical operators in the original physical execution plan. The operator correlation judgment is realized by using the AQE optimization rules, avoiding the matching of operators with low correlation and ensuring the reliability of the matching rules. After the operator matching and runtime metric data synchronization are completed, the physical operators of the original physical execution plan with runtime metric data are stored in the database. When calculating the cost, the execution cost of the original physical execution plan can be calculated based on the runtime metric data of the physical operators of the original physical execution plan, so as to be able to screen out the target physical execution plan with lower cost from multiple original physical execution plans. In this solution, the runtime metrics of Apache Spark applications can be automatically processed, and the results of AQE dynamic adjustment operators can be identified, providing a new solution for evaluating the execution cost of Apache Spark SQL.

[0069] How to obtain the runtime metric data of the original physical operator is crucial. Therefore, in the query processing method provided in the first embodiment of this application, obtaining the first runtime metric data of each first physical operator in each first physical execution plan includes: traversing each first physical execution plan and each second physical execution plan respectively according to the target traversal algorithm, and during the traversal, matching the operator relationships between the first physical operator and the second physical operator according to the target rules to obtain the matching result; determining the first runtime metric data of each first physical operator according to the matching result.

[0070] Optionally, the target traversal algorithm can be the inorder traversal algorithm of the depth-first traversal algorithm. For example, during the process of obtaining the runtime metric data of the original physical operator, the original physical execution plan and the physical execution plan after AQE adjustment can be traversed respectively according to the inorder traversal algorithm, and during the traversal, the operator relationships between the original physical operator and the adjusted physical operator are matched according to the target rules to obtain the matching result, where the target rules can be the rules for determining that the operator matching is successful if any of the following conditions is met:

[0071] a. The names of the two operators are the same;

[0072] b. The operator parameter is a part of another operator parameter;

[0073] c. The operator name is a part of another operator name;

[0074] d. The two operators perform a Join on the same pair of tables and the keys are the same;

[0075] e. The names of the two operators are both Scan and the scanned data sources are the same;

[0076] f. The two operators are both execution command operators and the commands are the same.

[0077] Optionally, the runtime metric data of the original physical operator can be determined based on the matching result. For example, if the matching result is that the original physical operator and the adjusted physical operator match successfully, the runtime metric data of the adjusted physical operator can be converted into a unified metric representation and synchronized to the original physical operator. For example, for the runtime metric data "running duration = 10 seconds", the adjusted physical operator represents the running duration with "fetch wait time", while the original physical operator represents the running duration with "time to build hash map". In the case where the original physical operator and the adjusted physical operator match successfully, the runtime metric data of the adjusted physical operator "fetch wait time = 10 seconds" can be converted into "running duration = 10 seconds" and synchronized to the original physical operator, so as to determine the runtime metric data of the original physical operator "time to build hash map = 10 seconds".

[0078] It should be noted that through the matching of physical operator relationships, the operator relationships between the physical execution plan adjusted by AQE and the original physical execution plan can be identified, so that the optimization effect and running cost of the original execution plan can be better evaluated.

[0079] In order to be able to identify the operator relationships between the physical execution plan adjusted by AQE and the original physical execution plan, in the query processing method provided in Embodiment 1 of this application, each first physical execution plan and each second physical execution plan are traversed respectively according to the target traversal algorithm, including: for each second physical execution plan, traversing the second physical execution plan according to the target traversal algorithm to determine the second target physical operator from multiple second physical operators; for each first physical execution plan, traversing the first physical execution plan based on the second target physical operator according to the target traversal algorithm to determine the first target physical operator from multiple first physical operators; taking the next physical operator of the first target physical operator in the first physical execution plan as the next physical operator to be matched in the second physical execution plan; traversing the first physical execution plan based on the next physical operator to be matched in the second physical execution plan according to the target traversal algorithm to determine the sub-physical operator of the first target physical operator from the first physical operators other than the first target physical operator.

[0080] The key to solving the problem that the runtime metrics cannot match the original physical execution plan caused by AQE lies in identifying the relationship between the actually running physical operators and the physical operators in the original physical execution plan. Therefore, in this solution, the inorder traversal algorithm is used to traverse the original physical execution plan and the physical execution plan adjusted by AQE respectively. For example, first, the inorder traversal algorithm is used to traverse the physical execution plan adjusted by AQE to determine the root operator (i.e., the second target physical operator) in the physical execution plan adjusted by AQE from multiple adjusted physical operators. As Figure 4 shown by the Broadcast hash join operator. For example, when traversing the physical execution plan adjusted by AQE, a physical operator without a parent node is obtained, and the second target physical operator is obtained.

[0081] After obtaining the root operator in the physical execution plan adjusted by AQE, the inorder traversal algorithm can be used to traverse the original physical execution plan based on this root operator to determine the root operator (i.e., the first target physical operator) in the original physical execution plan from multiple original physical operators. As Figure 4 shown by the Sort merge join operator. Then, the next physical operator of the root operator in the original physical execution plan (such as Figure 4 shown by the Sort operator) can be used as the next physical operator to be matched in the physical execution plan adjusted by AQE. For example, Figure 4 the next physical operator of the Sort merge join operator shown is the Sort operator. The next operator of the corresponding Broadcast hash join operator, that is, the empty node or the Broadcast operator, can be used as the next physical operator to be matched.

[0082] For example, based on the empty node or the Broadcast operator, the inorder traversal algorithm is used to traverse the original physical execution plan to determine the sub-physical operators of the root operator from the original physical operators other than the root operator. For example, Figure 4 the Sort merge join operator shown is the root operator, and the Sort operator, Shuffle operator, Scan operator, and Filter operator on its two branches below are all sub-physical operators of the Sort merge join operator.

[0083] It should be noted that in this solution, first, the physical execution plan after AQE adjustment is traversed in inorder until the root operator of the original physical execution plan is matched. Then, the next operator in the original physical execution plan is traversed in inorder. Starting from the root operator matched in the physical execution plan after AQE adjustment, its child operators are traversed in inorder again until the physical operator in the original physical execution plan is matched. This loop continues until all physical operators in the two physical execution plans are traversed, enabling the identification of the operator relationships between the physical execution plan adjusted by AQE and the original physical execution plan, and thus enabling a better evaluation of the optimization effect and running cost of the original execution plan.

[0084] In order to determine the operator correspondence, in the query processing method provided in Embodiment 1 of this application, during the traversal process, the operator relationships between the first physical operator and the second physical operator are matched according to the target rule to obtain a matching result, including: obtaining the operator information of the first physical operator and the operator information of the second physical operator; judging the operator information of the first physical operator and the operator information of the second physical operator according to the target rule to obtain a judgment result; if the judgment result indicates that the operator information of the first physical operator and the operator information of the second physical operator meet the target rule, it is determined that the matching result is that the first physical operator and the second physical operator are successfully matched.

[0085] During the traversal process, the operator relationships between the original physical operator and the adjusted physical operator can be matched according to the target rule to obtain a matching result. First, the operator information of the original physical operator and the operator information of the adjusted physical operator are obtained. The operator information can be information such as the operator name, operator parameters, and operator type. Then, the operator information of the original physical operator and the operator information of the adjusted physical operator can be judged according to the target rule to obtain a judgment result. For example, if any of the following conditions is met, it is determined that the operator matching is successful:

[0086] a. The names of the two operators are the same;

[0087] b. The operator parameters are part of the operator parameters of the other operator;

[0088] c. The operator name is part of the operator name of the other operator;

[0089] d. The two operators perform a Join on the same pair of tables and have the same key;

[0090] e. The names of the two operators are both Scan and the scanned data sources are the same;

[0091] f. The two operators are both execution command operators and have the same command.

[0092] If the judgment result indicates that the operator information of the original physical operator and the operator information of the adjusted physical operator meet the target rule (for example, meet any of the above conditions), it can be determined that the matching result is that the original physical operator and the adjusted physical operator match successfully, that is, the corresponding relationship between the two physical operators is obtained. For example, the adjusted physical operator Broadcast hashjoin corresponds to the original physical operator Sort merge join.

[0093] In order to be able to calculate costs based on the runtime metric data of the physical execution plan adjusted by AQE, in the query processing method provided in Embodiment 1 of this application, before receiving the target query statement, multiple runtime metrics are obtained, where the multiple runtime metrics include a first runtime metric and a second runtime metric; the multiple runtime metrics are processed according to the semantic representation information of the runtime metrics to obtain a target mapping relationship.

[0094] Since many different metrics are generated during the execution of different physical operators, for example, metrics such as execution time and data size. Taking the data size as an example, operators such as TakeOrderedAndProject use the shuffle byteswritten metric to measure the data size processed during the operator runtime, while operators such as BroadcastExchange and ShuffleExchange use the data size metric to measure the data size processed during the operator runtime, and moreover, the physical operators in the physical execution plan adjusted by AQE will bring new runtime metrics. For example, for the runtime metric "running duration", the adjusted physical operator uses the fetch wait time to represent the running duration, while the original physical operator uses the time to buildhash map to represent the running duration, resulting in the inability to provide the runtime metrics to the corresponding physical operators in the original physical execution plan. Therefore, in this solution, the Apache Spark runtime metrics are abstractly represented, and the runtime metrics with the same meaning are unified to generate a mapping relationship between the runtime metrics and the metric information, obtaining the target mapping relationship.

[0095] For example, before receiving the target query statement, multiple runtime metrics are obtained, including the runtime metrics of the original physical execution plan (i.e., the first runtime metric) and the runtime metrics of the physical execution plan adjusted by AQE (i.e., the second runtime metric), and then the multiple runtime metrics can be inductively unified according to the semantic representation information of the runtime metrics (such as data volume, running duration, etc.) to obtain the target mapping relationship shown in Table 1.

[0096] Table 1

[0097]

[0098] In order to obtain the runtime metric data of the original physical operator, in the query processing method provided in the first embodiment of the present application, determining the first runtime metric data of each first physical operator according to the matching result includes: for each first physical operator, if the matching result is that the first physical operator and the second physical operator match successfully, determining the operator correspondence between the first physical operator and the second physical operator; obtaining the second runtime metric data of the second physical operator, and converting the second runtime metric data according to the target mapping relationship to obtain the target metric information, where the target mapping relationship is used to represent the correspondence between the semantic representation information and the runtime metric; and assigning the target metric information to the first physical operator according to the operator correspondence to obtain the first runtime metric data of each first physical operator.

[0099] Optionally, in the process of determining the runtime metric data of the original physical operator according to the matching result, if the matching result is that the original physical operator and the adjusted physical operator match successfully, the operator correspondence between the original physical operator and the adjusted physical operator can be determined. For example, the Broadcast hash join operator corresponds to the Sort merge join operator.

[0100] After the operators of the two physical execution plans match successfully, the runtime metric data of the adjusted physical operator can be converted into an abstract expression and assigned to the original physical operator. Therefore, obtain the runtime metric data of the adjusted physical operator (i.e., the second runtime metric data), and convert the runtime metric data of the adjusted physical operator according to the target mapping relationship to obtain the target metric information, where the target metric information may include semantic representation information (i.e., literal description meaning) and numerical information. For example, the target metric information is "running duration = 10 seconds", and then assign the target metric information to the corresponding original physical operator according to the operator correspondence to obtain the runtime metric data of the original physical operator.

[0101] For example, for the runtime metric data "running duration = 10 seconds", the adjusted physical operator represents the running duration with "fetch waittime", while the original physical operator represents the running duration with "time to build hash map". When the original physical operator and the adjusted physical operator match successfully, that is, the operator correspondence between the original physical operator and the adjusted physical operator is determined, the runtime metric data of the adjusted physical operator "fetch wait time = 10 seconds" can be converted into "running duration = 10 seconds" and synchronized to the original physical operator, and then the runtime metric data of the original physical operator "time to build hash map = 10 seconds" can be determined.

[0102] In order to obtain the runtime metric data of the original physical operator, in the query processing method provided in the first embodiment of this application, the target metric information is assigned to the first physical operator according to the operator correspondence relationship to obtain the first runtime metric data of each first physical operator, including: for each first physical operator, the semantic representation information in the target metric information is assigned to the key field of the first runtime metric of the first physical operator according to the operator correspondence relationship, and the numerical information in the target metric information is assigned to the value field of the first runtime metric of the first physical operator; the key field and the value field of the first runtime metric are combined to form the first runtime metric data of the first physical operator.

[0103] Optionally, in the process of assigning the target metric information to the original physical operator according to the operator correspondence relationship to obtain the runtime metric data of the original physical operator, the semantic representation information in the target metric information can be assigned to the key field of the runtime metric of the original physical operator according to the operator correspondence relationship, and the numerical information in the target metric information can be assigned to the value field of the runtime metric of the original physical operator, and then the key field and the value field are combined to form the runtime metric data of the original physical operator. For example, if the target metric information is "data volume = 1000", the key field Key of the runtime metric of the original physical operator is assigned the value of data volume, and the value field Value of the runtime metric of the original physical operator is assigned the value of 1000.

[0104] It should be noted that by extracting the runtime metric data of the physical execution plan adjusted by AQE and converting it into an abstract expression to provide to the original physical execution plan, a data basis is provided for cost calculation, so that the optimization effect and runtime cost of the original execution plan can be better evaluated, and the problem that the runtime metrics cannot match the original physical execution plan caused by the AQE adjustment of the physical operator is solved.

[0105] In order to be able to perform cost calculation, in the query processing method provided in the first embodiment of this application, the execution cost corresponding to each first physical execution plan is calculated according to the first runtime metric data, including: for each first physical execution plan, the runtime metric data of multiple sub-physical operators of the first target physical operator in the first physical execution plan is summed to obtain the sum of the initial metric data; the sum of the initial metric data and the runtime metric data of the first target physical operator are summed to obtain the sum of the target metric data; the sum of the target metric data is used as the execution cost corresponding to the first physical execution plan.

[0106] Based on the runtime metric data of the original physical operators, the execution cost corresponding to the original physical execution plan can be calculated. For example, after performing operator relationship matching and synchronizing the runtime metric data, the runtime metric data of all physical operators in the original physical execution plan can be obtained. First, sum the runtime metric data of multiple sub-physical operators of the root operator in the original physical execution plan to obtain the sum of the initial metric data. Then, sum the sum of the initial metric data and the runtime metric data of the root operator to obtain the sum of the target metric data, and use the sum of the target metric data as the execution cost corresponding to the original physical execution plan. For example, for the root operator, its runtime cost can be merged with the runtime costs of the sub-operators to obtain the total runtime cost of the physical execution plan. Taking the runtime duration as an example, this solution can aggregate the runtime durations in the runtime metric data of physical operators from bottom to top, and the total runtime duration obtained by the root operator after calculation is the runtime duration cost of the physical execution plan.

[0107] In order to be able to determine a better physical execution plan, in the query processing method provided in the first embodiment of this application, the target physical execution plan is determined from multiple first physical execution plans according to the execution cost, including: sorting the sums of the target metric data corresponding to the multiple first physical execution plans to obtain a sorting result; determining the target physical execution plan from the multiple first physical execution plans according to the sorting result.

[0108] Taking the runtime duration as an example, by summarizing the runtime duration data in the runtime metric data of physical operators, the runtime duration cost of the original physical execution plan can be calculated. For example, after summing the sum of the initial metric data and the runtime metric data of the root operator to obtain the sum of the target metric data, and using the sum of the target metric data as the execution cost corresponding to the original physical execution plan, the runtime duration costs corresponding to multiple original physical execution plans can be sorted, and the original physical execution plan corresponding to the lower runtime duration cost in the sorting result is used as the target physical execution plan, so as to determine a better physical execution plan from multiple original physical execution plans.

[0109] It should be noted that in this solution, a method for calculating the cost of the Apache Spark physical execution plan based on runtime metrics is provided. By abstractly expressing the runtime metrics of Apache Spark physical operators, the metric measurement of different physical operators is unified, improving the comparability of runtime metrics between different physical operators. Moreover, this solution can identify the operator correspondence between the physical execution plan adjusted by AQE and the original physical execution plan, intuitively understand the operator optimization results, and thus be able to extract the runtime metric data of the physical execution plan adjusted by AQE, convert it into an abstract expression and provide it to the original physical execution plan, and then be able to perform cost calculation, better evaluate the optimization effect and runtime cost of the original execution plan, solve the problem that the runtime metrics cannot match the original physical execution plan due to the adjustment of physical operators by AQE, and provide a reliable cost calculation method for the implementation of cost optimization in Apache Spark 3.

[0110] In an embodiment of the present application, by receiving a target query statement and generating an original physical execution plan for query processing based on the target query statement, a plurality of first physical execution plans are obtained. Each first physical execution plan includes a plurality of first physical operators, and each first physical operator is used to implement query processing. Obtain the first runtime metric data of each first physical operator in each first physical execution plan, where the first runtime metric data is determined after the first physical operator performs operator relationship matching with a second physical operator. The second physical operator is a physical operator of a second physical execution plan obtained by optimizing the first physical execution plan according to the adaptive query execution method. Calculate the execution cost corresponding to each first physical execution plan based on the first runtime metric data, and determine a target physical execution plan from the plurality of first physical execution plans according to the execution cost. Execute the target physical execution plan to obtain a query result corresponding to the target query statement. This solves the technical problem in the related art that when using the adaptive query execution method to optimize the original physical execution plan, it is impossible to calculate the cost based on the runtime metrics of the optimized physical execution plan, resulting in the inability to determine the target physical execution plan. Through physical operator relationship matching, the operator relationship between the physical execution plan adjusted by AQE (i.e., the optimized second physical execution plan) and the original physical execution plan (i.e., the first physical execution plan) can be identified, intuitively understanding the operator optimization result, so as to better evaluate the optimization effect and running cost of the original execution plan. By extracting the runtime metric data of the physical operators of the physical execution plan adjusted by AQE and providing it to the original physical execution plan, the problem that the runtime metrics cannot be matched with the original physical execution plan after the physical execution plan is adjusted and optimized by AQE is solved, providing a reliable cost calculation method for the implementation of cost optimization in Apache Spark 3, achieving the purpose of effectively identifying the operator relationship between the physical execution plan adjusted by AQE and the original physical execution plan, thereby realizing the technical effect of being able to calculate the cost based on the runtime metrics of the optimized physical execution plan, and thus being able to determine a better physical execution plan.

[0111] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0112] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0113] Embodiment 2

[0114] According to an embodiment of the present application, there is also provided a query processing method, as Figure 5 shown, the method includes:

[0115] Step S501, obtaining a target query statement uploaded by a client;

[0116] Step S502, generating an original physical execution plan for query processing in a cloud server according to the target query statement to obtain a plurality of first physical execution plans. Among them, each first physical execution plan includes a plurality of first physical operators, and each first physical operator is used to implement query processing; obtaining first runtime metric data of each first physical operator in each first physical execution plan, where the first runtime metric data is determined after the first physical operator performs operator relationship matching with a second physical operator, and the second physical operator is a physical operator of a second physical execution plan obtained by optimizing the first physical execution plan according to an adaptive query execution method; calculating the execution cost corresponding to each first physical execution plan according to the first runtime metric data, and determining a target physical execution plan from the plurality of first physical execution plans according to the execution cost; executing the target physical execution plan to obtain a query result corresponding to the target query statement;

[0117] Step S503, feeding back the query result corresponding to the target query statement to the client.

[0118] Through the above solution, by matching physical operator relationships, the operator relationships between the physically executed plan adjusted by AQE (i.e., the second physically executed plan obtained after optimization) and the original physically executed plan (i.e., the first physically executed plan) can be identified, intuitively understanding the operator optimization results, so as to better evaluate the optimization effect and running cost of the original execution plan; by extracting the runtime index data of the physical operators of the physically executed plan adjusted by AQE and providing it to the original physically executed plan, the problem that the runtime index cannot match the original physically executed plan due to the adjustment and optimization of the physically executed plan by AQE is solved, providing a reliable cost calculation method for the implementation of cost optimization in Apache Spark 3.

[0119] In the cloud server, the specific method for query processing is the same as that in the first embodiment, and will not be elaborated here.

[0120] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0121] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of this application, in essence, or the part that makes a contribution to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including several instructions to enable a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of this application.

[0122] Embodiment 3

[0123] According to an embodiment of the present application, there is also provided a query processing device for implementing the above query processing method, as Figure 6 shown, the device includes: a first receiving unit 601, a first obtaining unit 602, a first determining unit 603, and a first processing unit 604.

[0124] A first receiving unit 601, configured to receive a target query statement, generate an original physical execution plan for query processing based on the target query statement, and obtain a plurality of first physical execution plans, where each first physical execution plan includes a plurality of first physical operators, and each first physical operator is used to implement query processing;

[0125] A first obtaining unit 602, configured to obtain first runtime metric data of each first physical operator in each first physical execution plan, where the first runtime metric data is determined after the first physical operator matches an operator relationship with a second physical operator, and the second physical operator is a physical operator of a second physical execution plan obtained by optimizing the first physical execution plan according to an adaptive query execution method;

[0126] A first determining unit 603, configured to calculate an execution cost corresponding to each first physical execution plan according to the first runtime metric data, and determine a target physical execution plan from the plurality of first physical execution plans according to the execution cost;

[0127] A first processing unit 604, configured to execute the target physical execution plan to obtain a query result corresponding to the target query statement.

[0128] In the query processing device provided in the third embodiment of the present application, the first receiving unit 601 receives a target query statement, and generates an original physical execution plan for query processing based on the target query statement, obtaining a plurality of first physical execution plans. Each first physical execution plan includes a plurality of first physical operators, and each first physical operator is used to implement query processing. The first obtaining unit 602 obtains first runtime metric data of each first physical operator in each first physical execution plan, where the first runtime metric data is determined after the first physical operator matches the operator relationship with the second physical operator. The second physical operator is a physical operator of a second physical execution plan obtained by optimizing the first physical execution plan according to the adaptive query execution method. The first determining unit 603 calculates the execution cost corresponding to each first physical execution plan based on the first runtime metric data, and determines a target physical execution plan from the plurality of first physical execution plans based on the execution cost. The first processing unit 604 executes the target physical execution plan to obtain a query result corresponding to the target query statement. In this solution, through the matching of physical operator relationships, the operator relationship between the physical execution plan adjusted by AQE (i.e., the optimized second physical execution plan) and the original physical execution plan (i.e., the first physical execution plan) can be identified, and the operator optimization result can be intuitively understood, so that the optimization effect and running cost of the original execution plan can be better evaluated. By extracting the runtime metric data of the physical operators of the physical execution plan adjusted by AQE and providing it to the original physical execution plan, the problem that the runtime metrics cannot match the original physical execution plan due to the adjustment and optimization of the physical execution plan by AQE is solved, providing a reliable cost calculation method for the implementation of cost optimization in Apache Spark 3, achieving the purpose of effectively identifying the operator relationship between the physical execution plan adjusted by AQE and the original physical execution plan, thereby realizing the technical effect of being able to calculate the cost based on the runtime metrics of the optimized physical execution plan, and thus being able to determine a better physical execution plan, and further solving the technical problem in the related art that when the original physical execution plan is optimized by the adaptive query execution method, it is impossible to calculate the cost based on the runtime metrics of the optimized physical execution plan, resulting in the inability to determine the target physical execution plan.

[0129] Optionally, in the query processing device provided in the third embodiment of the present application, the first obtaining unit 602 includes: a first processing subunit, configured to traverse each first physical execution plan and each second physical execution plan respectively according to a target traversal algorithm, and match the operator relationship between the first physical operator and the second physical operator according to a target rule during the traversal process to obtain a matching result; a first determining subunit, configured to determine the first runtime metric data of each first physical operator according to the matching result.

[0130] Optionally, in the query processing device provided in the third embodiment of the present application, the first processing subunit includes: a first determination module, configured to, for each second physical execution plan, traverse the second physical execution plan according to a target traversal algorithm to determine a second target physical operator from multiple second physical operators; a second determination module, configured to, for each first physical execution plan, traverse the first physical execution plan based on the second target physical operator according to the target traversal algorithm to determine a first target physical operator from multiple first physical operators; a third determination module, configured to use the next physical operator of the first target physical operator in the first physical execution plan as the next physical operator to be matched in the second physical execution plan; a fourth determination module, configured to traverse the first physical execution plan based on the next physical operator to be matched in the second physical execution plan according to the target traversal algorithm to determine a sub-physical operator of the first target physical operator from the first physical operators other than the first target physical operator.

[0131] Optionally, in the query processing device provided in the third embodiment of the present application, the first processing subunit further includes: a first acquisition module, configured to acquire the operator information of the first physical operator and the operator information of the second physical operator; a first judgment module, configured to judge the operator information of the first physical operator and the operator information of the second physical operator according to a target rule to obtain a judgment result; a fifth determination module, configured to, if the judgment result indicates that the operator information of the first physical operator and the operator information of the second physical operator satisfy the target rule, determine that the matching result is that the first physical operator and the second physical operator are successfully matched.

[0132] Optionally, in the query processing device provided in the third embodiment of the present application, the first determination subunit includes: a sixth determination module, configured to, for each first physical operator, if the matching result is that the first physical operator and the second physical operator are successfully matched, determine the operator correspondence between the first physical operator and the second physical operator; a first processing module, configured to acquire the second runtime index data of the second physical operator and convert the second runtime index data according to a target mapping relationship to obtain target index information, where the target mapping relationship is used to represent the correspondence between semantic representation information and runtime indexes; a second processing module, configured to assign the target index information to the first physical operator according to the operator correspondence to obtain the first runtime index data of each first physical operator.

[0133] Optionally, in the query processing device provided in the third embodiment of the present application, the second processing module includes: a first processing sub-module, configured to, for each first physical operator, assign the semantic representation information in the target metric information to the key field of the first runtime metric of the first physical operator according to the operator correspondence relationship, and assign the numerical information in the target metric information to the value field of the first runtime metric of the first physical operator; a second processing sub-module, configured to form the first runtime metric data of the first physical operator by combining the key field of the first runtime metric and the value field of the first runtime metric.

[0134] Optionally, in the query processing device provided in the third embodiment of the present application, the first determination unit 603 includes: a first calculation sub-unit, configured to, for each first physical execution plan, perform a summation calculation on the runtime metric data of multiple sub-physical operators of the first target physical operator in the first physical execution plan to obtain an initial metric data sum; a second calculation sub-unit, configured to perform a summation calculation on the initial metric data sum and the runtime metric data of the first target physical operator to obtain a target metric data sum; a second determination sub-unit, configured to use the target metric data sum as the execution cost corresponding to the first physical execution plan.

[0135] Optionally, in the query processing device provided in the third embodiment of the present application, the first determination unit 603 further includes: a sorting sub-unit, configured to sort the target metric data sums corresponding to multiple first physical execution plans to obtain a sorting result; a third determination sub-unit, configured to determine a target physical execution plan from multiple first physical execution plans according to the sorting result.

[0136] Optionally, in the query processing device provided in the third embodiment of the present application, the query processing device further includes: a second acquisition unit, configured to acquire multiple runtime metrics before receiving a target query statement, where the multiple runtime metrics include a first runtime metric and a second runtime metric; a second processing unit, configured to process the multiple runtime metrics according to the semantic representation information of the runtime metrics to obtain a target mapping relationship.

[0137] It should be noted here that the above first receiving unit 601, first acquisition unit 602, first determination unit 603, and first processing unit 604 correspond to steps S201 to S204 in Embodiment 1. The above units and the corresponding steps have the same implemented examples and application scenarios, but are not limited to the content disclosed in the above Embodiment 1. It should be noted that the above modules, as part of the device, can run in the computer terminal 10 provided in Embodiment 1.

[0138] It should be noted that the preferred implementation schemes involved in the above embodiments of the present application are the same as the schemes, application scenarios, and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0139] Example 4

[0140] An embodiment of the present application may provide a computer terminal, which may be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the above computer terminal may also be replaced with a terminal device such as a mobile terminal.

[0141] Optionally, in this embodiment, the above computer terminal may be located in at least one of multiple network devices in a computer network.

[0142] In this embodiment, the above computer terminal may execute the program code of the following steps in the query processing method: receiving a target query statement, and generating an original physical execution plan for query processing based on the target query statement to obtain a plurality of first physical execution plans, where each first physical execution plan includes a plurality of first physical operators, and each first physical operator is used to implement query processing; obtaining first runtime metric data of each first physical operator in each first physical execution plan, where the first runtime metric data is determined after the first physical operator is matched with a second physical operator in terms of operator relationship, and the second physical operator is a physical operator of a second physical execution plan obtained by optimizing the first physical execution plan according to the adaptive query execution method; calculating the execution cost corresponding to each first physical execution plan based on the first runtime metric data, and determining a target physical execution plan from the plurality of first physical execution plans based on the execution cost; executing the target physical execution plan to obtain a query result corresponding to the target query statement.

[0143] The above computer terminal may also execute the program code of the following steps in the query processing method: obtaining first runtime metric data of each first physical operator in each first physical execution plan, including: traversing each first physical execution plan and each second physical execution plan respectively according to a target traversal algorithm, and matching the first physical operator with the second physical operator in terms of operator relationship according to a target rule during the traversal process to obtain a matching result; determining the first runtime metric data of each first physical operator based on the matching result.

[0144] The above computer terminal can also execute the program code for the following steps in the query processing method: traversing each first physical execution plan and each second physical execution plan respectively according to the target traversal algorithm, including: for each second physical execution plan, traversing the second physical execution plan according to the target traversal algorithm to determine a second target physical operator from multiple second physical operators; for each first physical execution plan, traversing the first physical execution plan based on the second target physical operator according to the target traversal algorithm to determine a first target physical operator from multiple first physical operators; taking the next physical operator of the first target physical operator in the first physical execution plan as the next physical operator to be matched in the second physical execution plan; traversing the first physical execution plan based on the next physical operator to be matched in the second physical execution plan according to the target traversal algorithm to determine the sub-physical operator of the first target physical operator from the first physical operators other than the first target physical operator.

[0145] The above computer terminal can also execute the program code for the following steps in the query processing method: performing operator relationship matching on the first physical operator and the second physical operator according to the target rule during the traversal process to obtain a matching result, including: obtaining the operator information of the first physical operator and the operator information of the second physical operator; judging the operator information of the first physical operator and the operator information of the second physical operator according to the target rule to obtain a judgment result; if the judgment result indicates that the operator information of the first physical operator and the operator information of the second physical operator satisfy the target rule, determining that the matching result is that the first physical operator and the second physical operator are successfully matched.

[0146] The above computer terminal can also execute the program code for the following steps in the query processing method: determining the first runtime index data of each first physical operator according to the matching result, including: for each first physical operator, if the matching result is that the first physical operator and the second physical operator are successfully matched, determining the operator correspondence between the first physical operator and the second physical operator; obtaining the second runtime index data of the second physical operator, and converting the second runtime index data according to the target mapping relationship to obtain target index information, where the target mapping relationship is used to represent the correspondence between semantic representation information and runtime indexes; assigning the target index information to the first physical operator according to the operator correspondence to obtain the first runtime index data of each first physical operator.

[0147] The above computer terminal can also execute the program code for the following steps in the query processing method: Assign the target metric information to the first physical operator according to the operator correspondence relationship to obtain the first run-time metric data of each first physical operator, including: for each first physical operator, assign the semantic representation information in the target metric information to the key field of the first run-time metric of the first physical operator, and assign the numerical information in the target metric information to the value field of the first run-time metric of the first physical operator; Combine the key field and the value field of the first run-time metric to form the first run-time metric data of the first physical operator.

[0148] The above computer terminal can also execute the program code for the following steps in the query processing method: Calculate the execution cost corresponding to each first physical execution plan according to the first run-time metric data, including: for each first physical execution plan, perform a summation calculation on the run-time metric data of multiple sub-physical operators of the first target physical operator in the first physical execution plan to obtain the sum of the initial metric data; Perform a summation calculation on the sum of the initial metric data and the run-time metric data of the first target physical operator to obtain the sum of the target metric data; Use the sum of the target metric data as the execution cost corresponding to the first physical execution plan.

[0149] The above computer terminal can also execute the program code for the following steps in the query processing method: Determine the target physical execution plan from multiple first physical execution plans according to the execution cost, including: Sort the sums of the target metric data corresponding to multiple first physical execution plans to obtain a sorting result; Determine the target physical execution plan from multiple first physical execution plans according to the sorting result.

[0150] The above computer terminal can also execute the program code for the following steps in the query processing method: Before receiving the target query statement, obtain multiple run-time metrics, where the multiple run-time metrics include the first run-time metric and the second run-time metric; Process the multiple run-time metrics according to the semantic representation information of the run-time metrics to obtain a target mapping relationship.

[0151] Optionally, Figure 7 is a structural block diagram of a computer terminal according to an embodiment of the present application. As Figure 7 shown, the computer terminal 10 may include: one or more ( Figure 7 only one is shown in the figure) processors 102, a memory 104. The computer terminal 10 may also include a storage controller to control and manage the memory 104 through the storage controller; The computer terminal 10 may also include a peripheral interface to connect a radio frequency module, an audio module, a display screen, etc. through the peripheral interface.

[0152] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the query processing method and device in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implements the above-mentioned query processing method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory may further include a memory remotely disposed relative to the processor, and these remote memories may be connected to the terminal 10 through a network. Examples of the above network include but are not limited to the Internet, enterprise intranet, local area network, mobile communication network, and combinations thereof.

[0153] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: receiving a target query statement, and generating an original physical execution plan for query processing based on the target query statement to obtain a plurality of first physical execution plans, where each first physical execution plan includes a plurality of first physical operators, and each first physical operator is used to implement query processing; obtaining first runtime metric data of each first physical operator in each first physical execution plan, where the first runtime metric data is determined after the first physical operator and a second physical operator perform operator relationship matching, and the second physical operator is a physical operator of a second physical execution plan obtained by optimizing the first physical execution plan according to the adaptive query execution method; calculating the execution cost corresponding to each first physical execution plan based on the first runtime metric data, and determining a target physical execution plan from the plurality of first physical execution plans based on the execution cost; executing the target physical execution plan to obtain a query result corresponding to the target query statement.

[0154] Optionally, the above processor may also execute the program code of the following steps: obtaining first runtime metric data of each first physical operator in each first physical execution plan, including: traversing each first physical execution plan and each second physical execution plan respectively according to the target traversal algorithm, and performing operator relationship matching between the first physical operator and the second physical operator according to the target rule during the traversal process to obtain a matching result; determining the first runtime metric data of each first physical operator based on the matching result.

[0155] Optionally, the above-mentioned processor may also execute the program code of the following steps: traverse each first physical execution plan and each second physical execution plan respectively according to the target traversal algorithm, including: for each second physical execution plan, traverse the second physical execution plan according to the target traversal algorithm to determine a second target physical operator from multiple second physical operators; for each first physical execution plan, traverse the first physical execution plan based on the second target physical operator according to the target traversal algorithm to determine a first target physical operator from multiple first physical operators; use the next physical operator of the first target physical operator in the first physical execution plan as the next physical operator to be matched in the second physical execution plan; traverse the first physical execution plan based on the next physical operator to be matched in the second physical execution plan according to the target traversal algorithm to determine a sub-physical operator of the first target physical operator from the first physical operators other than the first target physical operator.

[0156] Optionally, the above-mentioned processor may also execute the program code of the following steps: match the operator relationships between the first physical operator and the second physical operator according to the target rules during the traversal process to obtain a matching result, including: obtain the operator information of the first physical operator and the operator information of the second physical operator; judge the operator information of the first physical operator and the operator information of the second physical operator according to the target rules to obtain a judgment result; if the judgment result indicates that the operator information of the first physical operator and the operator information of the second physical operator meet the target rules, determine that the matching result is that the first physical operator and the second physical operator are successfully matched.

[0157] Optionally, the above-mentioned processor may also execute the program code of the following steps: determine the first runtime index data of each first physical operator according to the matching result, including: for each first physical operator, if the matching result is that the first physical operator and the second physical operator are successfully matched, determine the operator correspondence between the first physical operator and the second physical operator; obtain the second runtime index data of the second physical operator, and convert the second runtime index data according to the target mapping relationship to obtain target index information, where the target mapping relationship is used to represent the correspondence between semantic representation information and runtime indexes; assign the target index information to the first physical operator according to the operator correspondence to obtain the first runtime index data of each first physical operator.

[0158] Optionally, the above-mentioned processor may also execute the program code of the following steps: assign the target metric information to the first physical operator according to the operator correspondence relationship, and obtain the first runtime metric data of each first physical operator, including: for each first physical operator, assign the semantic representation information in the target metric information to the key field of the first runtime metric of the first physical operator, and assign the numerical information in the target metric information to the value field of the first runtime metric of the first physical operator; form the first runtime metric data of the first physical operator by combining the key field and the value field of the first runtime metric.

[0159] Optionally, the above-mentioned processor may also execute the program code of the following steps: calculate the execution cost corresponding to each first physical execution plan according to the first runtime metric data, including: for each first physical execution plan, perform a summation calculation on the runtime metric data of multiple sub-physical operators of the first target physical operator in the first physical execution plan to obtain the sum of the initial metric data; perform a summation calculation on the sum of the initial metric data and the runtime metric data of the first target physical operator to obtain the sum of the target metric data; use the sum of the target metric data as the execution cost corresponding to the first physical execution plan.

[0160] Optionally, the above-mentioned processor may also execute the program code of the following steps: determine the target physical execution plan from multiple first physical execution plans according to the execution cost, including: sort the sums of the target metric data corresponding to multiple first physical execution plans to obtain a sorting result; determine the target physical execution plan from multiple first physical execution plans according to the sorting result.

[0161] Optionally, the above-mentioned processor may also execute the program code of the following steps: before receiving the target query statement, obtain multiple runtime metrics, where the multiple runtime metrics include the first runtime metric and the second runtime metric; process the multiple runtime metrics according to the semantic representation information of the runtime metrics to obtain a target mapping relationship.

[0162] Those of ordinary skill in the art can understand that Figure 7 the structure shown is only for illustration, and the computer terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, and terminal devices such as Mobile Internet Devices (MID), PAD, etc. Figure 7 It does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 may also include more or fewer components (such as a network interface, a display device, etc.) than those shown Figure 7 in, or have a different configuration from that shown Figure 7 .

[0163] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by a program instructing the relevant hardware of the terminal device. The program can be stored in a computer-readable storage medium, and the storage medium can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, etc.

[0164] Embodiment 5

[0165] An embodiment of the present application further provides a computer-readable storage medium. Optionally, in this embodiment, the above storage medium can be used to store the program code executed by the query processing method provided in the first embodiment above.

[0166] Optionally, in this embodiment, the above storage medium can be located in any one of the computer terminals in a computer terminal group in a computer network, or in any one of the mobile terminals in a mobile terminal group.

[0167] The serial numbers of the above embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0168] In the above embodiments of the present application, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0169] In the several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces, and the indirect coupling or communication connection of units or modules can be in an electrical or other form.

[0170] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0171] In addition, in each embodiment of the present application, each functional unit may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0172] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs that can store program codes.

[0173] The above are only the preferred embodiments of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A query processing method, characterized in that, Including: Receiving a target query statement, and generating an original physical execution plan for query processing according to the target query statement to obtain a plurality of first physical execution plans, wherein each first physical execution plan includes a plurality of first physical operators, and each first physical operator is used to implement the query processing; Obtaining first runtime metric data of each first physical operator in each first physical execution plan, wherein the first runtime metric data is determined after the first physical operator matches the operator relationship with a second physical operator, and the second physical operator is a physical operator of a second physical execution plan obtained by optimizing the first physical execution plan according to an adaptive query execution method; Calculating the execution cost corresponding to each first physical execution plan according to the first runtime metric data, and determining a target physical execution plan from the plurality of first physical execution plans according to the execution cost; Executing the target physical execution plan to obtain a query result corresponding to the target query statement.

2. The method according to claim 1, wherein Obtaining first runtime metric data of each first physical operator in each first physical execution plan includes: Traversing each first physical execution plan and each second physical execution plan respectively according to a target traversal algorithm, and matching the operator relationship between the first physical operator and the second physical operator according to a target rule during the traversal process to obtain a matching result; Determining the first runtime metric data of each first physical operator according to the matching result.

3. The method according to claim 2, wherein Traversing each first physical execution plan and each second physical execution plan respectively according to a target traversal algorithm includes: For each second physical execution plan, traversing the second physical execution plan according to the target traversal algorithm to determine a second target physical operator from a plurality of second physical operators; For each first physical execution plan, traversing the first physical execution plan based on the second target physical operator according to the target traversal algorithm to determine a first target physical operator from a plurality of first physical operators; Taking the next physical operator of the first target physical operator in the first physical execution plan as the next physical operator to be matched in the second physical execution plan; Traversing the first physical execution plan based on the next physical operator to be matched in the second physical execution plan according to the target traversal algorithm to determine a sub-physical operator of the first target physical operator from the first physical operators other than the first target physical operator.

4. The method according to claim 2, wherein Matching the operator relationship between the first physical operator and the second physical operator according to a target rule during the traversal process to obtain a matching result includes: Obtaining the operator information of the first physical operator and the operator information of the second physical operator; Judging the operator information of the first physical operator and the operator information of the second physical operator according to the target rule to obtain a judgment result; If the determination result indicates that the operator information of the first physical operator and the operator information of the second physical operator satisfy the target rule, determine that the matching result is that the first physical operator and the second physical operator match successfully.

5. The method according to claim 2, wherein Determine the first runtime index data of each first physical operator according to the matching result, including: For each first physical operator, if the matching result is that the first physical operator and the second physical operator match successfully, determine the operator correspondence between the first physical operator and the second physical operator; Obtain the second runtime index data of the second physical operator, and convert the second runtime index data according to the target mapping relationship to obtain target index information, where the target mapping relationship is used to represent the correspondence between semantic representation information and runtime indexes; Assign the target index information to the first physical operator according to the operator correspondence to obtain the first runtime index data of each first physical operator.

6. The method according to claim 5, wherein Assign the target index information to the first physical operator according to the operator correspondence to obtain the first runtime index data of each first physical operator, including: For each first physical operator, assign the semantic representation information in the target index information to the key field of the first runtime index of the first physical operator, and assign the numerical information in the target index information to the value field of the first runtime index of the first physical operator; Combine the key field of the first runtime index and the value field of the first runtime index to form the first runtime index data of the first physical operator.

7. The method according to claim 3, characterized in that, Calculate the execution cost corresponding to each first physical execution plan according to the first runtime index data, including: For each first physical execution plan, perform a summation calculation on the runtime index data of multiple sub-physical operators of the first target physical operator in the first physical execution plan to obtain the sum of the initial index data; Perform a summation calculation on the sum of the initial index data and the runtime index data of the first target physical operator to obtain the sum of the target index data; Use the sum of the target index data as the execution cost corresponding to the first physical execution plan.

8. The method according to claim 7, wherein Determine the target physical execution plan from the multiple first physical execution plans according to the execution cost, including: Sort the sums of the target index data corresponding to the multiple first physical execution plans to obtain a sorting result; Determine the target physical execution plan from the multiple first physical execution plans according to the sorting result.

9. The method according to claim 5, wherein Before receiving the target query statement, the method further includes: Obtain multiple runtime indexes, where the multiple runtime indexes include a first runtime index and a second runtime index; Process the multiple runtime indexes according to the semantic representation information of the runtime indexes to obtain the target mapping relationship.

10. A query processing method, characterized in that, Including: Obtain the target query statement uploaded by the client; Generate an original physical execution plan for query processing based on the target query statement in the cloud server, obtaining multiple first physical execution plans. Each first physical execution plan includes multiple first physical operators, and each first physical operator is used to implement the query processing; obtain the first runtime metric data of each first physical operator in each first physical execution plan, where the first runtime metric data is determined after the first physical operator matches the operator relationship with the second physical operator, and the second physical operator is the physical operator of the second physical execution plan obtained by optimizing the first physical execution plan according to the adaptive query execution method; calculate the execution cost corresponding to each first physical execution plan based on the first runtime metric data, and determine the target physical execution plan from the multiple first physical execution plans according to the execution cost; execute the target physical execution plan to obtain the query result corresponding to the target query statement; Feed back the query result corresponding to the target query statement to the client.

11. A query processing device, characterized in that, Comprising: A first receiving unit, configured to receive a target query statement, and generate an original physical execution plan for query processing based on the target query statement, obtaining multiple first physical execution plans. Each first physical execution plan includes multiple first physical operators, and each first physical operator is used to implement the query processing; A first obtaining unit, configured to obtain the first runtime metric data of each first physical operator in each first physical execution plan, where the first runtime metric data is determined after the first physical operator matches the operator relationship with the second physical operator, and the second physical operator is the physical operator of the second physical execution plan obtained by optimizing the first physical execution plan according to the adaptive query execution method; A first determining unit, configured to calculate the execution cost corresponding to each first physical execution plan based on the first runtime metric data, and determine the target physical execution plan from the multiple first physical execution plans according to the execution cost; A first processing unit, configured to execute the target physical execution plan to obtain the query result corresponding to the target query statement.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, where, when the program runs, it controls the device where the storage medium is located to execute the query processing method according to any one of claims 1 to 10.

13. An electronic device, characterized in that, Comprising: A memory, storing an executable program; A processor, configured to run the program, where, when the program runs, it executes the query processing method according to any one of claims 1 to 10.

Images