Database query processing method, device, and apparatus, and storage medium

Abstract

The application provides a database query processing method, device and equipment and a storage medium, and the method is used for a database, the database includes heterogeneous hardware for executing intensive operation, an extension plug-in and a storage engine plug-in; the plug-in is used for realizing connection between the heterogeneous hardware and a database server, and can unload intensive operation of the database; the method comprises the following steps: generating a plurality of query plan trees based on an obtained query statement of the database, and calculating target total costs corresponding to the query plan trees; the target total cost comprises a processor cost, a scanning disk cost, a communication cost and a hardware calculation cost of the heterogeneous hardware; and the optimal plan tree is determined according to the target total cost to process the query statement. By using the heterogeneous hardware to execute intensive operation involved in the database query statement, the computing efficiency of the database is improved, the response time of the database is reduced, the server CPU resource can be released, and the problem of insufficient database computing power is solved.

Description

Technical Field

[0001] This invention relates to the field of database technology, and in particular to a database query processing method, apparatus, device, and storage medium. Background Technology

[0002] With the widespread application of relational databases such as OLTP (On-Line Transaction Processing) and OLAP (On-Line Analytical Processing), business systems have placed higher demands on the response time of database servers. As the amount of data grows, traditional server architectures such as x86 can no longer meet the computing resource requirements of databases, and insufficient server computing power has become a major problem faced by databases. Summary of the Invention

[0003] This invention provides a database query processing method, apparatus, device, and storage medium to address the shortcomings of insufficient computing power in existing database servers, thereby accelerating the database and reducing its response time.

[0004] This invention provides a database query processing method applied to a database, the database including heterogeneous hardware, extension plugins, and storage engine plugins; the extension plugins and the storage engine plugins are used to establish a connection between the heterogeneous hardware and the database server; the heterogeneous hardware is used to execute the intensive computations involved in the database query statements; the method includes:

[0005] Obtain the query statement from the database, and generate multiple query plan trees based on the query statement;

[0006] Calculate the target total cost corresponding to each of the query plan trees; the target total cost is the total cost required to process the query statement according to the query plan tree; the total cost includes processor cost, disk scan cost, communication cost, and hardware computing cost of the heterogeneous hardware;

[0007] The optimal plan tree is determined from the plurality of query trees based on the target total cost, and the query statement is processed based on the optimal plan tree.

[0008] According to the database query processing method provided by the present invention, the database includes an executor adaptation layer, wherein an executor is adapted in the executor adaptation layer; the step of processing the query statement according to the optimal plan tree includes:

[0009] Using the executor adapted in the executor adaptation layer, read the page data of the data table to be queried by the query statement;

[0010] The page data is encapsulated to obtain the data packet corresponding to the page data;

[0011] The data packet is sent to the heterogeneous hardware to invoke the heterogeneous hardware to process the query statement according to the optimal plan tree.

[0012] According to the database query processing method provided by the present invention, the database further includes a hardware driver layer, which is used to provide the underlying operation interface of the heterogeneous hardware, manage the memory and data transmission of the heterogeneous hardware, and start the computing resources of the heterogeneous hardware to call the heterogeneous hardware to perform calculations.

[0013] The step of sending the data packet to the heterogeneous hardware to invoke the heterogeneous hardware to process the query statement according to the optimal plan tree includes:

[0014] The hardware driver layer calls the operation interface of the heterogeneous hardware and sends the data packet to the buffer of the heterogeneous hardware through the operation interface; the operation interface includes a computing interface.

[0015] The computing interface of the heterogeneous hardware is activated to call the heterogeneous hardware to perform the computing operations corresponding to each node in the optimal plan tree according to the data packet, and to process the query statement.

[0016] According to the database query processing method provided by the present invention, the step of calling the heterogeneous hardware to execute the calculation operations corresponding to each node in the optimal plan tree based on the data packet includes:

[0017] The data in the data packet is parsed into line data;

[0018] Obtain the computational strategy corresponding to the optimal plan tree, and determine the target operator corresponding to the computational strategy;

[0019] Based on the row data and the target operator, the heterogeneous hardware is invoked to execute the computational operations corresponding to each node in the optimal plan tree.

[0020] According to the database query processing method provided by the present invention, the database further includes a query engine adaptation layer, wherein the query engine adaptation layer adapts to a query engine; the step of generating multiple query plan trees based on the query statement includes:

[0021] The query statement is parsed into a syntax tree by the query engine adapted by the query engine adaptation layer; the query engine includes a queryer and an optimizer.

[0022] Using the queryer and the optimizer, the syntax tree is converted into multiple query trees;

[0023] Multiple query plan trees are generated based on the multiple query trees.

[0024] According to the database query processing method provided by the present invention, the plurality of query plan trees includes a first query plan tree and a second query plan tree, and the step of generating a plurality of query plan trees based on the plurality of query trees includes:

[0025] From the plurality of query trees, determine a first query tree that requires calling the heterogeneous hardware and a second query tree that does not require calling the heterogeneous hardware;

[0026] The hardware computation processing of the heterogeneous hardware and the corresponding hardware computation cost are added to the first query tree to generate a first query plan tree corresponding to the first query tree; and a second query plan tree corresponding to the second query tree is generated.

[0027] According to the database query processing method provided by the present invention, the database further includes a data query layer for receiving the query statement; after processing the query statement according to the optimal plan tree, the method further includes:

[0028] Obtain the processing result of the query statement and return the processing result to the data query layer.

[0029] The present invention also provides a database query processing apparatus, comprising:

[0030] The statement processing module is used to obtain the query statement from the database and generate multiple query plan trees based on the query statement;

[0031] The cost calculation module is used to calculate the target total cost corresponding to each of the query plan trees; the target total cost is the total cost required to process the query statement according to the query plan tree; the total cost includes processor cost, disk scan cost, communication cost and hardware computing cost of the heterogeneous hardware;

[0032] The query processing module is used to determine the optimal plan tree from the plurality of query trees based on the target total cost, and to process the query statement based on the optimal plan tree.

[0033] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a database query processing method as described above.

[0034] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a database query processing method as described above.

[0035] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a database query processing method as described above.

[0036] The database query processing method, apparatus, device, and storage medium provided by this invention generate multiple query plan trees corresponding to a database query statement and calculate the target total cost of each query plan tree, thereby determining the optimal plan data from the multiple query plan trees. Processing the query statement according to this optimal plan tree can improve the database query execution efficiency. The database includes heterogeneous hardware, extension plugins, and storage engine plugins. The extension plugins and storage engine plugins are used to establish a connection between the heterogeneous hardware and the database server, offloading the database's intensive computations to the heterogeneous hardware, which then performs these computations. By offloading the intensive computations involved in the database query statement to the heterogeneous hardware and selecting the optimal plan tree based on the total cost required to process the query statement, the database's computational efficiency can be improved, its response time reduced, and its response speed increased. Using heterogeneous hardware can free up server CPU resources, solving the problem of insufficient computing power on the database server. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0038] Figure 1 This is a flowchart illustrating the database query processing method provided by the present invention;

[0039] Figure 2 This is a schematic diagram of the processing flow of the database query engine provided by the present invention;

[0040] Figure 3 This is a schematic diagram of the structure of the database query processing device provided by the present invention;

[0041] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0043] The following is combined with Figures 1-2 The present invention describes a database query processing method.

[0044] The database query processing method provided by this invention is applied to a relational database, which includes heterogeneous hardware, extension plugins, and storage engine plugins. The extension plugins and storage engine plugins are used to establish the connection between the heterogeneous hardware and the database server. By using the database extension plugins and storage engine plugins, the intensive computations involved in the database query statements are offloaded to the heterogeneous hardware for execution. The heterogeneous hardware is used to execute the intensive computations involved in the database query statements. By using heterogeneous hardware to participate in the database server's computation, the database's computational efficiency is improved, and the CPU resources of the database server are freed up for use by the business system, thereby accelerating the database. Specifically, refer to... Figure 1 , Figure 1 This is a flowchart illustrating the database query processing method provided in an embodiment of the present invention, based on... Figure 1 The database query processing method provided in this embodiment of the invention includes:

[0045] Step 100: Obtain the query statement from the database and generate multiple query plan trees based on the query statement;

[0046] In this embodiment, the database includes heterogeneous hardware used to execute the intensive computations involved in database queries. Heterogeneous hardware is generally not directly compatible with traditional database servers and requires storage layer adaptation to offload the database's intensive computations to the heterogeneous hardware. Therefore, database extension plugins and storage engine plugins are needed to establish the connection between the heterogeneous hardware and the database server. The intensive computations involved in database queries, such as joins, aggregations, sorting, and filtering, can be offloaded to the heterogeneous hardware for processing. Leveraging the parallel processing capabilities and advantages of heterogeneous hardware improves the database's computational efficiency, reduces database query response time, and frees up CPU resources, thus accelerating the database.

[0047] Upon receiving a database query statement, multiple query plan trees are first generated based on the query statement. Different query plan trees correspond to different data query strategies. Each query plan tree includes one or more data tables that the query statement needs to query, as well as the operations to be performed on these tables, such as joins, aggregations, sorting, and filtering. Based on this, the data query strategy includes the query and operation order of the data tables to be queried, as well as the resources to be invoked and the resource invocation strategy. In other words, depending on the one or more data tables that the query statement needs to query and the related operations to be performed on them, different resources can be invoked to process the query task corresponding to the query statement in response to the received query statement. Based on different query strategies for the data to be queried, multiple query plan trees corresponding to the query statement are generated. Each generated query plan tree includes computational processing nodes on heterogeneous hardware for the intensive operations involved in the query statement.

[0048] Step 200: Calculate the target total cost corresponding to each of the query plan trees; the target total cost is the total cost required to process the query statement according to the query plan tree; the total cost includes processor cost, disk scan cost, communication cost, and hardware computing cost of the heterogeneous hardware;

[0049] The target total cost corresponding to each query plan tree is calculated. This target total cost is a relative unit of measurement used to measure the resource and time costs required to execute the query task corresponding to the query statement. For example, the cost of CPU computing a row of data is 0.01, and the I / O cost of scanning a page is 1.0. The total cost corresponding to each query plan tree includes processor cost (CPU computing cost), disk scanning cost (I / O cost), communication cost, and hardware computing cost of heterogeneous hardware; the total cost is the sum of CPU computing cost, disk scanning cost, communication cost, and hardware computing cost of heterogeneous hardware. When executing the query task corresponding to the query statement according to different query plan trees, different resources are called and different query strategies are used to perform queries and related operations on the data to be queried. The resources and time required are different, and therefore the total cost is also different.

[0050] Step 300: Determine the optimal plan tree from the plurality of query plan trees based on the target total cost, and process the query statement based on the optimal plan tree.

[0051] Based on the target total cost corresponding to each query plan tree, an optimal plan tree with the lowest total cost is determined from multiple query plan trees. The query statement is then processed according to this optimal plan tree. Query processing, i.e., executing the query task corresponding to the query statement, requires minimal query cost, highest query execution efficiency, and / or shortest database response time when executing the query task based on the optimal plan tree.

[0052] In this embodiment, by generating multiple query plan trees corresponding to the database query statements and calculating the target total cost of each query plan tree, the optimal plan data is determined from the multiple query plan trees. Processing the query statement according to this optimal plan tree can improve the database query execution efficiency. The database includes heterogeneous hardware, extension plugins, and storage engine plugins. The extension plugins and storage engine plugins are used to establish the connection between the heterogeneous hardware and the database server, offloading the database-intensive computations to the heterogeneous hardware, which then performs these computations. By offloading the database-intensive computations to the heterogeneous hardware and selecting the optimal plan tree based on the total cost required to process the query statement, the database's computational efficiency can be improved, its response time reduced, and its response speed increased. Using heterogeneous hardware can free up the database server's CPU resources, solving the problem of insufficient database server computing power.

[0053] Furthermore, the database also includes an executor adaptation layer, which adapts one or more executors. In step 300, the query statement is processed according to the optimal plan tree, specifically including:

[0054] Step 301: Using the executor adapted in the executor adaptation layer, read the page data of the data table to be queried by the query statement;

[0055] Step 302: Encapsulate the page data to obtain the data packet corresponding to the page data;

[0056] Step 303: Send the data packet to the heterogeneous hardware to invoke the heterogeneous hardware to process the query statement according to the optimal plan tree.

[0057] In this embodiment, the database executor adaptation layer is equipped with a database executor for performing database-related calculation operations. Specifically, it performs calculation operations on each node in the generated query plan tree, such as joining multiple data tables, grouping column data, summing, and condition filtering.

[0058] When processing a query, the executor reads the page data of the data table to be queried. This page data contains information about the data table within a page in the database. The read page data is then encapsulated to obtain a corresponding data packet. To improve data transmission efficiency, the size of this data packet is at least a preset threshold. The data packet is then sent to heterogeneous hardware, specifically to its buffer, thereby invoking the heterogeneous hardware to process the query according to the optimal plan tree.

[0059] Furthermore, the database also includes a hardware driver layer, which provides low-level operation interfaces for heterogeneous hardware, including computation interfaces. The hardware driver layer also manages the memory and data transmission of the heterogeneous hardware, and initiates the computational resources of the heterogeneous hardware to invoke its participation in computation. In step 303, the data packet is sent to the heterogeneous hardware to invoke it to process the query statement according to the optimal plan tree, specifically including:

[0060] Step 3031: The operation interface of the heterogeneous hardware is called through the hardware driver layer, and the data packet is sent to the buffer of the heterogeneous hardware through the operation interface; the operation interface includes a computing interface;

[0061] Step 3032: Start the computing interface of the heterogeneous hardware to call the heterogeneous hardware to perform the computing operations corresponding to each node in the optimal plan tree according to the data packet, and process the query statement.

[0062] The database's hardware driver layer calls the heterogeneous hardware's operation interface, which sends data packets to the heterogeneous hardware's buffer. This operation interface includes a computation interface. The heterogeneous hardware's computation interface is then activated, invoking its computational resources. Based on the data packets cached in the heterogeneous hardware's buffer, the computational operations corresponding to each node in the optimal query plan tree are executed to process the query statement. It's important to note that the optimal query plan tree consists of multiple nodes, each executing a portion of the query task corresponding to the query statement. Therefore, when processing the query statement, the computational resources of the heterogeneous hardware need to be invoked to execute the computational operations corresponding to each node in the optimal query plan tree.

[0063] Furthermore, the hardware driver layer provides the underlying operation interface for the heterogeneous hardware and sends execution commands downwards to complete the connection calculations between the database server host and the heterogeneous hardware. The hardware driver layer also handles read / write notification signals from the heterogeneous hardware. The hardware driver layer includes a hardware kernel computing unit, which parses page data in data packets and performs database operator calculations, executing corresponding computational operations.

[0064] In step 3032, the heterogeneous hardware is invoked to perform the computational operations corresponding to each node in the optimal plan tree according to the data packet, specifically including:

[0065] Step 310: Parse the data in the data packet into line data;

[0066] Step 320: Obtain the computational strategy corresponding to the optimal plan tree, and determine the target operator corresponding to the computational strategy;

[0067] Step 330: Based on the row data and the target operator, call the heterogeneous hardware to execute the calculation operations corresponding to each node in the optimal plan tree.

[0068] The data in the data packets cached in the heterogeneous hardware cache is parsed into row data to obtain the computation strategy corresponding to the optimal plan tree, and the target operator corresponding to the computation strategy is determined. The target operator includes one or more, with different operators corresponding to different types of operations, including join operators, aggregation operators, sorting operators, and filtering operators, used to represent different types of computational operations on the data table to be queried. If multiple target operators exist, the computational operations corresponding to each target operator are executed according to the optimal plan tree. This can be sequential, parallel, or a combination of both. When executing sequentially, after completing the computational operation corresponding to the current operator, the calculation result is sent to the next operator, thus executing the computational operation corresponding to the next operator. No specific restrictions are placed on the execution method for multiple target operators.

[0069] Preferably, the processing steps of the actuator specifically include:

[0070] 1. Read the page data of the data table to be queried by the query statement, and encapsulate the read page data into a large data packet. In order to improve data transmission efficiency, the size of the encapsulated data packet exceeds the preset threshold.

[0071] 2. Call the operation interface of the heterogeneous hardware and transfer the data to the hardware buffer;

[0072] 3. Activate the computing interface of the heterogeneous hardware and call upon the heterogeneous hardware to participate in the computation;

[0073] 4. Wait to receive a notification signal indicating that the hardware calculation is complete;

[0074] 5. Upon receiving a notification signal indicating that the hardware calculation is complete, read the return result of the hardware calculation and return it to the data query layer.

[0075] The hardware kernel computing unit in the hardware driver layer processes the query statement as follows:

[0076] 1. Parse the data in the data packet into line data;

[0077] 2. Select the corresponding objective operator based on the computational strategy corresponding to the optimal plan tree;

[0078] 3. If multiple operators exist, after completing the calculation of the current operator, the calculation is passed to the next operator for calculation;

[0079] 4. Summarize the calculation results of each operator and return them to the database server host.

[0080] The database also includes a data query layer, which receives query statements from the database, including SQL statements. After processing the received query statements, it also includes:

[0081] Step 400: Obtain the processing result of the query statement and return the processing result to the data query layer.

[0082] The system retrieves the processing result of the query statement and returns it to the data query layer. Specifically, when retrieving the processing result, the hardware driver layer aggregates the calculation results of each target operator into the query statement's processing result and sends a completion notification through the heterogeneous hardware's operation interface, instructing the executor to read the processing result. After sending the data packet to the heterogeneous hardware's buffer and invoking the heterogeneous hardware to participate in the calculation, the executor waits for the hardware's completion notification. Upon receiving this notification, the executor reads the processing result returned by the heterogeneous hardware and returns it to the data query layer, thus responding to the query statement and completing the corresponding query task.

[0083] In this embodiment, by encapsulating the page data of the data packet to be queried by the query statement, the corresponding data packet is obtained, which can improve data transmission efficiency. Based on the database executor adaptation layer and hardware driver layer, heterogeneous hardware is called to participate in the calculation according to the data packet, and the calculation operations corresponding to each node in the optimal plan tree are executed. The intensive calculations involved in the database query statement can be offloaded to heterogeneous hardware for execution, thereby releasing the database processor resources, improving calculation efficiency, reducing response time, and improving the database response speed.

[0084] Reference Figure 2 The schematic diagram of the query engine processing flow shown illustrates that, in one embodiment, the database further includes a query engine adaptation layer, which adapts one or more query engines. The query tree corresponding to the query statement is generated within the query engine. Specifically, step 100, generating the query tree based on the query statement, includes:

[0085] Step 101: Parse the query statement into a syntax tree using the query engine adapted by the query engine adaptation layer; the query engine includes a queryer and an optimizer.

[0086] Step 102: Using the queryer and the optimizer, the syntax tree is converted into multiple query trees;

[0087] Step 103: Generate multiple query plan trees based on the multiple query trees.

[0088] Based on the query engine adapted in the database's query engine adaptation layer, the obtained query statement is parsed into a syntax tree. This syntax tree is a graphical representation of the query statement structure, displaying the data tables to be queried and the calculation operations to be performed on each data table. The query engine adapted in the query engine adaptation layer includes a query engine and an optimizer. Based on the parsed syntax tree, the query engine and optimizer process it to transform the syntax tree into multiple query trees.

[0089] Furthermore, the multiple query plan trees include a first query plan tree and a second query plan tree, based on Figure 2 In step 103, multiple query plan trees are generated based on multiple query trees, including:

[0090] Step 1031: Determine from the plurality of query trees a first query tree that needs to call the heterogeneous hardware and a second query tree that does not need to call the heterogeneous hardware;

[0091] Step 1032: Add the hardware computation processing of the heterogeneous hardware and the hardware computation cost corresponding to the heterogeneous hardware to the first query tree to generate a first query plan tree corresponding to the first query tree; and generate a second query plan tree corresponding to the second query tree.

[0092] From the generated multiple query trees, a first query tree requiring heterogeneous hardware and a second query tree not requiring heterogeneous hardware are determined. Both the first and second query trees can include one or more query trees. Hardware computing processing nodes are added to the first query tree, generating a first query plan tree corresponding to the first query tree, and a second query plan tree corresponding to the second query tree, thus obtaining multiple query plan trees. For each generated query tree, the optimizer calculates the cost of each stage, including processor computation cost, I / O cost, and communication cost; the calculated costs are added to the query tree to obtain the corresponding query plan tree. For the first query tree requiring hardware computing processing, the corresponding hardware computing cost needs to be added to its corresponding first query plan tree. The first query tree requiring hardware computing processing is a query tree containing sub-query tasks that need to be offloaded to heterogeneous hardware for computation; these sub-query tasks involve intensive operations that need to be offloaded to heterogeneous hardware for execution.

[0093] It is understandable that when processing a query statement, heterogeneous hardware can be invoked for computation, or heterogeneous hardware can be omitted. Therefore, the generated multiple query plan trees may include a first query plan tree that requires the addition of computing processing nodes with heterogeneous hardware, or a second query plan tree that does not require the addition of computing processing nodes with heterogeneous hardware.

[0094] In this embodiment, multiple query trees are generated based on the query statement through a query engine adapted to the database. These trees are then processed by an optimizer, and hardware computation is added to the query trees to generate corresponding query plan trees. This enables the use of heterogeneous hardware, allowing the intensive computations involved in the query statement to be offloaded to heterogeneous hardware for execution. This improves the database query efficiency by accelerating the query process using heterogeneous hardware.

[0095] The database query processing apparatus provided by the present invention will be described below. The database query processing apparatus described below can be referred to in correspondence with the database query processing method described above.

[0096] Reference Figure 3 The database query processing apparatus provided in this embodiment of the invention includes:

[0097] The statement processing module 10 is used to obtain the query statement of the database and generate multiple query plan trees based on the query statement;

[0098] The cost calculation module 20 is used to calculate the target total cost corresponding to each of the query plan trees; the target total cost is the total cost required to process the query statement according to the query plan tree; the total cost includes processor cost, disk scan cost, communication cost and hardware computing cost of the heterogeneous hardware;

[0099] The query processing module 30 is used to determine the optimal plan tree from the plurality of query trees based on the target total cost, and to process the query statement based on the optimal plan tree.

[0100] In one embodiment, the database includes an executor adaptation layer, wherein executors are adapted in the executor adaptation layer; the query processing module 30 is further configured to:

[0101] Using the executor adapted in the executor adaptation layer, read the page data of the data table to be queried by the query statement;

[0102] The page data is encapsulated to obtain the data packet corresponding to the page data;

[0103] The data packet is sent to the heterogeneous hardware to invoke the heterogeneous hardware to process the query statement according to the optimal plan tree.

[0104] In one embodiment, the database further includes a hardware driver layer, which is used to provide the low-level operation interface of the heterogeneous hardware, manage the memory and data transmission of the heterogeneous hardware, and start the computing resources of the heterogeneous hardware to call the heterogeneous hardware for computation; the query processing module 30 is further used to:

[0105] The hardware driver layer calls the operation interface of the heterogeneous hardware and sends the data packet to the buffer of the heterogeneous hardware through the operation interface; the operation interface includes a computing interface.

[0106] The computing interface of the heterogeneous hardware is activated to call the heterogeneous hardware to perform the computing operations corresponding to each node in the optimal plan tree according to the data packet, and to process the query statement.

[0107] In one embodiment, the query processing module 30 is further configured to:

[0108] The data in the data packet is parsed into line data;

[0109] Obtain the computational strategy corresponding to the optimal plan tree, and determine the target operator corresponding to the computational strategy;

[0110] Based on the row data and the target operator, the heterogeneous hardware is invoked to execute the computational operations corresponding to each node in the optimal plan tree.

[0111] In one embodiment, the database further includes a query engine adaptation layer, which adapts to a query engine; the statement processing module 10 is further configured to:

[0112] The query statement is parsed into a syntax tree by the query engine adapted by the query engine adaptation layer; the query engine includes a queryer and an optimizer.

[0113] Using the queryer and the optimizer, the syntax tree is converted into multiple query trees;

[0114] Multiple query plan trees are generated based on the multiple query trees.

[0115] In one embodiment, the plurality of query plan trees includes a first query plan tree and a second query plan tree; the statement processing module 10 is further configured to:

[0116] From the plurality of query trees, determine a first query tree that requires calling the heterogeneous hardware and a second query tree that does not require calling the heterogeneous hardware;

[0117] The hardware computation processing of the heterogeneous hardware and the corresponding hardware computation cost are added to the first query tree to generate a first query plan tree corresponding to the first query tree; and a second query plan tree corresponding to the second query tree is generated.

[0118] In one embodiment, the database further includes a data query layer for receiving the query statement; the database query processing device further includes a query response module for:

[0119] Obtain the processing result of the query statement and return the processing result to the data query layer.

[0120] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a database query processing method, which includes:

[0121] Obtain the query statement from the database, and generate multiple query plan trees based on the query statement;

[0122] Calculate the target total cost corresponding to each of the query plan trees; the target total cost is the total cost required to process the query statement according to the query plan tree; the total cost includes processor cost, disk scan cost, communication cost, and hardware computing cost of the heterogeneous hardware;

[0123] The optimal plan tree is determined from the plurality of query trees based on the target total cost, and the query statement is processed based on the optimal plan tree.

[0124] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0125] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the database query processing method provided by the above methods, the method comprising:

[0126] Obtain the query statement from the database, and generate multiple query plan trees based on the query statement;

[0127] Calculate the target total cost corresponding to each of the query plan trees; the target total cost is the total cost required to process the query statement according to the query plan tree; the total cost includes processor cost, disk scan cost, communication cost, and hardware computing cost of the heterogeneous hardware;

[0128] The optimal plan tree is determined from the plurality of query trees based on the target total cost, and the query statement is processed based on the optimal plan tree.

[0129] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a database query processing method provided by the methods described above, the method comprising:

[0130] Obtain the query statement from the database, and generate multiple query plan trees based on the query statement;

[0131] Calculate the target total cost corresponding to each of the query plan trees; the target total cost is the total cost required to process the query statement according to the query plan tree; the total cost includes processor cost, disk scan cost, communication cost, and hardware computing cost of the heterogeneous hardware;

[0132] The optimal plan tree is determined from the plurality of query trees based on the target total cost, and the query statement is processed based on the optimal plan tree.

[0133] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0134] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A database query processing method, applied to a database, characterized in that, The database includes heterogeneous hardware, extension plugins, and storage engine plugins. The extension plugins and storage engine plugins are used to establish a connection between the heterogeneous hardware and the database server. The heterogeneous hardware is used to perform intensive computations involved in the database's query statements. The method includes the following steps: The system obtains a query statement from the database and generates multiple query plan trees based on the query statement. The database also includes a query engine adaptation layer, which adapts to a query engine. The step of generating multiple query plan trees based on the query statement includes: parsing the query statement into a syntax tree using the query engine adapted by the query engine adaptation layer; the query engine includes a queryer and an optimizer; using the queryer and the optimizer, the syntax tree is converted into multiple query trees; multiple query plan trees are generated based on the multiple query trees; the multiple query plan trees include a first query plan tree and a second query plan tree. The step of generating multiple query plan trees based on the multiple query trees includes: determining from the multiple query trees a first query tree that needs to call the heterogeneous hardware and a second query tree that does not need to call the heterogeneous hardware; adding the hardware computation processing of the heterogeneous hardware and the hardware computation cost corresponding to the heterogeneous hardware to the first query tree to generate a first query plan tree corresponding to the first query tree; and generating a second query plan tree corresponding to the second query tree. Calculate the target total cost corresponding to each of the query plan trees; the target total cost is the total cost required to process the query statement according to the query plan tree; the total cost includes processor cost, disk scan cost, communication cost, and hardware computing cost of the heterogeneous hardware; The optimal plan tree is determined from the plurality of query trees based on the target total cost, and the query statement is processed based on the optimal plan tree; The database includes an executor adaptation layer, in which executors are adapted; the step of processing the query statement according to the optimal plan tree includes: Using the executor adapted in the executor adaptation layer, the page data of the data table to be queried by the query statement is read; the page data is encapsulated to obtain the data packet corresponding to the page data; the data packet is sent to the heterogeneous hardware to call the heterogeneous hardware to process the query statement according to the optimal plan tree.

2. The database query processing method according to claim 1, characterized in that, The database also includes a hardware driver layer, which is used to provide the underlying operation interface of the heterogeneous hardware, manage the memory and data transmission of the heterogeneous hardware, and start the computing resources of the heterogeneous hardware to call the heterogeneous hardware to perform calculations. The step of sending the data packet to the heterogeneous hardware to invoke the heterogeneous hardware to process the query statement according to the optimal plan tree includes: The hardware driver layer calls the operation interface of the heterogeneous hardware and sends the data packet to the buffer of the heterogeneous hardware through the operation interface; the operation interface includes a computing interface. The computing interface of the heterogeneous hardware is activated to call the heterogeneous hardware to perform the computing operations corresponding to each node in the optimal plan tree according to the data packet, and to process the query statement.

3. The database query processing method according to claim 2, characterized in that, The step of invoking the heterogeneous hardware to execute the computational operations corresponding to each node in the optimal plan tree based on the data packet includes: The data in the data packet is parsed into line data; Obtain the computational strategy corresponding to the optimal plan tree, and determine the target operator corresponding to the computational strategy; Based on the row data and the target operator, the heterogeneous hardware is invoked to execute the computational operations corresponding to each node in the optimal plan tree.

4. The database query processing method according to claim 1, characterized in that, The database further includes a data query layer for receiving the query statement; after processing the query statement according to the optimal plan tree, the process further includes: Obtain the processing result of the query statement and return the processing result to the data query layer.

5. A database query processing apparatus, characterized in that, include: The statement processing module is used to obtain the query statement from the database and generate multiple query plan trees based on the query statement; The database also includes a query engine adaptation layer, which is adapted to a query engine; The step of generating multiple query plan trees based on the query statement includes: parsing the query statement into a syntax tree using a query engine adapted by the query engine adaptation layer; the query engine includes a queryer and an optimizer; using the queryer and the optimizer, the syntax tree is converted into multiple query trees; multiple query plan trees are generated based on the multiple query trees; the multiple query plan trees include a first query plan tree and a second query plan tree; the step of generating multiple query plan trees based on the multiple query trees includes: determining from the multiple query trees a first query tree that needs to call heterogeneous hardware and a second query tree that does not need to call the heterogeneous hardware; adding the hardware computation processing of the heterogeneous hardware and the hardware computation cost corresponding to the heterogeneous hardware to the first query tree to generate a first query plan tree corresponding to the first query tree; and generating a second query plan tree corresponding to the second query tree; The cost calculation module is used to calculate the target total cost corresponding to each of the query plan trees; the target total cost is the total cost required to process the query statement according to the query plan tree; the total cost includes processor cost, disk scan cost, communication cost and hardware computing cost of the heterogeneous hardware; A query processing module is used to determine the optimal plan tree from the plurality of query trees based on the target total cost, and to process the query statement according to the optimal plan tree; the database includes an executor adaptation layer, in which executors are adapted; the step of processing the query statement according to the optimal plan tree includes: Using the executor adapted in the executor adaptation layer, the page data of the data table to be queried by the query statement is read; the page data is encapsulated to obtain the data packet corresponding to the page data; the data packet is sent to the heterogeneous hardware to call the heterogeneous hardware to process the query statement according to the optimal plan tree.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the database query processing method as described in any one of claims 1 to 5.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the database query processing method as described in any one of claims 1 to 5.

Images