A data query method, a data query device and a computing device cluster

Abstract

This paper provides a data query method, a data query device, and a computing device cluster. In this method, multiple execution plans for a user's Structured Query Language (SQL) statement are generated. Then, a performance evaluation model is used to determine the predicted performance values ​​of each execution plan. Next, based on the predicted performance values, multiple first execution plans that meet the performance requirements are selected from the multiple execution plans. Then, based on the predicted cost values ​​of each first execution plan, a second execution plan is selected from the multiple first execution plans. Finally, the second execution plan is executed to obtain the query results corresponding to the SQL statement. This method can solve the problem of poor database query performance and improve user experience.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data query method, a data query device, and a computing device cluster. Background Technology

[0002] Current databases typically optimize user-submitted Structured Query Language (SQL) statements by comparing the costs of various plans and selecting the lowest-cost one as the target plan, then using that plan to retrieve the query results. However, this optimization approach only considers cost and cannot guarantee that the database's query performance will always be optimal. In other words, even if the lowest-cost execution plan is selected, the database may still experience poor performance during actual execution, impacting the user experience. Summary of the Invention

[0003] This application provides a data query method, a data query device, and a computing device cluster, which can solve the problem of poor database query performance and improve user experience.

[0004] Firstly, this application provides a data query method. This method can be applied to a database to respond to user queries. The method may include: generating multiple execution plans for an SQL statement based on the user's Structured Query Language (SQL) statement; using a performance evaluation model to determine the predicted performance values ​​of each execution plan among the multiple execution plans; then, based on the predicted performance values ​​of each execution plan, determining multiple first execution plans that meet the performance requirements from the multiple execution plans; and based on the predicted cost values ​​of the multiple first execution plans, determining second execution plans from the multiple first execution plans; and executing the second execution plan to obtain the query results corresponding to the SQL statement.

[0005] In the above scheme, the database predicts the performance of each execution plan and selects the execution plan that meets the performance and cost requirements to execute, which can improve the database query performance and thus enhance the user experience.

[0006] In one possible implementation, the aforementioned performance may include accuracy and / or latency.

[0007] In the above scheme, the database can predict the performance of the execution plan from two dimensions: accuracy and / or latency. Therefore, the second execution plan executed by the database can meet the accuracy and / or latency requirements. If the second execution plan meets the accuracy requirements, the accuracy of the query results can be improved. If the second execution plan meets the accuracy requirements, the query time can be shortened, thereby reducing user waiting time.

[0008] In one possible implementation, where performance includes accuracy, the performance evaluation model includes an accuracy prediction model. The aforementioned method of using the performance evaluation model to determine the predicted performance of each execution plan among multiple execution plans includes: inputting the index types corresponding to the data columns related to the SQL statements included in each execution plan into the accuracy prediction model to obtain the predicted accuracy of each execution plan.

[0009] In one possible implementation, the inputs to the aforementioned accuracy prediction model further include: the join relationships between data tables related to the SQL statement, and the selectivity of data columns related to the SQL statement. The data tables related to the SQL statement can include the data tables accessed by the SQL statement, and the data columns related to the SQL statement can include data columns from the data tables related to the SQL statement. Optionally, the SQL statement can include the table identifier of the data table accessed by the SQL statement. The SQL statement can also include column identifiers corresponding to the table identifier. Therefore, the database can determine the data tables related to the SQL statement and / or the data columns related to the SQL statement based on the table identifiers and / or column identifiers in the SQL statement. Where the SQL statement does not include column identifiers corresponding to the table identifiers, the database can use all or some data columns from the data tables related to the SQL statement as the data columns related to the SQL statement.

[0010] In one possible implementation, the above method may further include: receiving the actual value of the precision of the query results as feedback from the user; and updating the parameters of the precision prediction model based on the actual value of the precision.

[0011] In one possible implementation, where performance includes latency, the performance evaluation model includes a latency prediction model. The aforementioned method of using the performance evaluation model to determine the predicted performance of each execution plan among multiple execution plans includes: inputting the operators included in each execution plan and the amount of data processed by each operator into the latency prediction model to obtain the predicted latency of each execution plan.

[0012] In one possible implementation, the above method further includes receiving a user's SQL statement, which includes a performance requirement value indicating performance requirements.

[0013] In the above solution, the user carries performance requirements in the SQL statement, so that the second execution plan of the database can meet the user's performance requirements.

[0014] Secondly, this application provides a data query apparatus. This data query apparatus can be applied to a database. The apparatus may include: a generation module, an evaluation module, and a determination module.

[0015] The generation module is used to generate multiple execution plans for a user's Structured Query Language (SQL) statement.

[0016] The evaluation module is used to determine the predicted performance of each execution plan among the multiple execution plans using a performance evaluation model.

[0017] The determining module is configured to determine a plurality of first execution plans that meet the performance requirements from the plurality of execution plans based on the predicted performance values ​​of each execution plan, and to determine a second execution plan from the plurality of first execution plans based on the predicted cost values ​​of each first execution plan.

[0018] The execution module is used to execute the second execution plan and obtain the query results corresponding to the SQL statement.

[0019] In one possible implementation, this performance includes accuracy and / or latency.

[0020] In one possible implementation, where the performance includes accuracy, the performance evaluation model includes an accuracy prediction model, and the evaluation module is further configured to: input the index type corresponding to the data columns related to the SQL statement included in each execution plan into the accuracy prediction model to obtain the predicted value of the accuracy of each execution plan.

[0021] In one possible implementation, the inputs to the accuracy prediction model also include: the join relationships between the data tables related to the SQL statement, and the selectivity of the data columns related to the SQL statement.

[0022] In one possible implementation, the evaluation module is also used to: receive the actual value of the accuracy of the query result as reported by the user; and update the parameters of the accuracy prediction model based on the actual value of the accuracy.

[0023] In one possible implementation, where the performance includes latency, the performance evaluation model includes a latency prediction model, and the evaluation module is further configured to: input the operators included in each execution plan and the amount of data processed by each operator into the latency prediction model to obtain the predicted latency value of each execution plan.

[0024] In one possible implementation, the device further includes a receiving module for receiving the user's SQL statement, which includes a performance requirement value indicating the performance requirement.

[0025] Thirdly, this application also provides a computing device cluster. The cluster may include at least one computing device, each computing device including a processor and a memory. The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, causing the computing device cluster to perform the methods provided by the first aspect or any possible implementation thereof.

[0026] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium includes computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method provided by the first aspect or any possible implementation thereof.

[0027] Fifthly, this application also provides a computer program product. This computer program product includes computer program instructions. When the computer program instructions are executed by a cluster of computing devices, the cluster of computing devices performs the method provided by the first aspect or any possible implementation thereof.

[0028] Any of the devices, computing equipment, computing equipment clusters, computer storage media, or computer program products provided above are used to execute the methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects of the corresponding solutions in the corresponding methods provided above, and will not be repeated here. Attached Figure Description

[0029] Figure 1 This is a schematic diagram illustrating an application scenario of database responding to user queries, provided in an embodiment of this application.

[0030] Figure 2 This application provides a method for applying... Figure 1 The flowchart shown illustrates the data query method in the database.

[0031] Figure 3 This is one of the embodiments provided in this application. Figure 1 The diagram shows the functionality of the optimizer in the database.

[0032] Figure 4a and Figure 4b This is a functional schematic diagram of the performance evaluation model provided in the embodiments of this application;

[0033] Figure 4c This is a functional schematic diagram of a cost prediction model provided in an embodiment of this application;

[0034] Figure 5 This is a schematic diagram of an operator included in an execution plan provided in an embodiment of this application;

[0035] Figure 6 This is a schematic diagram of the encoding information of the prediction model for determining input accuracy provided in an embodiment of this application;

[0036] Figure 7 This is a schematic diagram of the input data for the latency prediction model provided in the embodiments of this application.

[0037] Figure 8 This is a schematic diagram illustrating how the optimizer provided in this application selects execution plans based on accuracy prediction values ​​and latency prediction values;

[0038] Figure 9 This application provides a method based on... Figure 2 The flowchart of the data query method shown is as follows;

[0039] Figure 10 This is a training diagram of a performance evaluation model and a cost prediction model provided in an embodiment of this application;

[0040] Figure 11 This is a schematic diagram of the structure of a data query device provided in an embodiment of this application;

[0041] Figure 12 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0042] Figure 13 and Figure 14 This is a schematic diagram of the structure of a computing device cluster provided in an embodiment of this application. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be described below with reference to the accompanying drawings.

[0044] In the description of the embodiments of this application, the words "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the words "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a specific manner.

[0045] In the description of the embodiments in this application, the term "and / or" is merely a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, B existing alone, and A and B existing simultaneously. Furthermore, unless otherwise stated, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals.

[0046] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0047] Before introducing the embodiments of this application, the technical terms mentioned in the embodiments of this application will be explained below.

[0048] A knowledge base is a specialized database used for knowledge management. It stores a large amount of knowledge in a specific domain using a defined data structure. The knowledge in a knowledge base is typically the result of analysis and summarization of actual data within that domain; therefore, a knowledge base can provide accurate data for that domain.

[0049] An execution plan is the detailed operation information provided by the database optimizer for a user-provided Structured Query Language (SQL) statement. An execution plan can include index information for the relevant tables, multiple operators, and information about each operator. Index information can include the type of the index. Operators are the specific operations or functions executed by the database executor. Operator information can include the operator's type, attributes, and the amount of data processed. The amount of data processed by an operator can be represented by the number of rows in the relevant data columns of the table.

[0050] In the field of computer science, as the market for generative artificial intelligence (AI) continues to expand, AI applications are increasing daily. However, a significant problem—the illusion problem—has also emerged. The illusion problem refers to the situation where AI applications output content that is inconsistent with reality or completely fictitious when users engage with them. This inaccurate output can lead to misunderstandings and potentially incorrect decisions, severely impacting the reliability and usability of AI applications. To address the illusion problem, retrieval augmented generation (RAG) technology has emerged. AI applications can use RAG technology to retrieve reliable data from knowledge bases and provide users with reliable content accordingly. This effectively reduces the probability of AI applications generating illusions, thereby improving the reliability and usability of AI applications.

[0051] Figure 1 This is a schematic diagram of the structure of a database provided in an embodiment of this application. For example... Figure 1As shown, the database may include a database and multiple functional components. The database may include the aforementioned knowledge base. The multiple functional components may include, for example, […]. Figure 1 The diagram shows the optimizer, executor, and retrieval unit. The optimizer performs logical and physical optimizations on the SQL statement from the client and outputs a detailed execution plan. The executor executes the execution plan output by the optimizer to obtain the query results corresponding to the SQL statement and returns them to the user. The retrieval unit retrieves data from the storage nodes using the configured retrieval method and provides the retrieved data to the executor. The executor determines the query results corresponding to the SQL statement based on the data retrieved by the retrieval unit.

[0052] The retrieval methods may include, but are not limited to, hybrid retrieval, fusion retrieval, and federation retrieval methods built upon a basic index. The basic index may include, but is not limited to, bm25 indexes, sparse vector indexes, dense vector indexes, graph indexes, secondary indexes, and full-text indexes.

[0053] In some related technologies, optimizers typically evaluate the costs of multiple execution plans during the physical optimization phase based on a cost model, and output the lowest-cost target execution plan to the executor for execution. However, for databases, higher query precision often comes with higher costs. This approach, which only evaluates the cost of the execution plan, reduces query precision and degrades the user experience.

[0054] Therefore, this application provides a data query method. In this data query method, the database generates multiple execution plans for the user's Structured Query Language (SQL) statement. A performance evaluation model is used to determine the predicted performance values ​​of each execution plan. Then, based on the predicted performance values, multiple first execution plans that meet the performance requirements are determined from the multiple execution plans. Furthermore, based on the predicted costs of the multiple first execution plans, a second execution plan is determined from the multiple first execution plans. The second execution plan is then executed to obtain the query results corresponding to the SQL statement. Thus, the second execution plan executed by the database can meet certain performance requirements and improve the user experience.

[0055] Figure 2 This is a flowchart of a data query method provided in an embodiment of this application. For example... Figure 2As shown, the method may include S201-S206. This method can be applied to... Figure 1 The optimizer in the database shown uses a performance evaluation model to determine the target execution plan that meets the user's performance requirements from multiple execution plans, and then outputs it to the executor for execution. The following section discusses... Figure 2 The steps of the method shown will be described in detail.

[0056] S201, the optimizer obtains the user's SQL statement and determines the performance requirements based on the SQL statement.

[0057] In this embodiment, the user can send SQL statements to the database through the database client according to actual needs. After receiving the SQL statement, the database optimizer parses the SQL statement to obtain the performance requirements. The SQL statement may include performance requirement values ​​indicating the performance requirements.

[0058] In some embodiments, the database may also pre-set performance requirements or set different performance requirements for different users of the database. In this case, the user's SQL statement may not include the performance requirements. For example, the database may pre-set performance requirements for each data center included in the database, based on the actual conditions of each data center, such as traffic volume and / or network quality.

[0059] The performance requirements mentioned above can include parameters representing performance, such as precision requirements and / or latency requirements. For example, the SQL statement "table_1.output(xxx).match(precision=0.8&latency=30s)" shows a precision requirement of 0.8 and a latency requirement of 30 seconds. This indicates that the user expects the database precision to be no less than 0.8 and the latency to be no greater than 30 seconds.

[0060] It should be noted that accuracy and latency are merely two performance categories exemplified in this application's embodiments to illustrate the invention's purpose, and this application's embodiments do not impose specific limitations on the specific categories of performance. That is to say, in other embodiments, performance may also include other parameters besides accuracy and latency.

[0061] S202, the optimizer generates multiple execution plans based on the SQL statement.

[0062] After obtaining the SQL statement, the optimizer generates multiple execution plans based on different operator types and attributes.

[0063] In some embodiments, such as Figure 3As shown, the optimizer can perform logical optimization on SQL statements, avoiding logical errors in queries, improving query efficiency, and ensuring query accuracy. After logical optimization, as... Figure 3 As shown, the optimizer can perform physical optimization on the results of logical optimization and output the target execution plan. During the physical optimization phase, the optimizer can generate multiple execution plans based on the results of logical optimization.

[0064] S203, the optimizer uses a performance evaluation model to determine the predicted performance of each execution plan among multiple execution plans.

[0065] During the physical optimization phase, after the optimizer obtains multiple execution plans, it can use a performance evaluation model to assess the performance of these plans and obtain predicted performance values ​​for each plan. This performance evaluation model can include one or more of a precision prediction model and a latency prediction model.

[0066] The following section introduces how the optimizer uses a precision prediction model to evaluate the accuracy of the execution plan.

[0067] The optimizer can use a precision prediction model to determine the predicted precision of each execution plan based on the index types of the data columns related to the SQL statements included in each execution plan. For example... Figure 4a As shown, the optimizer can input the index types corresponding to the data columns related to the SQL statements included in each execution plan into the accuracy prediction model to obtain the predicted accuracy values ​​for each execution plan.

[0068] Taking execution plan P1 among multiple execution plans as an example, execution plan P1 may include Figure 5 The SQL statement-related data column operation information shown includes vector indexes, index traversal operators, index scan operators, filter operators, and sorting operators (order by withlimit). It can be seen that the index type corresponding to the SQL statement-related data columns is a vector index. Therefore, the optimizer can input the vector indexes corresponding to the SQL statement-related data columns included in execution plan P1 into the accuracy prediction model for evaluation, obtaining the predicted accuracy value of execution plan P1.

[0069] The aforementioned index traversal operators are used to traverse and / or iterate over vector indexes, and output data. The index scan operator scans the data output by the index traversal operator and outputs data that meets the scan conditions. The filtering operator filters the data output by the index scan operator and outputs data that meets the filtering conditions. The sorting operator sorts the data output by the filtering operator and outputs the query results corresponding to the SQL statement based on the sorting results. For example, Figure 5 As shown, the index scan operator can perform a lenient monotonic performance check on the output of the index traversal operator according to the scan conditions. The filtering operator can write its output to a priority queue. During the sorting process, if it is determined that the termination check condition has been met, the sorting operator can terminate the sorting early and output a certain amount of data as the query result corresponding to the SQL statement. If, at the end of the sorting process, it is determined that none of the data in the priority queue meets the termination check condition, the sorting operator can return to the index scan operator to continue scanning.

[0070] Optionally, the input to the accuracy prediction model may also include the join relationships between data tables related to the SQL statement, and / or the selectivity of data columns related to the SQL statement (selectivity can also be called column predicates). For example Figure 4b As shown, the optimizer can evaluate the join relationships between data tables related to the SQL statement, the selectivity of data columns related to the SQL statement, and the index types of data columns related to the SQL statement included in each execution plan by inputting them into the accuracy prediction model, thus obtaining the predicted accuracy value for each execution plan. Taking execution plan P1 as an example... Figure 5 Taking the information as an example, the optimizer can evaluate the join relationships between data tables related to the SQL statement, the selectivity of data columns related to the SQL statement, and the vector index input precision prediction model to obtain the predicted value of the precision of the execution plan P1.

[0071] Optionally, the input to the accuracy prediction model may also include data distribution information of the tables related to the SQL statements. The optimizer can input the data distribution information of the tables related to the SQL statements, the join relationships between the tables related to the SQL statements, the selectivity of the columns related to the SQL statements, and the index types corresponding to the columns related to the SQL statements included in each execution plan into the accuracy prediction model for evaluation, to obtain the predicted accuracy value of each execution plan.

[0072] The data distribution information of a data table can include the data distribution information of each data column in the table. The data distribution information of a data table can also include the total number of rows in the table and / or the number of partitions. The data distribution information of data columns with different data types can include different information. Specifically, the data type of a data column can be numeric or string. When a data column is numeric, its data distribution information can include the number of times each data value appears in the column, and / or the number of data points in each data range. When a data column is string, its data distribution information can include the number of times each character appears.

[0073] For the join relationships between data tables related to an SQL statement, the optimizer can determine them based on the constraints of the data tables included in the SQL statement. Taking a database containing data tables A through E, and the SQL statement as... Figure 6 The example shown, "SELECT * FROM A, B, C, D WHERE A.3 = C.3 AND A.4 = D.4 AND C.5 = B.5 AND A.2 < 5 AND B.1 = 'h'", has table constraints including "A.3 = C.3 AND A.4 = D.4 AND C.5 = B.5". Here, "A.3 = C.3" indicates that the data in column 3 of table A is similar to the data in column 3 of table C; "A.4 = D.4" indicates that the data in column 4 of table A is similar to the data in column 4 of table D; and "C.5 = B.5" indicates that the data in column 5 of table C is similar to the data in column 5 of table B. Therefore, the optimizer can determine the appropriate optimization method based on the table constraints in this SQL statement. Figure 6 The diagram illustrates the join relationships between data tables related to SQL statements. Specifically, table A is joined with tables C and D; table C is joined with table B; table B is joined with table C; and table C is joined with table B. Similarity can include identical or related parameters.

[0074] Optionally, the optimizer can represent the join relationships between tables in the database using a join graph, which includes join relationships between tables related to SQL statements. For example, the optimizer can... Figure 6 The join graph shown represents the connection relationships between data tables A through E. These connections include joins between data table A and data tables C and D, and a join between data table C and data table B. Figure 6 In the join graph shown, "1" indicates a connection and "0" indicates no connection.

[0075] The optimizer determines the selectivity for data columns related to an SQL statement based on the constraints of those columns and the data distribution information of the tables associated with the statement. Again, assuming a database includes tables A through E, and the SQL statement is... Figure 6 The example shown, "SELECT * FROM A, B, C, D WHERE A.3 = C.3 AND A.4 = D.4 AND C.5 = B.5 AND A.2 < 5 AND B.1 = 'h'", has constraints on the data columns including "A.2 < 5 AND B.1 = 'h'". The optimizer can determine the selectivity of column 2 in table A and column 1 in table B based on these constraints. For instance, the optimizer can determine the number of data values ​​(Na) less than 5 in column 2 of table A based on the data distribution information, and then divide Na by the total number of data values ​​in column 2 to obtain the selectivity of column 2. Similarly, the optimizer can determine the number of data values ​​(Nb) containing the character "h" in column 1 of table B based on the data distribution information, and then divide Nb by the total number of data values ​​in column 1 to obtain the selectivity of column 1.

[0076] In some embodiments, the optimizer can encode the join relationships between tables related to the SQL statement and / or the selectivity of columns related to the SQL statement, obtaining SQL statement-related encoded information, which is then used as input to the accuracy prediction model. In other words, the SQL statement-related encoded information is used to represent the join relationships between tables related to the SQL statement and / or the selectivity of columns related to the SQL statement. For example... Figure 6 As shown, the optimizer can concatenate the data in the upper right corner of the join graph and the selectivity of the data columns related to the SQL statement to obtain the SQL statement-related encoded information "[0 1 1 0 1 0 0 0 0 0 0 1···1···]".

[0077] The following section introduces how the optimizer uses a latency prediction model to evaluate the latency of the execution plan.

[0078] The optimizer can utilize a pre-trained latency prediction model to obtain predicted latency values ​​for each execution plan based on the operators included in each execution plan and the amount of data processed by those operators. For example... Figure 4a As shown, the optimizer can input the operators included in each execution plan and the amount of data processed by the operators into a pre-trained latency prediction model to obtain the predicted latency values ​​for each execution plan.

[0079] Optionally, the input to the latency prediction model may also include predicted costs of the operators included in each execution plan. For example... Figure 4b As shown, the optimizer can input the predicted values ​​of the operators included in each execution plan, the amount of data processed by the operators, and the cost of the operators into a pre-trained latency prediction model to obtain the predicted latency values ​​of each execution plan.

[0080] Taking execution plan P2 as an example, it can include operators such as indexscan, seq scan, loop join, sort, merge join, and aggregate. The optimizer can obtain the input data for the latency prediction model corresponding to execution plan P2 based on the data volume (cards) processed by each operator and the execution order of each operator. The input data for the latency prediction model corresponding to execution plan P2 includes a tree structure composed of the encoded information corresponding to each operator. For example... Figure 7 The encoded information corresponding to each operator shown, from left to right, represents the aggregation operator, merge join operator, sorting operator, nested loop join operator, sequential scan operator, index scan operator, null operation, data volume card (number of rows), and the predicted cost.

[0081] Among them, Figure 7 In the tree structure shown, the encoding information corresponding to the index scan operator is [0,0,0,0,0,1,0,25,0.08], the encoding information corresponding to the sequential scan operator associated with the loop nested join operator is [0,0,0,0,1,0,0,9000,0.12], the encoding information corresponding to the loop nested join operator is [0,0,0,1,0,0,0,25,0.53], the encoding information for the no-operation associated with the sorting operator is [0,0,0,0,0,0,1,0,0.0], and the encoding information corresponding to the sorting operator is... The information is [0,0,1,0,0,0,0,250,0.62], the encoding information corresponding to the sequential scan operator associated with the merge join operator is [0,0,0,0,1,0,0,110,0.32], the encoding information corresponding to the merge join operator is [0,1,0,0,0,0,0,10,0.78], the encoding information of the no-operation associated with the aggregation operator is [0,0,0,0,0,0,1,0,0.0], and the encoding information corresponding to the aggregation operator is [1,0,0,0,0,0,0,10,0.98].

[0082] The optimizer can determine the predicted cost of each operator included in the execution plan based on one or more of the predicted costs of indexing, input / output, executor, and communication. When multiple cost types are involved, the optimizer can perform a weighted summation based on the weights corresponding to the predicted cost of each type of operator in the execution plan to obtain the predicted cost of each operator. Executor cost refers to the resources consumed by the executor in executing the operator. Communication cost can include resource consumption between the executor and memory. The executor can include one or more of a central processing unit (CPU), graphics processing unit (GPU), and neural processing unit (NPU). When the executor is a CPU, communication cost can include resource consumption when communicating with dynamic random access memory (DRAM); when the executor is a GPU or NPU, communication cost can include resource consumption when communicating with high bandwidth memory (HBM).

[0083] Regarding index costs, the optimizer can determine the predicted index costs of the operators included in the execution plan based on the index type corresponding to the data columns in the SQL statement and the amount of data processed by the operators included in the execution plan. The following section will introduce the examples where the index type can include vector indexes or graph indexes.

[0084] When the index of the data table is a vector index, the optimizer can determine the predicted value C of the index cost of the operators included in the execution plan according to the following formula (1). v .

[0085] C v =N c ×C v,step +max([sel(q)N / N p ],m v )×N p ×C v,step (1)

[0086] In formula (1), N c C represents the amount of data (or rows) processed by the operator. v,stepThe stride of the vector index is represented by sel(q), the selectivity of the data columns related to the SQL statement is represented by sel(q), and N represents the total number of rows in the data table related to the SQL statement. p m represents the number of partitions in the data table related to the SQL statement. v The threshold for the number of samples for a vector index.

[0087] When the index of the data table is a graph index, the optimizer can determine the predicted value of the index cost of the operators included in the execution plan according to the following formula (2).

[0088] C g =N start ×C g,step +max(sel(q)N,m g )×R×C g,step (2)

[0089] In formula (2), N start The number of data rows (or index starting point) of a graph index, C g,step The step size of the graph index is m. g R represents the sampling threshold for the graph index, and R represents the number of iterations for the graph index.

[0090] Regarding IO costs, the optimizer can determine the predicted IO costs for each operator in the execution plan based on the data volume and IO unit cost of each operator in the execution plan.

[0091] Regarding executor costs, the optimizer can determine the predicted executor costs for each operator in the execution plan based on the data volume of each operator and the unit cost of the executor.

[0092] Regarding communication costs, the optimizer can determine the predicted communication costs of each operator in the execution plan based on the data volume and unit communication cost of each operator in the execution plan.

[0093] In some embodiments, the optimizer may utilize a pre-trained cost prediction model to determine predicted costs for each operator included in each execution plan. For example... Figure 4c As shown, the optimizer can input the amount of data processed by each operator included in the execution plan, the total number of rows and partitions of the data tables related to the SQL statement, and the selectivity and index type of the data columns related to the SQL statement into the cost prediction model to obtain the predicted cost of each operator included in the execution plan.

[0094] S204, the optimizer determines one or more first execution plans that meet the performance requirements from multiple execution plans based on the predicted performance values ​​of each execution plan.

[0095] The optimizer can compare the predicted performance of each execution plan with the performance requirement value, and select the execution plan that meets the performance requirement value as the first execution plan.

[0096] Taking execution plan P1 as an example, the optimizer can compare the predicted accuracy of execution plan P1 with the required accuracy (0.8). If the predicted accuracy of execution plan P1 is less than 0.8, the optimizer switches to the next execution plan to continue the accuracy judgment. If the predicted accuracy of execution plan P1 is equal to or greater than 0.8, the optimizer can continue to compare the predicted latency of execution plan P1 with the required latency (30). If the predicted latency of execution plan P1 is greater than 30, the optimizer switches to the next execution plan to continue the latency evaluation. If it is less than or equal to 30, execution plan P1 can be used as the first execution plan. Thus, the optimizer can obtain one or more first execution plans from multiple execution plans.

[0097] In some embodiments, the optimizer may first determine whether the current execution plan meets the user's query performance requirements, and then evaluate the performance of the next execution plan, that is, determine the predicted performance value of the next execution plan. For example Figure 8 As shown, the optimizer can first use the accuracy prediction model to evaluate the accuracy of execution plan P1. If the predicted accuracy of execution plan P1 is less than 0.8, it switches to the next execution plan to continue the accuracy evaluation. If the predicted accuracy of execution plan P1 is equal to or greater than 0.8, the optimizer can use the latency prediction model to evaluate the latency of execution plan P1. If the predicted latency of execution plan P1 is greater than 30, it switches to the next plan to continue the latency evaluation. If it is less than or equal to 30, execution plan P1 is added to the first execution plan set as the first execution plan.

[0098] In some embodiments, the database may have built-in weights corresponding to different performance levels, and these weights determine the first execution plan. That is, for each execution plan among multiple execution plans, the optimizer can perform a weighted sum based on the predicted performance value and the corresponding weight of each execution plan to obtain an evaluation value for each execution plan. Then, based on the evaluation value of each execution plan and an evaluation threshold, one or more first execution plans are determined. Taking performance including accuracy and latency as an example, the weight corresponding to accuracy is w1, and the weight corresponding to latency is w2. The optimizer can use w1 and w2 to perform a weighted sum on the predicted accuracy and predicted latency values ​​of each execution plan to obtain an evaluation value for each execution plan. Then, the optimizer can compare the evaluation value of each execution plan with the evaluation threshold to determine one or more first execution plans.

[0099] S205, the optimizer determines a second execution plan from multiple first execution plans based on the predicted costs of each first execution plan that meets the performance requirements. This second execution plan is the target execution plan to be executed by the executor.

[0100] The optimizer compares the predicted costs of each first execution plan and determines the second execution plan from among them. The second execution plan is the one with the lowest predicted cost among the multiple first execution plans.

[0101] It should be noted that when the input data for the latency prediction model includes predicted cost values, the predicted cost value for the first execution plan in this step is obtained before the predicted latency value. When the input data for the latency prediction model does not include predicted cost values, in this step, the optimizer can first determine the predicted cost values ​​for each of the first execution plans, and then perform the comparisons described above to determine the second execution plan.

[0102] S206, the executor executes the second execution plan and obtains the query results corresponding to the SQL statement.

[0103] The executor executes each operator in the target execution plan output by the optimizer to determine the query results corresponding to the SQL statement. During the execution of each operator, the executor can... Figure 1 The retriever shown retrieves the required data from the database and determines the query result corresponding to the SQL statement based on the data provided by the retriever.

[0104] In some embodiments, without considering the cost of the executor, the first execution plan obtained by the optimizer based on the performance predictions of multiple execution plans is executed by the executor. That is, if a first execution plan is obtained, this first execution plan is the target execution plan to be executed by the executor. If multiple first execution plans are obtained, the optimizer can select one as the target execution plan from among the multiple first execution plans. This application does not impose specific restrictions on the specific process by which the optimizer selects a first execution plan from among the multiple first execution plans. For example, the optimizer can arbitrarily select one from among the multiple first execution plans, or it can select according to pre-set filtering rules.

[0105] In some embodiments, after sending the query results corresponding to the SQL statement to the user, the database can receive the actual value of the precision of the query results as reported by the user. This actual value of precision can be used to update the parameters of the precision prediction model.

[0106] In the above scheme, the optimizer selects the execution plan that meets the performance requirements based on the predicted performance values ​​of multiple execution plans during the physical optimization phase. This can improve the performance of the database in obtaining query results, thereby enhancing the user experience.

[0107] based on Figure 2 The method embodiments shown in this application also provide a data query method.

[0108] Figure 9 This is a flowchart of another data query method provided in an embodiment of this application. This method can be applied to a database to improve user query performance, thereby enhancing the user experience. The database may include... Figure 1 The database shown. Figure 9 As shown, the method may include S901-S905. The following describes... Figure 9 The steps shown will be explained.

[0109] S901 generates multiple execution plans for SQL statements based on the user's Structured Query Language (SQL) statements.

[0110] Before this step, the database can receive SQL statements from users. These SQL statements can include performance requirement values ​​indicating performance needs. The specific process for this step can be referred to the above. Figure 2 The description of S201 in the illustrated embodiment will not be repeated here.

[0111] In this step, the database can generate multiple execution plans after receiving the user's SQL statement. The specific process for this step can be found above. Figure 2 The description of S202 in the illustrated embodiment will not be repeated here.

[0112] S902 uses a performance evaluation model to determine the predicted performance values ​​of each execution plan among multiple execution plans.

[0113] In this embodiment, performance may include accuracy and / or latency.

[0114] In cases where performance includes accuracy, the performance evaluation model can include an accuracy prediction model. In this step, the database can input the index types corresponding to the data columns related to the SQL statements in each execution plan into the accuracy prediction model to obtain the predicted accuracy values ​​for each execution plan. Furthermore, the input to the accuracy prediction model can also include: the join relationships between the data tables related to the SQL statements, and the selectivity of the data columns related to the SQL statements.

[0115] In cases where performance includes latency, the performance evaluation model can include a latency prediction model. In this step, the database can input the operators included in each execution plan and the amount of data processed by each operator into the latency prediction model to obtain the predicted latency values ​​for each execution plan. Furthermore, the input to the latency prediction model can also include the predicted cost values ​​of the operators included in each execution plan.

[0116] S903 determines multiple first execution plans that meet the performance requirements from multiple execution plans based on the predicted performance values ​​of each execution plan.

[0117] S904, determine the second execution plan from multiple first execution plans based on the predicted cost of each first execution plan.

[0118] S905, execute the second execution plan and obtain the query results corresponding to the SQL statement.

[0119] In this embodiment, the specific processes of S903 to S905 can be referred to the above. Figure 2 The descriptions of S204 to S206 in the illustrated method embodiment will not be repeated here.

[0120] In this embodiment, after sending the query results to the user, the database can also receive the actual value of the query result's accuracy from the user, and then update the parameters of the accuracy prediction model based on the actual accuracy value. In this way, the database can continuously improve the accuracy of the accuracy prediction model, providing users with more accurate query results in subsequent services.

[0121] Execute in the database Figure 2 and Figure 9 Prior to the method described, the initial accuracy prediction model, latency prediction model, and cost prediction model can be pre-trained offline by a training device. After training, the accuracy prediction model, latency prediction model, and cost prediction model can be deployed to a database and provided to the optimizer for performance evaluation. The accuracy prediction model, latency prediction model, and cost prediction model can adopt any structure of artificial intelligence (AI) model. This application embodiment does not impose specific limitations on the structure of these three models. Any AI model needs to be trained before it can be used to solve a specific technical problem. AI model training refers to the process of using a specified initial model to calculate training samples, and adjusting the parameters in the initial model according to the calculation results, so that the AI ​​model gradually learns certain rules and possesses specific functions. After training, an AI model with stable functions can be used to evaluate performance or cost.

[0122] like Figure 10As shown, the accuracy prediction model, latency prediction model, and cost prediction model can be obtained using supervised learning, unsupervised learning, or reinforcement learning. In some embodiments, the accuracy prediction model, latency prediction model, and cost prediction model can also be obtained using a combination of supervised learning (or unsupervised learning) and reinforcement learning.

[0123] Generally speaking, AI models in machine learning often need to be trained using supervised learning. Supervised learning allows AI models to learn more effectively the relationship between training data and corresponding labels in a labeled training set, resulting in higher accuracy when the trained AI model is used to predict other input data.

[0124] Supervised learning (or supervised training) involves determining the parameters of an initial AI model based on data from a given training dataset and the labels corresponding to each data point. A model that uses training dataset data and its labels to determine the parameters of an initial AI model is also called a supervised model. The labels on the data in the training dataset are usually manually labeled to identify the correct answer for a specific task. Typical supervised learning models include, but are not limited to: Support Vector Machines, Neural Network Models, Logistic Regression Models, Decision Trees, Naive Bayes Models, and Gaussian Discriminant Models.

[0125] In supervised learning, the training device can construct a training set for the AI ​​model based on the objective. This training set includes multiple training samples. Specifically, when the AI ​​model is an accuracy prediction model, the accuracy training samples can include the index types corresponding to the data columns related to the SQL statements in the execution plan, as well as the join relationships between the data tables related to the SQL statements and the selectivity of the data columns related to the SQL statements. When the AI ​​model is a latency prediction model, the latency training samples can include the operators included in the execution plan and the amount of data processed by each operator, as well as the predicted cost of the operators. When the AI ​​model is a cost prediction model, the cost training samples can include the amount of data processed by each operator in the execution plan, the total number of rows and partitions of the data tables related to the SQL statements, and the selectivity and index types of the data columns related to the SQL statements.

[0126] Each training sample is labeled, and the label of the training sample can represent the target of training the AI ​​model using the training sample. For example, when the AI ​​model is an accuracy prediction model, the label of the accuracy training sample represents the actual value of accuracy; when the AI ​​model is a latency prediction model, the label of the latency training sample represents the actual value of latency; when the AI ​​model is a cost prediction model, the label of the cost training sample represents the actual value of cost.

[0127] In supervised learning, the training device can input multiple training samples in batches into an AI model after parameter initialization. The AI ​​model performs calculations (i.e., evaluates performance or cost) on the input training samples and outputs predicted values ​​for the training samples. For example, when the AI ​​model is an accuracy prediction model, its output is the predicted accuracy value; when it is a latency prediction model, its output is the predicted latency value; and when it is a cost prediction model, its output is the predicted cost value. Then, the training device uses the model's predicted values ​​and the corresponding labels of the training samples as data to calculate the loss based on the loss function. The loss function is used during model training to calculate the difference (i.e., the loss value) between the model's prediction result for the training data and the label of that training data. Different mathematical functions can be used to implement the loss function; commonly used loss function expressions include mean squared error loss function, logarithmic loss function, and least squares method. Finally, the loss value calculated by the training device based on the loss function can be used to update the parameters of the AI ​​model. Specifically, gradient descent can be used to update the parameters.

[0128] Model training is an iterative process. Each iteration infers from different training data and calculates the loss value. The goal of multiple iterations is to continuously update the parameters of the deep learning model and find the parameter configuration that minimizes or stabilizes the loss function.

[0129] Unsupervised learning (or unsupervised training): This involves determining the parameters of an initial AI model based on unlabeled data from a given training dataset. Models that use unlabeled training data to determine the initial AI model's parameters are also called unsupervised models. Through unsupervised learning, models can discover meaningful information and correlations in the data, thereby enabling them to predict outcomes. The structures of unsupervised learning models can include, but are not limited to: clustering models, principal component analysis (PCA), anomaly detection models, autoencoders, and generative adversarial networks (GANs).

[0130] In unsupervised learning, the training device can construct a training set for the AI ​​model based on the target, which includes multiple training samples. The device can then input these training samples into the AI ​​model in batches after parameter initialization, iteratively optimizing the model parameters to better fit the data. The training device can determine whether the model has converged, and terminate the AI ​​model training if convergence is achieved. Convergence conditions may include the change in model parameters meeting a preset threshold or reaching a preset number of iterations.

[0131] Specifically, when the AI ​​model is an accuracy prediction model, the accuracy training samples can include the index types corresponding to the data columns related to the SQL statements in the execution plan, as well as the join relationships between the data tables related to the SQL statements and the selectivity of the data columns related to the SQL statements. When the AI ​​model is a latency prediction model, the latency training samples can include the operators included in the execution plan and the amount of data processed by each operator, as well as the predicted cost of the operators. When the AI ​​model is a cost prediction model, the cost training samples can include the amount of data processed by each operator included in the execution plan, the total number of rows and partitions of the data tables related to the SQL statements, and the selectivity and index types of the data columns related to the SQL statements.

[0132] Reinforcement learning differs from supervised and unsupervised learning. In reinforcement learning, there are two interacting entities: the agent and the environment. Reinforcement learning is a process of finding better solutions through trial and error. The agent must develop existing experience to gain benefits while also exploring new areas to obtain a better range of action choices in the future (i.e., learning from mistakes).

[0133] Before training begins, the training equipment initializes the reinforcement learning environment and the agent. The environment is the place where the agent interacts, including the states that the agent can perceive and the actions that it can perform. The agent is the subject that makes decisions, selects actions based on the current state, and receives rewards from the environment, including positive and negative rewards.

[0134] When the AI ​​model is an accuracy prediction model, the state can include: the accuracy training samples and the actual accuracy value of the execution plan. The accuracy training samples may include, for example, the index types corresponding to the data columns related to the SQL statements included in the execution plan. The state can also include: the join relationships between the data tables related to the SQL statements, and the selectivity of the data columns related to the SQL statements. The reward can include the difference between the actual accuracy value and the predicted accuracy value output by the model.

[0135] When the AI ​​model is a latency prediction model, the state can include: latency training samples and the actual latency values ​​of each operator in the execution plan. Latency training samples may include, for example, the operators included in the execution plan, the amount of data processed by each operator, and the actual latency value of the operator. The state can also include: the actual cost of the operator. The reward can include the difference between the actual latency value and the predicted latency value output by the model.

[0136] When the AI ​​model is a cost prediction model, the state can include: cost training samples and the actual cost values ​​of each operator in the execution plan. Latency training samples may include, for example, the amount of data processed by each operator in the execution plan, the total number of rows and partitions in the data tables related to the SQL statements, and the selectivity and index type of the data columns related to the SQL statements. The reward can include the difference between the actual cost value and the predicted cost value output by the model.

[0137] During training, the agent selects actions based on its current state and a given policy. The policy is a mapping from the environment state to actions, determining which action the agent should perform in a given state. The agent can select actions through random selection, greedy policies, or probability-based policies.

[0138] After an agent performs an action, the environment provides a feedback signal—a reward—based on the current state and the agent's action. A positive reward indicates that the action was beneficial; a negative reward indicates that the action was detrimental. Simultaneously, the environment transitions to the next state based on the agent's action and the current state.

[0139] After receiving feedback from the environment, an agent updates its policy based on that feedback. The goal of this update is to enable the agent to obtain more rewards in the future. There are various methods for policy updating, including value-based learning methods (such as Q-learning) and policy-based learning methods. These methods update the agent's policy by calculating the value function of the state or action, or by directly learning the policy function.

[0140] As the agent continuously selects actions, receives feedback, and updates its policy, it gradually learns the optimal decision sequence. When the agent's policy no longer changes significantly within a certain period of time, or when a preset number of training rounds is reached, the training process can be considered to have converged, and the policy obtained at this point is the optimal policy.

[0141] In some embodiments, such as Figure 10 As shown, the training device can also update the parameters of the accuracy prediction model again using supervised learning methods based on the predicted accuracy value output by the accuracy prediction model and the actual accuracy value reported by the user, thereby further improving the accuracy of the accuracy prediction model.

[0142] In some embodiments, such as Figure 10 As shown, the training device can also update the parameters of the latency prediction model again using supervised learning methods based on the predicted latency value output by the latency prediction model and the actual latency value fed back by the actuator, thereby further improving the accuracy of the latency prediction model.

[0143] In some embodiments, the functionality of the training device described above can be implemented by one or more computing devices in a computing device cluster. In some embodiments, the functionality of the training device described above can also be implemented by any one or more computing nodes in a database.

[0144] based on Figure 2 and Figure 9 The method embodiments shown in this application also provide a data query device.

[0145] Figure 11 This is a schematic diagram of the structure of a data query device 1100 provided in an embodiment of this application. The data query device 1100 can be deployed in... Figure 1 The database shown. The data query device 1100 is deployed in... Figure 1 In the case of the database shown, the data query device 1100 performs... Figure 2 and / or Figure 9 The steps are shown.

[0146] The data query device 1100 may include: a generation module 1101, an evaluation module 1102, a determination module 1103, and an execution module 1104.

[0147] The generation module 1101 is used to generate multiple execution plans for the user's Structured Query Language (SQL) statement.

[0148] The evaluation module 1102 is used to determine the predicted performance of each execution plan among the multiple execution plans using a performance evaluation model.

[0149] The determining module 1103 is configured to determine a plurality of first execution plans that meet the performance requirements from the plurality of execution plans based on the predicted performance values ​​of each execution plan, and to determine a second execution plan from the plurality of first execution plans based on the predicted cost values ​​of each first execution plan.

[0150] The execution module 1104 is used to execute the second execution plan and obtain the query results corresponding to the SQL statement.

[0151] In one possible implementation, this performance includes accuracy and / or latency.

[0152] In one possible implementation, where the performance includes accuracy, the performance evaluation model includes an accuracy prediction model, and the evaluation module is further configured to: input the index type corresponding to the data columns related to the SQL statement included in each execution plan into the accuracy prediction model to obtain the predicted value of the accuracy of each execution plan.

[0153] In one possible implementation, the inputs to the accuracy prediction model also include: the join relationships between the data tables related to the SQL statement, and the selectivity of the data columns related to the SQL statement.

[0154] In one possible implementation, the evaluation module is also used to: receive the actual value of the accuracy of the query result as reported by the user; and update the parameters of the accuracy prediction model based on the actual value of the accuracy.

[0155] In one possible implementation, where the performance includes latency, the performance evaluation model includes a latency prediction model, and the evaluation module is further configured to: input the operators included in each execution plan and the amount of data processed by each operator into the latency prediction model to obtain the predicted latency value of each execution plan.

[0156] In one possible implementation, the device further includes a receiving module for receiving the user's SQL statement, which includes a performance requirement value indicating the performance requirement.

[0157] It should be noted that, Figure 11 The data processing apparatus 1100 provided in the illustrated embodiment, when executing the data query method, is only illustrated by the division of the above-described functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the data processing apparatus provided in the above embodiment and... Figure 2 and Figure 9 The data query method embodiments shown are based on the same concept, and their specific implementation process is detailed in [link to implementation details]. Figure 2 and Figure 9 The method embodiments shown are not described in detail here.

[0158] When the above-described module is used as an example of a software functional unit, the data processing device 1100 may include code running on a computing instance. The computing instance may be at least one of a physical host (computing device), a virtual machine, a container, or other computing device. Further, the aforementioned computing device may be one or more. For example, the data processing device 1100 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the application may be distributed in the same region or in different regions. The multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0159] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0160] When the above-mentioned module is used as an example of a hardware functional unit, the module may include at least one computing device, such as a server. Alternatively, the module may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0161] The multiple computing devices included in a module can be distributed within the same region or in different regions. Similarly, the multiple computing devices included in a module can be distributed within the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in a module can be distributed within the same Virtual Private Cloud (VPC) or multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0162] It should be noted that, in other embodiments, the generation module 1101, evaluation module 1102, determination module 1103, and execution module 1104 can be used to perform... Figure 2 and Figure 9 Any step in the method shown. The steps implemented by the generation module 1101, evaluation module 1102, determination module 1103, and execution module 1104 can be specified as needed and implemented by the generation module 1101, evaluation module 1102, determination module 1103, and execution module 1104 respectively. Figure 2 and Figure 9 The different steps in the method shown enable the full functionality of the data processing device 1200.

[0163] This application also provides a computing device 1200. For example... Figure 12 As shown, the computing device 1200 includes a bus 1201, a processor 1202, a memory 1203, and a communication interface 1204. The processor 1202, the memory 1203, and the communication interface 1204 communicate with each other via the bus 1201. The computing device 1200 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 1200.

[0164] Bus 1201 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 12 The bus 1201 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 1201 may include a path for transmitting information between various components of the computing device 1200 (e.g., memory 1203, processor 1202, communication interface 1204).

[0165] The processor 1202 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0166] The memory 1203 may include volatile memory, such as random access memory (RAM). The processor 1202 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0167] The memory 1203 stores executable program code, and the processor 1202 executes the executable program code to implement the functions of the aforementioned generation module 1101, evaluation module 1102, determination module 1103, and execution module 1104, thereby achieving... Figure 2 and / or Figure 9 The method shown. That is, the memory 1203 stores the method for execution. Figure 2 and / or Figure 9 The instructions for the method shown.

[0168] The communication interface 1204 uses modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 1200 and other devices or communication networks.

[0169] based on Figure 2 and Figure 9 The method embodiments shown in this application also provide a computing device cluster.

[0170] Figure 13 This application provides a computing device cluster 1200. This computing device cluster can be used for deployment. Figure 1 The database shown. That is to say, Figure 1 The database shown can run on Figure 13 The computing device cluster shown is 1200.

[0171] The computing device cluster 1200 may include the data processing platform described above. The computing device cluster includes at least one computing device. This computing device may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0172] like Figure 13 As shown, the computing device cluster 1200 includes at least one computing device 1200. The memory 1203 of one or more computing devices 1200 in the computing device cluster may store the same memory for executing... Figure 2 and / or Figure 9 The instructions for the method shown. Figure 1 The database shown can run on Figure 13 In the case of the computing device cluster 1200 shown, one or more computing devices 1200 can perform... Figure 2 and / or Figure 9 The steps are shown.

[0173] In some possible implementations, the memory 1203 of one or more computing devices 1200 in the computing device cluster may also store memory for execution. Figure 2 and / or Figure 9 The instructions of the method shown are partial. In other words, a combination of one or more computing devices 1200 can jointly execute instructions for performing... Figure 2 and / or Figure 9 The instructions for the method shown.

[0174] It should be noted that the memory 1203 in different computing devices 1200 within the computing device cluster can store different instructions, which are used to execute some functions of the data processing device 1200. That is, the instructions stored in the memory 1203 of different computing devices 1200 can implement the functions of one or more modules among the generation module 1101, evaluation module 1102, determination module 1103, and execution module 1104.

[0175] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 14 One possible implementation is shown. For example... Figure 14As shown, the two computing devices 1200A and 120B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this possible implementation, the memory 1203 in computing device 1200A stores instructions for executing the functions of the generation module 1101 and the evaluation module 1102. Meanwhile, the memory 1203 in computing device 1200B stores instructions for executing the functions of the determination module 1103 and the execution module 1104. Figure 14 The connection method between the computing device clusters shown can be such that, considering the data query method provided in this application requires a large amount of storage and computing resources (e.g., storing a large amount of data), the functions implemented by the generation module 1101 and the evaluation module 1102 are delegated to computing device 1200A, and the functions implemented by the determination module 1103 and the execution module 1104 are delegated to computing device 1200B. It should be understood that... Figure 14 The functions of the computing device 1200A shown can also be performed by multiple computing devices 1200. Similarly, the functions of the computing device 1200B can also be performed by multiple computing devices 1200.

[0176] This application also provides another computing device cluster. The connection relationships between the computing devices in this computing device cluster can be similarly referred to... Figure 13 and Figure 14 The connection method of the computing device cluster. The difference is that the memory 1203 of one or more computing devices 1200 in this computing device cluster can store the same data for execution. Figure 2 and / or Figure 9 The instructions for the data query method shown.

[0177] In some possible implementations, the memory 1203 of one or more computing devices 1200 in the computing device cluster may also store memory for execution. Figure 2 and / or Figure 9 The instructions for the data query method shown are partial. In other words, a combination of one or more computing devices 1200 can jointly execute instructions for performing... Figure 2 and / or Figure 9 The instructions for the data query method shown.

[0178] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform... Figure 2 and / or Figure 9 The data query method shown.

[0179] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute... Figure 2 and / or Figure 9 The data query method shown may instruct the computing device to perform... Figure 2 and / or Figure 9 The data query method shown.

[0180] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data query method, characterized by, Applied to a database, the method includes: Multiple execution plans for the SQL statement are generated based on the user's Structured Query Language (SQL) statement; The performance prediction values ​​of each execution plan in the plurality of execution plans are determined using a performance evaluation model; Based on the predicted performance values ​​of each execution plan, determine a plurality of first execution plans that meet the performance requirements from the plurality of execution plans; Based on the projected costs of each of the first execution plans, a second execution plan is determined from the plurality of first execution plans; The second execution plan is executed to obtain the query results corresponding to the SQL statement.

2. The method of claim 1, wherein, The performance includes accuracy and / or latency.

3. The method according to claim 1 or 2, characterized in that, When the performance includes accuracy, the performance evaluation model includes an accuracy prediction model, and the step of using the performance evaluation model to determine the predicted performance values ​​of each execution plan among the plurality of execution plans includes: The index type corresponding to the data columns related to the SQL statements included in each execution plan is input into the accuracy prediction model to obtain the predicted accuracy value of each execution plan.

4. The method of claim 3, wherein, The inputs to the accuracy prediction model also include: the join relationships between the data tables related to the SQL statement, and the selectivity of the data columns related to the SQL statement.

5. The method according to claim 3 or 4, characterized in that, The method further includes: Receive the actual value of the accuracy of the query results as reported by the user; The parameters of the accuracy prediction model are updated based on the actual value of the accuracy.

6. The method according to any one of claims 1-5, characterized in that, When the performance includes latency, the performance evaluation model includes a latency prediction model, and the step of using the performance evaluation model to determine the predicted performance values ​​of each execution plan among the plurality of execution plans includes: The operators included in each execution plan and the amount of data processed by each operator are input into the latency prediction model to obtain the predicted latency value of each execution plan.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Receive the user's SQL statement, which includes a performance requirement value indicating the performance requirement.

8. A data query device, characterized in that, The device includes: The generation module is used to generate multiple execution plans for the user's Structured Query Language (SQL) statement. An evaluation module is used to determine the predicted performance of each execution plan among the multiple execution plans using a performance evaluation model; The determining module is configured to determine a plurality of first execution plans that meet performance requirements from the plurality of execution plans based on the predicted performance values ​​of each execution plan, and to determine a second execution plan from the plurality of first execution plans based on the predicted cost values ​​of each first execution plan; The execution module is used to execute the second execution plan and obtain the query results corresponding to the SQL statement.

9. The apparatus according to claim 8, characterized in that, The performance includes accuracy and / or latency.

10. The apparatus of claim 8 or 9, wherein, When the performance includes accuracy, the performance evaluation model includes an accuracy prediction model, and the evaluation module is further configured to: input the index type corresponding to the data columns related to the SQL statements included in each execution plan into the accuracy prediction model to obtain the predicted value of the accuracy of each execution plan.

11. The apparatus of claim 10, wherein, The inputs to the accuracy prediction model also include: the join relationships between the data tables related to the SQL statement, and the selectivity of the data columns related to the SQL statement.

12. The apparatus according to claim 10 or 11, characterized in that, The evaluation module is also used to: receive the actual value of the accuracy of the query result as reported by the user; and update the parameters of the accuracy prediction model based on the actual value of the accuracy.

13. The device of any one of claims 8-12, wherein, When the performance includes latency, the performance evaluation model includes a latency prediction model, and the evaluation module is further configured to: input the operators included in each execution plan and the amount of data processed by each operator into the latency prediction model to obtain the predicted latency value of each execution plan.

14. The apparatus of any one of claims 8-13, wherein, The device further includes a receiving module for receiving the user's SQL statement, wherein the SQL statement includes a performance requirement value indicating the performance requirement.

15. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method according to any one of claims 1-7.

16. A computer-readable storage medium, characterized in that, Includes computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method as described in any one of claims 1-7.

17. A computer program product, characterised in that, Includes computer program instructions, which, when executed by a cluster of computing devices, perform the method as described in any one of claims 1-7.

Images