A data query method, device and equipment and readable storage medium

Abstract

The specification discloses a data query method, device and equipment and a readable storage medium. An award matrix is generated for a target database. A reward value is determined by predicting the cardinality of a query result obtained by executing each candidate action in a current state through a pre-trained neural network. The award matrix is updated according to the predicted reward value. When a data query request is received, the connection order of each target table is determined from the updated award matrix according to each target table contained in a target query statement. The target query statement is executed in the connection order to obtain a query result. It can be seen that the reward value of executing each candidate action in the current state is determined by the cardinality of the query result obtained by the neural network. The cardinality of the query result is used as feedback for selecting the connection order of the tables in the database. The hardware environment for executing the data query is not affected. The time cost is reduced while the accuracy is ensured.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a data query method, apparatus, device, and readable storage medium. Background Technology

[0002] With the explosive growth of data, database-related technologies have received widespread attention. Among these, databases can use multiple tables to store data for easier management, and how to quickly determine query results based on multiple tables has always been a key research focus in the database field.

[0003] To improve the efficiency of obtaining query results from a database based on query statements, this specification provides a data query method. Summary of the Invention

[0004] This specification provides a data query method, apparatus, device, and readable storage medium to partially solve the aforementioned problems existing in the prior art.

[0005] The following technical solution is adopted in this specification:

[0006] This manual provides a data query method, including:

[0007] A reward matrix is ​​generated for the target database. The reward matrix contains reward values ​​representing the execution of different actions in different states. The states are the order of the actions that have been executed, and the actions are querying a table in the target database.

[0008] Determine the current state, and determine at least one candidate action based on the current state;

[0009] For each candidate action, a pre-trained neural network predicts the reward value for performing the candidate action in the current state, wherein the smaller the cardinality of the query results obtained based on the already performed actions and the candidate actions, the larger the reward value.

[0010] The reward matrix is ​​updated based on the determined reward value, and a target action is selected from each candidate action based on the updated reward matrix. The target action is taken as the action that has been executed, and the current state is redefined. Candidate actions are determined based on the redefined current state until the preset termination condition is met, and the target reward matrix is ​​obtained.

[0011] In response to the target query statement, determine several target tables contained in the target query statement;

[0012] Based on the target reward matrix, the order in which the target query statement queries each target table is determined, and the target query statement is executed to obtain the query results.

[0013] This specification provides a data query device, including:

[0014] A generation module is used to generate a reward matrix for a target database. The reward matrix contains reward values ​​representing different actions performed under different states. The states are the order of the actions that have been performed, and the actions are querying a table in the target database.

[0015] The first determining module is used to determine the current state and determine at least one candidate action based on the current state;

[0016] The second determining module is used to predict the reward value of performing the candidate action in the current state for each candidate action by using a pre-trained neural network, wherein the smaller the cardinality of the query results obtained based on the already performed actions and the candidate actions, the larger the reward value.

[0017] The update module is used to update the reward matrix according to the determined reward value, select the target action from each candidate action according to the updated reward matrix, treat the target action as the executed action, redetermine the current state, continue to determine candidate actions according to the redetermined current state, until the preset end condition is reached, and determine the target reward matrix.

[0018] The third determining module is used to determine, in response to the target query statement, several target tables contained in the target query statement;

[0019] The execution module is used to determine the order in which the target query statement queries each target table based on the target reward matrix, and to execute the target query statement to obtain the query results.

[0020] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described data query method.

[0021] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described data query method.

[0022] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0023] The data query method provided in this specification generates a reward matrix for the target database. A pre-trained neural network predicts the cardinality of the query results obtained by executing each candidate action in the current state, determining the reward value. The reward matrix is ​​then updated based on the predicted reward values. Upon receiving a data query request, the connection order of the target tables in the target query statement is determined from the updated reward matrix, and the target query statement is executed according to this connection order to obtain the query results. Therefore, by using the cardinality of the query results obtained through the neural network to determine the reward value for executing each candidate action in the current state, and using the cardinality of the query results as feedback for selecting the connection order of tables in the database, this method is not affected by the hardware environment in which the data query is executed, thus achieving the goal of reducing time costs while ensuring accuracy. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0025] Figure 1 This is a flowchart illustrating one data query method described in this specification.

[0026] Figure 2 This is a flowchart illustrating one data query method described in this specification.

[0027] Figure 3 This is a flowchart illustrating one data query method described in this specification.

[0028] Figure 4 This is a schematic diagram of a data query device provided in this specification;

[0029] Figure 5 The corresponding information provided in this specification Figure 1 A schematic diagram of an electronic device. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0031] Additionally, it should be noted that all actions involving the acquisition of signals, information, or data in this invention are carried out in compliance with the relevant data protection laws and regulations of the country where the invention is located, and with authorization from the owner of the corresponding device.

[0032] With the development of information technology, databases continue to play a vital role as a means of supporting data storage and retrieval. The amount of data stored is increasing daily, and users' query needs for data in databases are becoming increasingly complex. Faced with massive databases, data retrieval efficiency has become a key concern for researchers. This necessitates optimizing database queries to ensure both efficiency and quality when processing a large number of complex query requests.

[0033] Currently, for relational databases, the data query engine is a key component for achieving good database system performance, optimizing the input query statement. The main task of the data query engine is to optimize the input declarative query statement into a detailed and efficient physical query plan with each step executed. Among these steps, optimizing the join order is the core of the database data query engine; the same query statement using different join orders can lead to response times differing by orders of magnitude. The optimization process typically involves the database receiving a Structured Query Language (SQL) query request and generating multiple execution plans based on it. Each execution plan is a query plan generated by the database for the query request, including the query algorithm used and the order of queries across multiple tables. The database then predicts the potential query time and resource consumption of each execution plan to select the optimal one, which is then used for subsequent data queries.

[0034] Currently, traditional data query engines typically use static join order enumeration algorithms. These algorithms lack feedback on the quality of the generated query plans, so optimizers often repeatedly select the same poor query plans. Furthermore, exhaustive traversal methods are too space-intensive. The ultimate goal of join order selection is to find the minimum-cost join order by enumerating the candidate space; therefore, the enumerator needs to find the minimum-cost join order while minimizing the candidate space.

[0035] Furthermore, existing machine learning-based join plan selection algorithms address the multi-table join query plan selection problem through cost-based optimization. This involves setting a cost model to pre-evaluate the cost of joining two tables, with the cost portion of training the model largely using query time as feedback to guide training. However, executing the query in the database to obtain the actual execution time is time-consuming, and query time is more susceptible to hardware factors in different environments, failing to accurately guide the training process. Moreover, executing all execution plans consumes significant time and processing resources. Considering the large number of query requests databases face in real-world applications, this clearly leads to excessive database workload, ultimately impacting query efficiency and hindering practical applications.

[0036] Based on this, this specification provides a data query method that combines join plan selection in the database with reinforcement learning. Relying on the self-learning and adaptive nature of machine learning, it obtains the cardinality of the query results when the query continues to query tables in the target database in different table query orders by constructing a reward matrix. This intuitively indicates the reward value obtained under different table query orders, resulting in higher efficiency and making it more suitable for use with a large number of tables. Furthermore, using the cardinality obtained from neural network training as feedback to guide the construction of the reinforcement learning reward matrix, compared to using query time as feedback, the results obtained from cardinality prediction are more stable and do not change significantly due to changes in the hardware environment executing the query. While ensuring accuracy, it greatly reduces time costs and improves query efficiency.

[0037] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0038] Figure 1 This is a flowchart illustrating a data query method provided in this specification.

[0039] S100: Generate a reward matrix for the target database. The reward matrix contains reward values ​​representing different actions performed under different states. The state is the order of the actions performed, and the action is querying a table in the target database.

[0040] Specifically, the data query method provided in this manual can be executed by any electronic device capable of performing data query methods, such as a server; this manual does not limit this.

[0041] In practical applications, data queries are executed by inputting query statements into the target database to obtain the desired data. The target database may contain several tables, leading to multi-table queries. Optimizing the table query order can improve query efficiency while maintaining accuracy. Therefore, this specification constructs a reward matrix corresponding to the target database. The reward values ​​in the matrix reflect the query efficiency of different table query orders, enabling the determination of the most efficient table query order for queries involving multiple tables in practical applications.

[0042] Specifically, a reward matrix is ​​generated for the target database. The reward matrix is ​​arranged with states as rows and actions as columns. The reward value corresponding to each state and each action in the reward matrix is ​​used to indicate the reward value that can be obtained by performing the action in that state.

[0043] To determine the reward value obtained by performing each action in each state within the reward matrix, we can first define the state and action, and then determine the reward value based on the state and action.

[0044] Specifically, in this case, the data query involved in the embodiments of this specification is a multi-table query problem, which requires optimizing the query order of the tables. Different table query orders will affect the data query performance of the same query statement. Therefore, in order for the reward value in the reward matrix to represent the performance of executing data queries under different table query orders, the state in the reward matrix is ​​defined as the query order of the queried tables, and the action is defined as querying a table in the target database. Different actions correspond to querying different tables in the target database. Executing different actions in a state represents querying different tables in the target database under the query order of the queried tables. The reward value is used to indicate the query performance when continuing to query a table under the current query order, such as the query time, query cost, or cardinality of the query result. It can be understood that the actions included in the state are different from the actions executed in the state, that is, the table to be queried after the current query is different from the queried tables. Specifically, the parameters of the query statement are determined according to the target database, and the query statement is generated. The parameters of the query statement represent that the query statement queries the target data from each table in the target database. Furthermore, because the query objectives differ, the parameters of different query statements may not be exactly the same. However, as long as the table being queried is the same, the corresponding actions are considered the same. That is, actions with different query parameters but querying the same table constitute the same action. In other words, for each action, the table queried is fixed, while the query parameters are set according to business requirements. The query parameters must at least include query conditions. For example, query statement A queries for males older than 20 years old in table x, query statement B queries for females older than 40 years old in table x, and query statement C queries for males older than 20 years old in table y. Query statements A and B have the same action, while the action of query statement C is different from both A and B. Even if query statements A and B query different data, as long as the table being queried is the same, they perform the same action.

[0045] Furthermore, specific query statements can be used. By inputting the query statements into the target database, the performance of the data query execution is obtained, and the reward values ​​in the reward matrix are updated. Specifically, before determining the current state, a training query statement containing a specified number of tables is generated for the target database. Each table in the target database covers each table in the training query statement. The training query statement instructs the querying of target data from the specified number of tables, and the parameters corresponding to the training query statement are determined based on the target data queried as instructed by the training query statement.

[0046] Wherein, when the training query statements contain the same tables, the parameters corresponding to the training query statements are different.

[0047] For example, the first training query statement can instruct the query to retrieve target data A from three tables: A, B, and C. The second training query statement can only retrieve target data B from three tables: A, B, and C. It can be seen that the first and second training query statements contain the same number and type of tables, but the data queried is different. Therefore, the parameters corresponding to the first training query statement are different from those corresponding to the second training query statement.

[0048] In addition, the reward matrix for the target database can be an initialized reward matrix, in which the reward value corresponding to each action in each state is the initialized reward value, such as zero.

[0049] S102: Determine the current state and determine at least one candidate action based on the current state.

[0050] Specifically, in the reward matrix, the current state is typically represented by rows, and each candidate action by columns. The matrix values ​​corresponding to the rows and columns represent the reward values ​​obtained by executing the candidate actions in the current state. Therefore, to update the reward matrix, we can first determine the current state and action corresponding to the reward value to be updated. Typically, multi-table queries do not require querying the same table repeatedly. That is, in the order of the tables queried representing the current state, there are usually no duplicate tables. Therefore, while determining the current state, we can select tables not included in the current state from the tables in the target database as candidate actions for the current state.

[0051] Specifically, the reward matrix is ​​generated and updated for the target database. Therefore, the reward matrix is ​​applicable to data queries targeting all tables within the target database. Consequently, the tables included in the current state can be a subset of the tables in the target database. The current state can contain one or more tables, and the tables in the current state are ordered. Even if the tables are the same, different query orders will result in different states. When a state contains multiple tables, the current state reflects the query order of the queried tables. When a state contains only one table, the current state represents the starting point for the query on that table.

[0052] Furthermore, for a target database containing multiple tables, multi-table queries are required to obtain query results. Multi-table queries necessitate determining the query order. Therefore, this can be achieved by continuing to query other unqueried tables within the current table query order; that is, by executing candidate actions in the current state. This allows for the subsequent determination of the reward value obtained when executing each candidate action in the current state, which will then serve as the reward value obtained when adding each candidate table within the current table query order.

[0053] Optionally, when performing the first update to the initialized reward matrix, the current state can be considered as an initial state with no executed actions. At least one table from the target database is selected as the candidate action corresponding to the current state. Typically, when selecting candidate actions for the initial state, they are randomly selected from the tables in the target database. To avoid getting trapped in local optima, all tables in the target database are usually used as candidate actions for the initial state, and these are executed sequentially. The reward values ​​obtained from executing each candidate action in the initial state are then updated in the reward matrix. Of course, the reward values ​​may not be updated in the reward matrix; this specification does not impose any restrictions on this.

[0054] S104: For each candidate action, predict the reward value of performing the candidate action in the current state using a pre-trained neural network, wherein the smaller the cardinality of the query results obtained based on the already performed actions and the candidate actions, the larger the reward value.

[0055] In practical applications, by determining the reward value for performing different candidate actions in the current state, the determined reward values ​​are used as feedback for query performance to update the reward matrix, thereby performing reinforcement learning. Based on the reward matrix obtained in the above process, the query experience of multi-table queries obtained in the reinforcement learning process can be used to obtain the optimized multi-table query order as the optimized query execution strategy when responding to data query requests.

[0056] Since the tables corresponding to each candidate action are all in an unqueried state, meaning the current state does not include the tables corresponding to each candidate action in the current table query order, to obtain the reward value for executing each candidate action in the current state, we can assume that for each candidate action's corresponding table, we add a query to that candidate table at the end of the current table query order. This adds a state after querying the candidate table, and the cardinality of the query results in the target database is used as feedback to obtain the reward value for executing that candidate action in the current state.

[0057] Currently, query time is often used as a feedback mechanism for query performance. However, query time can be affected by the hardware environment in which the data query is executed. Even with the same query statement and the same table query order, different hardware environments can result in different query times. Therefore, in this embodiment, the cardinality of the query results is used as a feedback mechanism for query performance. The cardinality of the query results indicates the number of different query results obtained when the query statement is executed in the target database. Normally, the same query statement executed in the same table query order will produce the same cardinality of the query results, unaffected by the hardware environment. Therefore, compared to query time, the cardinality of the query results is a more stable and accurate feedback mechanism for query performance.

[0058] The cardinality of the query results in the target database can be obtained by a pre-trained neural network for cardinality estimation. The query used by the neural network during training can be generated based on the tables contained in the target database, or it can be generated based on the tables contained in other databases. This specification does not limit this.

[0059] Furthermore, the cardinality of the query results obtained based on the executed actions and the candidate actions refers to the cardinality of the query results obtained by the training query statement in the target database when continuing to execute candidate actions in the current order of executed actions. Specifically, the currently queried tables included in the training query statement and the query order of the queried tables are determined. Based on the tables in the target database other than the currently queried tables, candidate tables corresponding to the training query statement are determined. For each candidate table, the training query statement continues to query that candidate table in the current query order, updates the training query statement, and inputs the updated training query statement into a pre-trained neural network to obtain the cardinality of the query results of the updated training query statement in the target database. The cardinality predicted by the neural network is used as feedback on the query performance obtained when continuing to query that candidate table in the current query order.

[0060] Furthermore, when determining the reward value for performing the candidate action in the current state through neural network prediction, the base value predicted by the neural network is negatively correlated with the reward value; that is, the smaller the base value, the larger the reward value.

[0061] S106: Update the reward matrix according to the determined reward value, select the target action from each candidate action according to the updated reward matrix, take the target action as the executed action, redetermine the current state, continue to determine candidate actions according to the redetermined current state, until the preset end condition is reached, and obtain the target reward matrix.

[0062] In reinforcement learning, an agent can perform actions based on its current state. During state transitions, the chosen action needs to be determined through environmental feedback. Specifically, in this case, feedback in query optimization typically comes from the actual execution performance of the query statement on the target database, such as query time. However, the above approach incurs a high time cost in obtaining the actual query time during training, which is detrimental to improving the efficiency of reinforcement learning. Furthermore, the actual query time may be affected by the hardware environment in which the query is executed, causing fluctuations in query time across different hardware environments, thus reducing the accuracy of reinforcement learning-based feedback.

[0063] Therefore, in this embodiment, the cardinality of the query results is used as the feedback, or reward, for executing each candidate action in the current state. After determining each cardinality, based on the current state and candidate actions at the time the cardinality is obtained, it is determined which candidate action is executed in which state for each cardinality. Then, based on the determined cardinality, the reward value in the reward matrix for executing each candidate action in the current state is updated. It is understood that the reward value in the reward matrix can be updated by overwriting the original reward value. For example, when determining the reward value for executing a candidate action in the current state, if the reward value obtained from executing the candidate action in the current state has already been filled into the first reward value, when determining the second reward value for executing the candidate action in the current state, the first reward value in the reward matrix can be directly replaced with the determined second reward value. Of course, the reward value in the reward matrix can also be updated in other ways, such as weighting the first reward value before the update, and determining the reward value for executing the candidate action in the current state based on the determined second reward value and the weighted first reward value. This specification does not limit this approach.

[0064] After traversing all candidate actions corresponding to the current state, an action for state transition is selected from each candidate action based on the reward value. The selected action is then executed in the current state to perform the state transition and update the current state. Based on the updated current state, candidate actions are re-determined, and reward values ​​are further determined based on the re-determined candidate actions to update the reward matrix.

[0065] Optionally, based on the reward values ​​of each candidate action executed in the current state in the updated reward matrix, the relationship between the reward values ​​is determined, and a target action is selected from each candidate action according to the size of the reward values. The selected target action is then executed in the current state to perform a state transition and update the current state.

[0066] The target database may contain a finite number of tables. For the reward matrix, the initial state is defined as having no executed actions, and the termination state is defined as having executed a specified number of actions. This specified number is typically no greater than the number of tables in the target database. If the current state is a termination state, the current state can be redefined based on the reward matrix, and steps S102 to S106 above can be repeated to update the reward matrix until the termination condition for updating the reward matrix is ​​met.

[0067] Optionally, performing the above steps S102 to S106 can be considered as updating the reward matrix once. The preset termination condition for updating the reward matrix can be that the number of updates to the reward matrix reaches a preset number.

[0068] Of course, the above-mentioned preset termination condition can also be other preset conditions, such as the number of times the table at the first position in the termination state contained in the reward matrix appears exceeds a preset threshold, etc., which is not limited in this specification.

[0069] S108: In response to the target query statement, determine several target tables contained in the target query statement.

[0070] When the update of the reward matrix reaches the preset termination condition, the obtained reward matrix is ​​used as the target reward matrix. The target reward matrix is ​​used as the query experience for data querying the target database. When responding to data query requests, the target reward matrix is ​​used as prior experience to guide the selection of the table query order of the target query statement.

[0071] Furthermore, the target query statement is parsed to obtain the tables required for the data to be queried as target tables. Based on the target tables and the target reward matrix, the query order of each target table is determined with the maximum reward as the query target.

[0072] S110: Based on the target reward matrix, determine the order in which the target query statement queries each target table, and execute the target query statement to obtain the query results.

[0073] The query process involves identifying the states within the target reward matrix, using the state encompassing all target tables as the target state, and determining the target state with the highest reward value as the query order for the target tables in the target query statement. The target query statement is then executed in the target database according to this table query order to obtain the query results, thereby improving query efficiency while ensuring accuracy.

[0074] The data query method provided in this specification generates a reward matrix for the target database. A pre-trained neural network predicts the cardinality of the query results obtained by executing each candidate action in the current state, determining the reward value. The reward matrix is ​​then updated based on the predicted reward values. Upon receiving a data query request, the connection order of the target tables in the target query statement is determined from the updated reward matrix, and the target query statement is executed according to this connection order to obtain the query results. Therefore, by using the cardinality of the query results obtained through the neural network to determine the reward value for executing each candidate action in the current state, and using the cardinality of the query results as feedback for selecting the connection order of tables in the database, this method is not affected by the hardware environment in which the data query is executed, thus achieving the goal of reducing time costs while ensuring accuracy.

[0075] In one or more embodiments of this specification, such as Figure 1 As shown in step S104, before predicting the reward value of performing the candidate action in the current state through a pre-trained neural network for each candidate action, a cardinality estimation model can be pre-trained, specifically implemented through the following scheme:

[0076] First, a query statement is generated based on the target database, and each action contained in the query statement is used as a training sample.

[0077] Several query statements are generated based on the tables contained in the target database, and the queries that query the tables in the target database, i.e., the actions, are used as training samples.

[0078] Next, the query statement is executed in the target database to obtain the cardinality of the query results of the training sample in the target database, which is used as the label of the training sample.

[0079] Then, the training samples are input into the neural network to be trained to obtain the prediction base of the training samples output by the neural network.

[0080] Finally, the neural network is trained with the goal of minimizing the difference between the predicted base and the labels of the training samples.

[0081] Specifically, a multi-layer neural network is constructed as the cardinality estimation model to be trained. Typically, a neural network based on three fully connected layers is chosen, but the dimensional parameters of the cardinality estimation model can be adjusted according to the number of tables loaded in the database.

[0082] When selecting an optimizer for the neural network with the training objective of minimizing the difference between the predicted cardinality and the labels of the training samples, any existing optimizer can be chosen, such as Stochastic Gradient Descent (SGD), Stochastic Gradient Descent with Momentum (SGDM), Adaptive Gradient (Adagrad), etc. In this embodiment, the optimizer for the technical estimation model can use the SGD optimization method. This is because current training methods require repeatedly feeding the entire training data into the neural network for training, which consumes significant computational resources and time. The SGD optimization method can automatically split the training set and then feed it into the neural network in batches for training. While the accuracy of batch training is not as high as that of training with the entire set of data, it does not lose too much accuracy and significantly accelerates the training process. However, the specific optimization method used by the cardinality estimation model during training can be determined based on the specific application scenario, and this specification does not impose any limitations on it.

[0083] In addition, a larger learning rate increases the probability of oscillations during the search process, while a smaller learning rate greatly increases the time to converge to the extreme value. In the embodiments of this specification, the learning rate can be selected as 0.005, but the specific learning rate of the algorithm can be determined according to the specific application scenario, and this specification does not limit it.

[0084] Furthermore, during the training of the cardinality estimation model, the loss can be determined based on the difference between the predicted cardinality output by the cardinality estimation model and the labels of the second training samples. The cardinality estimation model is trained with the goal of minimizing the loss. The choice of loss function can be determined according to the specific application scenario, such as L1 loss function, L2 loss function, cross-entropy loss function, and mean square error (MSE). After multiple training iterations to reach the maximum number of iterations, the training of the cardinality estimation model ends, and the trained parameters are saved.

[0085] The aforementioned neural network training method uses the true cardinality of the query results obtained by executing a query statement in the target database as a label, enabling the neural network to learn the predictive ability of outputting the cardinality of corresponding query results based on different table inputs. The cardinality output by the trained neural network is then used as... Figure 1 The updated reward values ​​in the reward matrix are based on the following: when constructing the reward matrix, the query time used to execute the query statement is reduced, thus improving the update efficiency of the reward matrix.

[0086] In one or more embodiments of this specification, such as Figure 1 Step S104 shows that for each candidate action, a pre-trained neural network predicts the reward value for performing the candidate action in the current state, such as... Figure 2 As shown, the specific implementation will be carried out through the following scheme:

[0087] S200: Based on the current state, determine whether there is a previous state. If yes, proceed to step S202; otherwise, proceed to step S204.

[0088] In practical applications, the reward value in the reward matrix can indicate the reward value for executing a candidate action in the current state, and can also include the maximum reward value corresponding to the previous state. This is because in reinforcement learning, the reward value can include the reward values ​​obtained from executing multiple actions, i.e., multi-step reward values. In this case, the reward value indicates the reward value that can be obtained by continuing to execute the candidate action given the current order of executed actions, rather than just including the reward value that can be obtained by executing the candidate action. Currently, when determining the reward value of the candidate action in the current state based on the maximum reward value of the previous state, a weight coefficient of the maximum reward value of the previous state can be set to indicate the proportion of influence of the maximum reward value of the previous state on the reward value obtained by executing the candidate action in the current state.

[0089] Optionally, when determining the previous state of the current state, the order of currently executed actions can be determined based on the current state, that is, the table query order of the currently queried tables, with the table at the end of the order serving as a reference table. The reference table included in the table query order indicated by the current state is then removed from the order, and the remaining table query order is the previous state of the current state. Since the current state is obtained from the previous state through state transition, if the current state contains only one action, then the current state does not have a previous state.

[0090] If the current state has a previous state, then when determining the reward value obtained by performing a candidate action in the current state, the influence of multi-step reward values ​​needs to be considered; that is, the reward value corresponding to the previous state of the current state needs to be considered.

[0091] If there is no previous state in the current state, then when determining the reward value obtained by performing a candidate action in the current state, there is no need to consider the influence of other factors. The reward value can be determined directly by using the cardinality of the query result predicted by the neural network when performing the candidate action in the current state.

[0092] S202: Based on the reward matrix, determine the maximum reward value that can be obtained by performing the action in the previous state, and based on the maximum reward value and the reward value predicted by the neural network for performing the candidate action in the current state, determine the reward value of the candidate action.

[0093] S204: Predict the reward value for performing the candidate action in the current state using the neural network.

[0094] Through such Figure 2 The method shown conditionally incorporates the maximum reward value of the previous state when predicting the reward value of each candidate action in the current state using a pre-trained neural network. By using the maximum reward value of the previous state, the reward value updated to the reward matrix includes the multi-step rewards of each action already executed, making the determination of the reward value more realistic and improving the accuracy of the reward matrix. In turn, this improves the accuracy of data querying based on the reward matrix.

[0095] In one optional embodiment of this specification, such as Figure 1 Step S106 shows the selection of a target action from each candidate action based on the updated reward matrix, with the target action being included among the executed actions. In the current state, at least one target action can be selected from each candidate action based on the updated reward matrix, which can be divided into the following two cases:

[0096] The first scenario involves selecting a target action from among the candidate actions. Specifically, based on the reward values ​​of each candidate action executed in the current state according to the updated reward matrix, the candidate action with the highest reward value is determined as the target action. In practical applications, since the smaller the cardinality of query results obtained by executing the candidate action in the current state, as predicted by the pre-trained neural network, the larger the reward value, the candidate action with the highest reward value in the updated reward matrix corresponds to the candidate action with the smallest cardinality of query results obtained by executing that candidate action in the current state. This means that executing that candidate action in the current state yields the fewest number of non-repeating query results, and updating the state with that candidate action as the target action results in a more efficient table query order.

[0097] The second scenario involves selecting a specified number of target actions from among the candidate actions. This specified number is less than the number of candidate actions, or each candidate action is used as a target action. Specifically, based on the reward value of each candidate action executed in the current state according to the updated reward matrix, the candidate actions are sorted in descending order of reward value. A specified number of candidate actions with larger reward values ​​are selected, and these selected actions are used as target actions in sequence to perform subsequent reward matrix updates. In this embodiment, the cardinality of the query results obtained from executing candidate actions in different states is negatively correlated with the reward value in the reward matrix. However, consistently selecting the candidate action with the largest reward value as the target action for each state update may lead to getting trapped in a local optimum. This is because consistently selecting the action with the largest reward value in each state update may miss situations where the current action has a smaller reward value, but the subsequent action has a larger reward value.

[0098] Therefore, by traversing a specified number of candidate actions with larger reward values ​​in the current state, even though expanding the size of the reward matrix takes time, it can effectively avoid getting trapped in local optima. In practical applications, when faced with a target table contained in a target query statement, it can obtain the globally optimal table query order to improve the efficiency of table queries.

[0099] In one or more embodiments of this specification, such as Figure 1 When determining the current state as shown in step S102, and determining at least one candidate action based on the current state, the candidate actions for the current state can be determined based on the actions included in the current state, such as... Figure 3 As shown, it is specifically determined in the following ways:

[0100] S300: Determine whether the current state includes all actions. If yes, proceed to step S304; otherwise, proceed to step S302.

[0101] Through such Figure 1 As can be seen from step S108, when the update of the reward matrix does not reach the preset end condition, it is necessary to return to step S102 to redetermine the current state. The redetermined state is determined by the current state and the target action selected from the current state. Therefore, the redetermined current state at least includes the actions contained in the current state before redetermining, as well as the selected target action.

[0102] S302: Based on the current state, determine at least one candidate action in the current state from among the actions that have not been executed.

[0103] Furthermore, if the redefined current state does not include all actions, it means that there are still actions not included in the state. Candidate actions in the current state can be identified from the unexecuted actions so that subsequent steps of determining reward values ​​and updating the reward matrix can be performed.

[0104] S304: Take the initial state as the current state, where there is no action executed. Determine the first action of each state in the reward matrix according to the reward matrix.

[0105] When the redefined current state contains all actions, it means that the reward matrix already contains the reward value corresponding to a state that includes all actions. At this point, all actions in the current state can be deleted, resulting in an initial state with no executed actions. This initial state can then be used as the current state, and actions can be re-executed as follows. Figure 1 Steps S102 to S106 are used to update the reward matrix.

[0106] Specifically, taking the initial state as the current state, at this time, the table does not exist in the current table query order. In one or more embodiments of this specification, the probability of executing each candidate action in the initial state can be the same. That is, in the reward matrix, the number of times each action is the first action can be in the state. Therefore, when redetermining the current state, an action can be selected as the first action for redetermining the current state based on the number of times each action is the first action.

[0107] S306: Determine if there is an action that is not the first action. If yes, proceed to step S308; otherwise, proceed to step S310.

[0108] Furthermore, to ensure that the number of states where each action is the first action is equal, the priority of determining the current state is increased based on actions that are not the first action. In other words, as long as there is an action that is not the first action, that action is given priority as a candidate action for the initial state. Of course, if there are multiple actions that are not the first action, then there are multiple candidate actions for the initial state.

[0109] S308: Determine candidate actions based on actions that are not the first action.

[0110] S310: Based on the reward matrix, determine the reward update counts for each state with the same first and second actions, and use these counts as the update counts for each first and second action. Then, determine the candidate actions in ascending order of the update counts for each first and second action.

[0111] If there is no action that is not the first action, it means that the state of each action as the first action already exists in the reward matrix. If the reward matrix still needs to be updated, the candidate actions for the initial state can be determined based on the number of times each action is the first action, provided that the probability of executing each candidate action in the initial state is the same.

[0112] Specifically, the reward update count for each state with the same first action is determined from the reward matrix; this count represents the update count of each first action. The update count of the first action indicates how many times a particular action appears first in the query order of the reward matrix across several updates. A higher update count indicates a greater probability of executing that action in the initial state. To ensure that the probability of executing each candidate action is equal in the initial state, actions with fewer update counts can be prioritized as candidate actions for the initial state. Optionally, the first actions can be arranged in ascending order of update count, and a specified number of first actions with fewer update counts can be selected as candidate actions for the initial state. That is, the target action for the initial state can be selected from the first actions with approximately the fewest update counts. Alternatively, the first action with the fewest update counts can be directly selected as the target action for the initial state.

[0113] In one or more embodiments of this specification, such as Figure 1 Step S102 shows determining at least one candidate action based on the current state, specifically through the following method:

[0114] First, obtain the table query records of the target database to determine the execution frequency of each action.

[0115] Secondly, candidate actions are determined based on actions whose execution frequency is higher than a preset threshold.

[0116] In practical applications, the target database may contain hot tables and cold tables. Hot tables are those with a high number of queries, meaning queries executed on those tables are performed frequently. Conversely, cold tables are those with a low number of queries. The query frequency of a table is obtained by querying its records, and this frequency is used as the execution frequency of each action. Constructing a reward matrix for a subset of tables in the target database can significantly improve the efficiency of updating the reward matrix. Simultaneously, by covering the tables with high execution frequency, it ensures the effectiveness of data queries in most data query scenarios.

[0117] Furthermore, when the training query used to update the reward matrix does not use all the tables contained in the target database during generation, it is possible that the target tables included in the target query statement differ from the original tables mentioned above when responding to data queries. In this case, as in... Figure 2As shown in step S112, before determining the order in which the target query statement queries each target table based on the target reward matrix, the reward matrix can be updated again based on the target query statement and each target table, according to the rewards for each action performed in each state contained in the reward matrix. This results in a reward matrix containing the target states corresponding to the target tables, thereby obtaining the execution order of the target query statement. The specific scheme is as follows:

[0118] Step 1: When the target table included in the target query statement does not belong to any candidate action query table, determine the action to query the target table based on the target table, and update the candidate actions based on the determined action.

[0119] In practical applications, it's possible that the target table included in the target query statement doesn't belong to any of the candidate action queries. This could be because the reward matrix update uses only a subset of tables from the target database, such as frequently accessed tables with a query frequency exceeding a preset threshold. Alternatively, it's possible that new tables have been added to the target database after the reward matrix update. In such cases, the complete query order of each target table cannot be obtained when faced with the target query statement. Therefore, the reward matrix can be updated again based on the newly added target tables.

[0120] Specifically, tables in each target table that differ from the candidate actions contained in the reward matrix are added as new tables, and candidate actions are updated based on queries of these new tables.

[0121] Step 2: Update the target reward matrix based on the updated candidate actions.

[0122] Based on the updated candidate actions and the states contained in the reward matrix, the reward value to be updated in the reward matrix is ​​determined by the neural network. It can be divided into two parts: one is the reward value when each candidate action is executed in each state when the action of querying the new table is the first action; the other is the reward value when each candidate action is executed in each state when the action of querying the new table is not the first action.

[0123] Step 3: Based on the updated target reward matrix, determine the order in which the target query statement queries each target table.

[0124] According to the neural network, when the target query statement prioritizes querying the newly added table as the first action, the reward value for executing each candidate action in each state is determined. Furthermore, when the target query statement prioritizes querying the newly added table as a non-first action, the newly added table is added as a new action in each state's table query order contained in the reward matrix, and the reward value for executing the new action in each state is determined. The reward values ​​of the above two parts are updated in the reward matrix. The updated target reward matrix can contain the complete table query order of each target table. At this point, the target reward matrix can be re-queried based on each target table to obtain the table query order of each target table, and the target query statement is executed in the table query order with the highest reward value.

[0125] Figure 4 This specification provides a schematic diagram of a data query device, which specifically includes:

[0126] The generation module 400 is used to generate a reward matrix for the target database. The reward matrix contains reward values ​​representing the execution of different actions in different states. The states are the order of the actions that have been executed, and the actions are querying a table in the target database.

[0127] The first determining module 402 is used to determine the current state and determine at least one candidate action based on the current state;

[0128] The second determining module 404 is used to predict the reward value of performing the candidate action in the current state for each candidate action by using a pre-trained neural network, wherein the smaller the cardinality of the query results obtained based on the executed actions and the candidate actions, the larger the reward value.

[0129] The update module 406 is used to update the reward matrix according to the determined reward value, select the target action from each candidate action according to the updated reward matrix, take the target action as the executed action, redetermine the current state, continue to determine candidate actions according to the redetermined current state, until the preset end condition is reached, and determine the target reward matrix.

[0130] The third determining module 408 is used to determine, in response to the target query statement, several target tables contained in the target query statement;

[0131] The execution module 410 is used to determine the order in which the target query statement queries each target table based on the target reward matrix, and to execute the target query statement to obtain the query results.

[0132] Optionally, the device further includes:

[0133] The training module 412 is specifically used to generate a query statement based on the target database, and use each action contained in the query statement as a training sample; execute the query statement in the target database to obtain the cardinality of the query results of the training sample in the target database, which is used as the label of the training sample; input the training sample into the neural network to be trained to obtain the predicted cardinality of the training sample output by the neural network; and train the neural network with the goal of minimizing the difference between the predicted cardinality and the label of the training sample.

[0134] Optionally, the second determining module 404 is specifically used to: when the current state has a previous state, determine the maximum reward value that can be obtained by performing an action in the previous state according to the reward matrix, and determine the reward value of the candidate action according to the maximum reward value and the reward value predicted by the neural network for performing the candidate action in the current state; when the current state does not have a previous state, predict the reward value for performing the candidate action in the current state through the neural network.

[0135] Optionally, the update module 406 is specifically used to determine the candidate action with the largest reward value as the target action based on the reward value of each candidate action performed in the current state in the updated reward matrix.

[0136] Optionally, the preset termination condition is that the number of times the reward matrix is ​​updated reaches a preset number.

[0137] Optionally, the update module 406 is specifically configured to: determine whether the current state contains all actions; if yes, take the initial state as the current state, and determine at least one candidate action based on the reward matrix and the current state; wherein the initial state has no executed actions; if no, determine at least one candidate action in the current state from the unexecuted actions based on the current state.

[0138] Optionally, the update module 406 is specifically configured to: determine the first action of each state in the reward matrix according to the reward matrix; determine whether there is an action that is not the first action; if so, determine candidate actions based on the actions that are not the first action; if not, determine the reward update count of each state with the same first action according to the reward matrix, and use it as the update count of each first action, and determine candidate actions in ascending order of the update count of each first action.

[0139] Optionally, the first determining module 402 is specifically used to obtain table query records of the target database, determine the execution frequency of each action, and determine candidate actions based on actions with execution frequencies higher than a preset threshold.

[0140] Optionally, the execution module 410 is specifically configured to: when the target table included in the target query statement does not belong to any table queried by the candidate action, determine the action to query the target table based on the target table, update the candidate actions based on the determined actions, update the target reward matrix based on the updated candidate actions, and determine the order in which the target query statement queries each target table based on the updated target reward matrix.

[0141] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 The data query method shown.

[0142] This instruction manual also provides Figure 5 The diagram shows a schematic structural representation of the electronic device. Figure 5 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1 The data query method is shown. Of course, in addition to the software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0143] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0144] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, ASICs, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0145] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0146] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0147] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0148] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0149] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0150] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0151] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0152] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0153] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0154] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0155] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0156] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0157] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0158] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A data query method, comprising: generating a reward matrix of reinforcement learning for a target database, wherein the reward matrix comprises reward values of performing different actions in different states, the states being sequences of performed actions, and the actions being queries to a table in the target database; determining a current state and at least one candidate action according to the current state; for each candidate action, predicting a reward value of performing the candidate action in the current state by a pre-trained neural network, wherein the reward value is greater when a cardinality of query results is smaller, the query results being obtained by querying the target database according to the performed actions and the candidate action; updating the reward matrix according to the determined reward values, and selecting a target action from the candidate actions according to the updated reward matrix, taking the target action as a performed action, and re-determining a current state, and continuing to determine candidate actions according to the re-determined current state until a preset ending condition is reached, to obtain a target reward matrix; in response to a target query statement, determining a plurality of target tables contained in the target query statement; determining an order of querying the target tables in the target query statement according to the target reward matrix, and executing the target query statement in the order of the target tables to obtain query results. 2.The method of claim 1, wherein the pre-training of the neural network comprises: generating a query statement according to the target database, and taking each action contained in the query statement as a training sample; executing the query statement in the target database to obtain a cardinality of query results of the training sample in the target database as a label of the training sample; inputting the training sample into the neural network to be trained to obtain a predicted cardinality of the training sample output by the neural network; training the neural network with a training target of minimizing a difference between the predicted cardinality and the label of the training sample. 3.The method of claim 1, wherein the predicting of the reward value of performing the candidate action in the current state by the pre-trained neural network comprises: when there is a previous state in the current state, determining a maximum reward value obtained by performing an action in the previous state according to the reward matrix, and determining the reward value of the candidate action according to the maximum reward value and the reward value of performing the candidate action in the current state predicted by the neural network; when there is no previous state in the current state, predicting the reward value of performing the candidate action in the current state by the neural network. 4.The method of claim 1, wherein the selecting of the target action from the candidate actions according to the updated reward matrix comprises: determining a candidate action with a maximum reward value as the target action according to the reward values of performing the candidate actions in the current state in the updated reward matrix. 5.The method of claim 1, wherein the preset ending condition is that an updating frequency of the reward matrix reaches a preset frequency. 6.The method of claim 1, wherein the determining the at least one candidate action based on the current state comprises: determining whether the current state contains all actions; if yes, taking an initial state as the current state, and determining the at least one candidate action based on the reward matrix and the current state, wherein the initial state is a state without any executed action; and if no, determining the at least one candidate action based on the current state from the actions that have not been executed. 7.The method of claim 6, wherein the determining the at least one candidate action based on the reward matrix and the current state comprises: determining a first action of each state of the reward matrix based on the reward matrix; determining whether there is an action that is not the first action; if yes, determining the candidate action based on the action that is not the first action; and if no, determining a reward update frequency of each state with the same first action based on the reward matrix, as an update frequency of each first action, and determining the candidate action according to the update frequencies of the first actions in ascending order. 8.The method of claim 1, wherein the determining the at least one candidate action based on the current state comprises: obtaining a table query record of the target database to determine an execution frequency of each action; and determining the candidate action based on an action with an execution frequency higher than a preset threshold. 9.The method of claim 8, wherein the determining the order of the target tables based on the target reward matrix comprises: when a target table included in the target query statement does not belong to any table queried by a candidate action, determining an action for querying the target table based on the target table, and updating the candidate action based on the determined action; updating the target reward matrix based on the updated candidate action; and determining the order of the target tables based on the updated target reward matrix. 10.A data query device, comprising: a generating module configured to generate a reward matrix of reinforcement learning for a target database, the reward matrix containing reward values of different actions performed in different states, the state being an order of executed actions, and the action being a query to a table in the target database; a first determining module configured to determine a current state, and determine at least one candidate action based on the current state; and a second determining module configured to, for each candidate action, predict a reward value of performing the candidate action in the current state by a pre-trained neural network, wherein the reward value is greater when a cardinality of a query result queried by the executed actions and the candidate action is smaller, and the cardinality of the query result is used to indicate a number of different query results when performing data query in the target database. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ an updating module, configured to update the reward matrix according to the determined reward value, and select a target action from the candidate actions according to the updated reward matrix, take the target action as an executed action, and re-determine a current state, continue to determine candidate actions according to the re-determined current state until a preset ending condition is reached, and determine a target reward matrix; a third determining module, configured to determine a plurality of target tables contained in the target query statement in response to the target query statement; an executing module, configured to determine an order of querying the target tables by the target query statement according to the target reward matrix, and execute the target query statement in the order of querying the target tables to obtain a query result.

11. The apparatus of claim 10, further comprising: a training module, specifically configured to generate a query statement according to the target database, take each action contained in the query statement as a training sample; execute the query statement in the target database to obtain a cardinality of a query result of the training sample in the target database as a label of the training sample; input the training sample into the neural network to be trained to obtain a predicted cardinality of the training sample output by the neural network, and train the neural network with minimization of a difference between the predicted cardinality and the label of the training sample as a training target.

12. The apparatus of claim 10, wherein the second determining module is specifically configured to, when the current state has a previous state, determine a maximum reward value obtainable by executing an action in the previous state according to the reward matrix, and determine a reward value of the candidate action according to the maximum reward value and a reward value of the candidate action predicted by the neural network in the current state; and when the current state does not have a previous state, predict a reward value of the candidate action in the current state by the neural network.

13. The apparatus of claim 10, wherein the updating module is specifically configured to determine a candidate action with a maximum reward value as a target action according to reward values of the candidate actions in the current state in the updated reward matrix.

14. The apparatus of claim 10, wherein the preset ending condition is that a number of times of updating the reward matrix reaches a preset number of times. 15.The apparatus of claim 10, wherein the updating module is specifically configured to determine whether the current state contains all actions; if yes, take the initial state as the current state, and determine at least one candidate action according to the reward matrix and the current state; wherein, the initial state is without an executed action; and if not, determine at least one candidate action in the current state from actions not executed according to the current state.

16. The apparatus of claim 15, wherein the updating module is specifically configured to determine a first action of each state of the reward matrix according to the reward matrix; determine whether there is an action not serving as a first action; if yes, determine a candidate action according to the action not serving as a first action; and if not, determine a number of times of reward updating of each state with the same first action as the number of times of updating each first action according to the reward matrix, and determine a candidate action in an order of the number of times of updating each first action from small to large. 17.The apparatus of claim 10, wherein the first determining module is specifically configured to acquire table query records of the target database, determine execution frequencies of each action, and determine candidate actions according to actions with execution frequencies higher than a preset threshold. 18.The apparatus of claim 17, wherein the execution module is specifically configured to, when a target table contained in the target query statement does not belong to any table of a candidate action query, determine an action of querying the target table according to the target table, update the candidate actions according to the determined action, update the target reward matrix according to the updated candidate actions, and determine an order of querying each target table of the target query statement according to the updated target reward matrix. 19.A computer readable storage medium, the storage medium storing a computer program, the computer program being executed by a processor to implement the method of any one of claims 1-9. 20.An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing the method of any one of claims 1-9 when executing the program.

Images