Database query optimization method, system, electronic device and storage medium

Abstract

The present invention discloses a database query optimization method, system, electronic device and storage medium, aiming to solve the technical problems of low efficiency, large computational complexity and low accuracy of existing query methods. The present invention adopts a dynamic double DQN connection order optimization method (Dynamic Double DQN order select, DDOS), firstly models the connection query as a Markov decision process (MDP), and uses a weighted dual deep Q network to train the neural network model to improve the prediction accuracy of the trained network. Actions are selected through a dynamic progressive search strategy to improve the randomness and depth of exploration to accumulate exploration with higher information gain. After the cost of each query plan is estimated, a connection plan that conforms to the data distribution and has query load balance is selected. The beneficial technical effect of the present invention is that it can effectively improve query performance and has good generalization ability and robustness.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a database query optimization method, system, electronic device, and storage medium. Background Art

[0002] The join order problem, which can be formulated as finding the optimal rearrangement of a multi-table join, is one of the most important issues in query optimization research. Different join orders for the same query can lead to various join execution plans, each with varying execution efficiency. In this join order problem, it is crucial to control the scope of the search space to reduce search overhead. This is because the search space grows exponentially with the number of table relations. For example, when N = 20, the search space reaches 2.4e18. When faced with joins exceeding 1,000 relations, it is even impossible to determine a reasonable optimization order, making it impossible to control execution overhead through efficient join plans.

[0003] Formulating efficient query plans quickly and accurately is key to improving database system performance and query efficiency. Traditional databases primarily use dynamic programming and heuristic methods to select the order of cost-based multi-table join queries. The core idea behind dynamic programming is to reduce the size of the search space by merging redundant states. However, the search space is still close to exhaustive global space, so it is only applicable to queries containing fewer relations. Heuristic join methods typically use specific search strategies to perform heuristic searches on parts of the state space. Compared to traversing the search space, this method simplifies the search space of potential query plans, which reduces the time spent optimizing the query, but also reduces the chance of finding the best query plan in the search space.

[0004] With the increasing maturity of deep learning (DL) and reinforcement learning (RL) methods, research in the past few years has focused on integrating these two approaches to solve problems. The DQN (Deep Q-Network) model proposed by Mnih et al. is a pioneering work in DRL (Deep Reinforcement Learning). This model uses a deep neural network to learn the Q matrix. It introduces two mechanisms, experience replay (learning memory) and a target network, to alleviate the convergence fluctuations that occur when neural networks approximate the Q function. Because deep neural networks provide rich high-dimensional representations and enable end-to-end learning, combining the two enables RL to address some action decision problems in high-dimensional state spaces. However, some researchers believe that integrating online RL with deep neural networks is unstable. Hasselt et al. proposed the DDQN algorithm to reduce overestimation. However, DDQN sometimes underestimates action values. To reduce training data while ensuring sampling efficiency during RL learning, Mahajan et al. proposed a new framework that combines a symmetry detection program and a symmetry implementation mechanism to discover symmetries in the environment and apply them to the function approximation process to accelerate the learning process of symmetric policies. In addition, in the continuous scope of the simple discretized DQN, the number of actions increases exponentially with the increase in degrees of freedom, resulting in an excessively large action space and extremely difficult convergence. Therefore, Lillicrap et al. proposed a model-free algorithm based on DPG and AC, DDPG, which applies the ideas of DQN to the continuous action domain and can be applied to more complex problems and networks. However, compared with DDQN, it requires slower updates to the network parameters of the actor and critic, so it requires a lot of training to adapt to the connection sequence. Summary of the Invention

[0005] The present invention provides a database query optimization method, system, electronic device and storage medium to solve the technical problems of low efficiency, large calculation amount and low accuracy of existing query methods.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] The first aspect of the present invention is to design a database query optimization method, which inputs the state and action information of the database query into a dynamic dual DQN network model, wherein the dynamic dual DQN network model includes an evaluation network and a target network. The evaluation network is used to mine the action corresponding to the maximum value of Q in the dual DQN network, and the target network is used to evaluate the maximum action value in the dual DQN network. * ;θ) and Q(s',a* θ - ) to calculate the target value as follows:

[0008] y WDDQN =r+γ[βQ(s',a * ;θ)+(1-β)Q(s',a * θ - )]

[0009] Among them, Q(s',a * ,θ - ) represents the maximum action value in the target network at state s', Q(s',a L ,θ - ) represents the minimum action value of state s' in the target network, β is defined as the weight, and the range of β belongs to [0,1]. When β is 0, the network is equivalent to the DDQN network. When β is 1, the model completely ignores the Double DQN evaluation and only uses the DQN network to select actions. The weight β is calculated as follows:

[0010]

[0011] Among them, action a * Denotes the action with the maximum action value for the evaluation network, a L represents the action with the minimum action value for the evaluation network,

[0012] a * =arg maxQ(s',a;θ)

[0013] a L =arg minQ(s',a,θ)

[0014] Where c is a hyperparameter used to calculate the weight β;

[0015] The Q value is derived based on the cost model and the execution delay caused by the action:

[0016]

[0017] And the updated value of Q:

[0018] Q(s,a)←Q(s,a)+α[r+γmax a' Q(s',a')-Q(s,a)]

[0019] Among them, r is the reward, α is the learning rate, which is used to determine the size of the error learning γ represents the learning rate, max a' Q(s',a') represents the target Q value, Q(s,a) represents the estimated Q value,

[0020] The loss function is defined as the difference between the estimated Q value and the actual Q value, that is, the mean square error L(θ):

[0021] L(θ)=E[||r+γmax a' Q(s',a';θ)-Q(s,a;θ)||].

[0022] Furthermore, for the hyperparameter c, the optimal value of c is different in problems with different characteristics. The hyperparameter c affects the network allocation weight. The size of the hyperparameter is set according to the characteristics of the connection task, and the value of c is selected based on multiple experiments.

[0023] Furthermore, the status and action information of database queries are used as inputs to the model. Vectorized expression is used for query encoding. When designing the intelligent agent to select actions, a tree structure representation method is adopted. Connection queries are represented by query trees. The connection between tables is defined as joining the connection tree, and leaf nodes are defined as tables to be joined. When a new leaf node is added, it is first connected to the corresponding subtree to preserve the sub-connection structure information between tables.

[0024] Furthermore, the evaluation network selects actions through a dynamic progressive search strategy. First, the uncertainty is evaluated, and the uncertain and unexplored actions during the unexplored period are taken as the focus of the training period. The exploration to utilization stage can be realized by increasing the ε value from 0 to 1. The dynamic switching between stages can be achieved by adjusting the parameter ε. The expression for controlling the change of the parameter ε is

[0025]

[0026] Among them, H is a continuous variable, and the probability of the difference in Q values between two alternative actions in a certain state is specified as ε(H), and the value range of ε(H) is (0, 1).

[0027] Furthermore, an adjustable threshold p is set, where U i (a) is the limit of the action in state s. When there is less training data in the early stage, the limit is set relatively large. As the training data increases, the estimation of the action becomes more and more accurate, and the limit will become smaller and smaller. The calculation formula is as follows:

[0028]

[0029] Furthermore, in the model training phase, only after cost training and with the data collected from the previous cost model training as a guide, the network weights of the output layer are reset, fine-tuned using execution delay, and then retrained on query data with actual execution runtime.

[0030] The second aspect of the present invention is to design a database query optimization system, comprising:

[0031] Database query statement input module, used to characterize the actions and states of database queries;

[0032] Network environment module, used for data interaction within the model;

[0033] An evaluation network module, configured to interact with the network environment module to obtain sample data of reward values, thereby updating the evaluation network;

[0034] The target network module is used to obtain the parameters of the evaluation network module, and then calculate the action value with the maximum Q value in the current state. After selecting the connected action according to the exploration strategy, it acts on the network environment module and obtains a new state;

[0035] The output module is used to hand over the final connection plan to the executor for execution.

[0036] The third aspect of the present invention is to design an electronic device, comprising

[0037] at least one actuator; and,

[0038] a memory communicatively connected to the at least one processor; wherein,

[0039] The memory stores instructions that can be executed by the at least one executor. The instructions are executed by the at least one executor to enable the at least one executor to perform the above-mentioned database query optimization method.

[0040] A fourth aspect of the present invention is to design a storage medium, which, when the instructions in the storage medium are executed by an executor of a computer, enables the computer to execute the above-mentioned database query optimization method.

[0041] Compared with the prior art, the beneficial technical effects of the present invention are:

[0042] (1) The present invention adopts a method combining weighted dual deep Q networks and dynamic search strategies to optimize the connection order, effectively solving the impact of DQN overestimation and DDQN underestimation on query performance. It adopts a progressive greedy strategy to train the connection actions, which enables the intelligent agent to learn the strategy more accurately, improves the efficiency of mining actions in the training phase, and thus gets closer to the optimal plan.

[0043] (2) The present invention can solve the problem of model estimation error in query connection by integrating the network model DQN and dual DQN weights into the query optimizer, thereby improving the quality of query plan formulation.

[0044] (3) The present invention uses a dynamic progressive search strategy to select actions, which can estimate the information gain of network uncertainty data, accelerate learning speed and achieve effective potential exploration.

[0045] (4) The experiments of the present invention in JOB show that, compared with other methods, the connection query plan generated by the DDOS method not only reduces the optimization delay time, but also improves the quality of the connection query plan. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Schematic diagram of the workflow of the query plan of the present invention.

[0047] Figure 2 Schematic diagram of the motion vector representation process of the present invention.

[0048] Figure 3 Schematic diagram of the network prediction model architecture of the present invention. DETAILED DESCRIPTION

[0049] The specific implementation modes of the present invention are described below with reference to the accompanying drawings and examples. However, the following examples are only used to illustrate the present invention in detail and are not intended to limit the scope of the present invention in any way.

[0050] The programs involved or relied upon in the following embodiments are all conventional or simple programs in the technical field, and those skilled in the art can make conventional selections or adaptive adjustments according to specific application scenarios.

[0051] Example 1: A database query optimization method. Join operations are a relatively common query type in relational databases. Normally, the number of joined tables does not exceed 20. However, in actual applications, there may be dozens or even hundreds of join queries. Existing methods generally rely on estimating the scale of intermediate query results. In order to reduce the delay time of plan formulation, the optimization effect of the plan will usually be affected to a certain extent.

[0052] When implementing its query optimizer, the MySQL database incorporated heuristic optimization into the Cost Based Optimizer (CBO). The Oracle optimizer also employs heuristic query optimization based on built-in rules. This approach persists throughout the entire query optimization phase, requiring extensive tuning and maintenance. System R uses dynamic programming for query optimization, parsing the plan structure into a left-deep join tree. However, this approach is often limited by the cost model and, under nonlinear cost curves, can result in missed plans that produce poor query quality. These methods cannot obtain feedback from actual query join plans executed in the past, preventing the optimizer from gaining effective information from past experience.

[0053] To address these issues, some researchers have applied various reinforcement learning algorithms to join order optimization. This approach can fundamentally reduce algorithmic overhead and improve query performance by leveraging feedback from past query executions. Ortiz et al., by training a model to predict the cardinality estimate of join query plans, proposed using a deep neural network to incrementally learn subquery state representations and using reinforcement learning to improve the join order. However, using query size as the cost function limits query exploration and can lead to suboptimal global query plans. Marcus et al. proposed a neural optimizer (Neo) that iteratively searches for the execution plan with the lowest relative cost using a best-first search algorithm. However, generalization of the optimizer across different databases is difficult. Subsequently, they proposed a join order enumerator, ReJOIN, that uses a policy gradient algorithm to learn a join action selection strategy. It first completes all joins, then calculates the reward for the entire join plan based on a cost model. This strategy requires extensive data training, and the join query plan is significantly affected by the cost estimate. Trummer et al. adopted an adaptive query processing strategy that does not maintain data statistics or use cost or cardinality models. Krishnan et al. used the DQN model to optimize join order, achieving superior performance with less training data than traditional methods. However, the encoding method used failed to capture the structural information of the join tree. Guo et al. proposed integrating display representation self-join encoding and beam search into join order optimization, improving model robustness and mitigating inherent model flaws. However, this method suffers from overestimation, which can lead to poor query plans.

[0054] The DQN model is a groundbreaking work in DRL. It uses a deep neural network to learn the Q matrix and introduces two mechanisms: experience replay (learning memory) and a target network. This alleviates the convergence fluctuations that occur when neural networks approximate the Q function. The DDQN algorithm addresses the overestimation that can occur in DQN. Therefore, integrating the DQN and DDQN models to optimize the join order in queries can further improve the performance of join queries.

[0055] The main function of the query optimizer is to improve the execution efficiency of SQL statements. The optimization process is divided into two stages: logical query optimization and physical query optimization. This embodiment focuses on the stage where the optimizer generates a physical query execution plan. The improved DRL model in DDOS (Dynamic Double DQN order select, DDOS) is used to guide the selection of the join order and output a predicted join plan for a given SQL query. Figure 1First, the SQL statement query information and the connection operation information represented by the left and right states are initialized. This information is then fed into the neural network to receive the query connection state. The training phase then begins, using a progressive search strategy to select connection actions. Different SQL statements have different network parameters and rewards. DDOS evaluates connection actions for subsequent queries based on the trained DRL model, aiming to maximize the reward, thereby developing an SQL connection plan. The physical query execution plan is then completed by adding plan nodes such as Agg and Group. Finally, the executor executes the query according to the physical execution plan.

[0056] DDOS's problem-solving philosophy is consistent with DQN. It requires formalizing the connection order problem into a series of value-based RL definitions and modeling the connection order problem as a Markov decision process (MDP). Q-learning is used to solve the Markov problem of connection order. In the MDP model, the agent starts from the initial connection state S0, interacts with the environment, observes the reward value after taking actions, and learns to obtain more reward strategies by changing its behavior. Finally, by taking a series of connection actions (a1, a2, a3, ..., a r ) and connect all the relations in the query, with the goal of minimizing the cost of the join operation and the execution time. Figure 1 As shown in the figure, after the SQL statement is input, it is first vectorized and initialized. This process does not require the addition of additional information. By interacting with the environment, the evaluation network evel_net obtains sample data of reward values, which is used to update the evaluation network. After a fixed number of time steps, the evaluation network parameters are copied to the target network target_net. The target value network is used to calculate the action value with the maximum Q value in the current state. Based on the exploration strategy, the connected action is selected and applied to the environment to obtain the new state. This process is repeated to obtain the final connection plan. Before being submitted to the executor, the query execution plan is completed by adding a plan node. When using the gradient descent function to update the evaluation network, a weighted method is used to balance the weights of the DQN and DDQN network parameters, effectively reducing estimation errors and improving the stability of the neural network.

[0057] To express the connection order as a problem that can be solved by DRL, it is necessary to first encode the database and query information as the input of the neural network. When encoding the query, vectorized expression is used. When designing the action selection of the intelligent agent, a tree structure representation method is used. Each connection query can be represented by a defined query tree. The connection between tables is defined as joining the connection tree, and the leaf node is defined as the table joining the connection. A query is a binary tree, in which each table is a leaf node, and T = {K, R} is a query tree. A connection of the table is represented as R = {K, R}. Each time a table connection is added, the corresponding relationship tree is inserted into the leaf and leaf node. And r i and r j After the corresponding leaf node is inserted, it will be removed from the action selection. When a new leaf is added, it will be added with (r i ,r j ) The corresponding subtrees are first connected. This preserves the sub-connection structure information between tables.

[0058] Each table to be connected is considered as a connection candidate, and all combinations of connecting all candidates are used to connect the tables to be connected. When the total number of a database table is n, the n-dimensional vector action selection is used to express the action space. During the connection process, the vector value is for the subtree corresponding to the connection. When a table is added to the connection, the action value of the corresponding position in the subtree is changed to If the corresponding connection table is not added to the subtree, the value is 0, and h is the height value of the subtree relationship when connecting.

[0059] like Figure 2 As shown, in the third line of state s3, the corresponding connection The 3 rows corresponding to the o table have a value of 1 / 4, the height of the subtree it is in is 2, and the corresponding value of n without connection is 0. Compared with the data characterization method of existing learning optimizers (such as DQ, in which each column of the database is used as a single feature, a state is represented as a binary one-hot vector, in which each number represents a column of the database.) This action behavior can well preserve the hierarchical information between tables, facilitate the capture of the structural information of the connection tree, and reduce the cost of the optimizer in cost estimation. At the same time, after training data containing local connections, the combination can be expanded to multi-table connections with a larger number of connections. This embodiment combines Multiple 1-hot pairs to implement self-join query encoding. The left and right relationships, physical operations, and the size of the intermediate relationships of the join action are represented as follows: A L and A R It represents the left and right relationship, using an n-dimensional vector, is a physical connection operation, using one-hot encoding, and c is the cardinality estimate of the connection intermediate relationship.

[0060] For the construction of DDOS network model, such as Figure 3As shown in the figure, in a standard DQN network, action selection and evaluation are both based on the same parameters, leading to overestimation of Q-values. This results in a deep Q-network's tendency to overestimate action values. In the Dual DQN proposed by Haselt et al., the estimation strategy is based on the online Q-network, while action selection uses the target network to estimate Q-values. This significantly alleviates the overfitting problem, but it also leads to the problem of underestimating action values. Both overestimation and underestimation can affect the learned strategy, resulting in reduced performance. During the logic optimization phase, to mitigate the negative impact of both DQN's overestimation and DDQN's underestimation, the concept of a weighted dual estimator is applied to Dual DQN, constructing a network that accurately screens and significantly improves the performance of the current query plan. By calculating action-state pairs and training a neural network to regress Q-values, this approach retains the characteristics of both DQN and DDQN, balancing their weights to reduce the error in target estimates and achieve more accurate Q-value estimates. The vectorized query plan tree is fed into a fully connected linear layer through the input layer of the neural network and the state information of the two hidden layers, and then the vector is mapped to the performance cost prediction using the ReLU activation function and layer normalization.

[0061] In this embodiment, the dual DQN network structure is still used, including the evaluation network and the target network, without adding an additional network. The purpose of the evaluation network is to mine the action corresponding to the maximum value. The target network is responsible for evaluating the maximum action value. * ;θ) and Q(s',a * ; θ-) to calculate the target value, the calculation formula is as follows:

[0062] y WDDQN =r+γ[βQ(s',a * ;θ)+(1-β)Q(s',a * θ - )] (1)

[0063] Among them, β is defined as the weight. The action selection is calculated by weighting the evaluation network and the target network. The range of β is [0,1]. When β is 0, the network is equivalent to the DDQN network. When β is 1, the algorithm completely ignores the Double DQN evaluation and only uses the DQN network to select actions. The weight β calculation formula is

[0064]

[0065] Action a * Denotes the action with the maximum action value for the evaluation network, a L represents the action with the minimum action value for the evaluation network, calculated as

[0066] a * =arg maxQ(s',a;θ) (3)

[0067] a L =arg minQ(s',a,θ) (4)

[0068] Therefore, Q(s',a * ,θ - ) represents the maximum action value in the target network at state s', Q(s',a L ,θ - ) represents the minimum action value for state s' in the target network. c is a hyperparameter used to calculate the weight β. The optimal value of c varies for problems with different characteristics. The hyperparameter c affects the network's weight allocation. The value of c is selected based on the characteristics of the connection task and after multiple experiments.

[0069] When solving query optimization, the traditional bottom-up optimization idea is used. When obtaining the Q value, the Q value is obtained based on the cost model and the execution delay caused by the action.

[0070]

[0071] When updating the Q value, the difference in the loss function is calculated by the mean square error. The traditional optimizer cost model sometimes has deviations, but it is still a good choice for training neural network models. The reward corresponding to the query action is obtained according to the cost model. Where r is the reward, α is the learning rate, which is used to determine the size of the error learning γ represents the learning rate, and max a' Q(s',a') represents the target Q value, Q(s,a) represents the estimated Q value, and the Q value update process is as follows:

[0072] Q(s,a)←Q(s,a)+α[r+γmax a' Q(s',a')-Q(s,a)] (6)

[0073] The loss function is defined as the difference between the estimated Q value and the actual Q value, that is, the mean square error L(θ):

[0074] L(θ)=E[||r+γmax a' Q(s',a';θ)-Q(s,a;θ||)] (7)

[0075] A dynamic progressive search strategy is used to train the DDOS network model. The ε-greedy strategy incorporates random action selection with a certain probability. Due to the role of the parameter ε in the Q algorithm, using a fixed value for each update reduces the agent's adaptability. When faced with complex scenarios, it becomes difficult to explore at large distances while maintaining the same ε value. Setting ε too low slows the network's learning rate, while setting it too high causes the agent's performance to decrease exponentially with increasing distance from the initial state. This results in exploration far from reaching a deep state, increasing the likelihood of reaching a local optimum. To avoid meaningless exploration, uncertainty must first be assessed. In unexplored new states, uncertain and unexplored actions generally have a higher information gain. During exploration, this aspect is prioritized during training. Noisy is added to increase the randomness of each exploration step, enabling a more efficient exploration environment. Increasing the ε value from 0 to 1 transitions from exploration to exploitation, and dynamic switching between stages is achieved by adjusting the parameter ε. Therefore, an expression for controlling the variation of the parameter ε is designed.

[0076]

[0077] Where H is a continuous variable, and the probability of the difference in Q values between two alternative actions in a certain state is defined as ε(H). When H gradually decreases, the ε(H) of the Q value difference will also increase; conversely, ε(H) decreases. k ,a i+1 )-Q(s k ,a i )]<0, so the value range of ε(H) in formula (8) is (0, 1). By adjusting the parameter H to change ε, the Q algorithm can jump freely between the three stages of exploration-learning-exploitation.

[0078] First, determine the action participation candidates within the following action value range and set an adjustable threshold p, where U i (a) is the limit of the action in state s. When there is less training data in the early stage, the limit is set relatively large. As the training data increases, the estimation of the action becomes more and more accurate, and the limit will become smaller and smaller. The calculation formula is as follows:

[0079]

[0080] The dynamic progressive search strategy controls the transition of greediness from the exploration phase to the utilization phase by changing the greed parameter ε, and explores potential available actions as widely as possible during the learning process, so that the algorithm can avoid falling into the local optimal state to a large extent.

[0081] The training process of DDOS is as follows: Due to the cost model, it is possible to quickly give the estimated cost in a short time. Although there is a partial deviation from the actual delay in terms of calculation accuracy, the calculated cost is still a reference. In addition, in order to alleviate the estimation deviation caused by the cost model, this embodiment adds execution delay training. Each query execution requires a long time to obtain actual feedback. Therefore, only after the cost training, and combined with the data collected in the previous cost model training as a guide, the network weights of the output layer are reset, and the execution delay is used for fine-tuning, and retrained on a small amount of data (queries with actual execution time). After the training is completed, the model can be used to optimize database queries. The SQL statement query information and the connection operation information represented by the left and right states are input into the model so that it receives the connection status of the query. The model then outputs the physical query execution plan to the executor for execution to complete the database query.

[0082] The present invention has been described in detail above with reference to the accompanying drawings and embodiments. However, those skilled in the art will appreciate that, without departing from the spirit of the present invention, the specific parameters in the above embodiments may be modified to form multiple specific embodiments, which are all within the common variation range of the present invention and will not be described in detail here.

Claims

1. A database query optimization method, characterized in that: The state and action information of the database query are input into the dynamic dual DQN network model. The state and action information of the database query are used as the input of the model. When encoding the query, vectorized expression is used. When designing the intelligent agent to select the action, a tree structure representation method is used. The connection query is represented by a query tree. The connection between tables is defined as joining the connection tree. The leaf node is defined as the table to be joined. When a new leaf node is added, it is first connected to the corresponding subtree to retain the sub-connection structure information between the tables. Among them, the dynamic dual DQN network model includes two parts: the evaluation network and the target network. The evaluation network is used to mine the action corresponding to the maximum value of Q in the dual DQN network. The target network is used to evaluate the maximum action value in the dual DQN network. Q(s', a * ;θ) and Q(s',a * θ - ) to calculate the target value as follows: and WDDQN =r+γ[βQ(s',a * ;θ)+(1-β)Q(s',a * ;θ - )] Among them, Q(s',a * ,θ - ) represents the maximum action value in the target network at state s', Q(s',a L ,θ - ) represents the minimum action value of state s' in the target network, β is defined as the weight, and the range of β belongs to [0,1]. When β is 0, the network is equivalent to the DDQN network. When β is 1, the model completely ignores the Double DQN evaluation and only uses the DQN network to select actions. The weight β is calculated as follows: Among them, action a * Denotes the action with the maximum action value for the evaluation network, a L represents the action with the minimum action value for the evaluation network, a * argmaxQ(s',a)θ) a L =argminQ(s',a,θ) Where c is a hyperparameter used to calculate the weight β; The Q value is derived based on the cost model and the execution delay caused by the action: And the updated value of Q: Q(s,a)←Q(s,a)+α[r+γmax a' Q(s',a')-Q(s,a)] Among them, r is the reward, α is the learning rate, which is used to determine the size of the error learning γ represents the learning rate, max a' Q(s',a') represents the target Q value, Q(s,a) represents the estimated Q value, The loss function is defined as the difference between the estimated Q value and the actual Q value, that is, the mean square error L(θ): L(θ)=E[||r+γmax a' Q(s',a';θ)-Q(s,a;θ)||].

2. The database query optimization method according to claim 1, characterized in that: For the hyperparameter c, the optimal value of c is different in problems with different characteristics. The hyperparameter c affects the network allocation weight. The size of the hyperparameter is set according to the characteristics of the connection task, and the value of c is selected based on multiple experiments.

3. The database query optimization method according to claim 1, characterized in that: The evaluation network selects actions through a dynamic progressive search strategy. First, the uncertainty is evaluated, and the uncertain and undiscovered actions in the unexplored period are taken as the focus of the training period. The exploration to utilization stage can be achieved by increasing the ε value from 0 to 1. The dynamic switching between stages can be achieved by adjusting the parameter ε. The expression for controlling the change of the parameter ε is Among them, H is a continuous variable, and the probability of the difference in Q values between two alternative actions in a certain state is specified as ε(H), and the value range of ε(H) is (0, 1).

4. The database query optimization method according to claim 3, characterized in that: Set an adjustable threshold p, where U i (a) is the limit of the action in state s. When there is less training data in the early stage, the limit is set relatively large. As the training data increases, the estimation of the action becomes more and more accurate, and the limit will become smaller and smaller. The calculation formula is as follows:

5. The database query optimization method according to claim 1, characterized in that: During the model training phase, only after cost training and with the data collected from the previous cost model training as a guide, the network weights of the output layer are reset, fine-tuned using execution delays, and then retrained on query data with actual execution runtime.

6. A database query optimization system, utilizing the database query optimization method according to claim 1, characterized in that: include: Database query statement input module, used to characterize the actions and states of database queries; Network environment module, used for data interaction within the model; An evaluation network module, configured to interact with the network environment module to obtain sample data of reward values, thereby updating the evaluation network; The target network module is used to obtain the parameters of the evaluation network module, and then calculate the action value with the maximum Q value in the current state. After selecting the connected action according to the exploration strategy, it acts on the network environment module and obtains a new state; The output module is used to hand over the final connection plan to the executor for execution.

7. An electronic device, characterized in that: include at least one actuator; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one executor, and the instructions are executed by the at least one executor to enable the at least one executor to execute the database query optimization method according to any one of claims 1 to 5.

8. A storage medium, characterized in that: When the instructions in the storage medium are executed by an executor of a computer, the computer is enabled to execute the database query optimization method according to any one of claims 1 to 5.

Images