Method and system for optimizing distributed database table join order and join operator

Abstract

The application discloses a distributed database table connection sequence and connection operator optimization method and system, belongs to the field of database query optimization, and comprises the following steps: encoding a query statement to obtain a statement feature vector; encoding data distribution of a data table to be connected to obtain a distribution feature vector; generating a connection forest and a forest feature vector according to the statement feature vector; generating a current state feature vector of a current connection state according to the statement feature vector, the distribution feature vector and the forest feature vector; constructing a deep reinforcement learning model based on Tree-LSTM according to the current state feature vector; selecting N optimal connection actions according to a Beam Search algorithm to generate N corresponding next-time connection states, repeating the process until an execution plan is generated; and taking an execution time of the execution plan as a feedback signal to optimize the deep reinforcement learning model. The application can improve the efficiency and accuracy of determining a data table connection sequence.

Description

Technical Field

[0001] This application relates to the field of database query optimization, specifically to a method and system for optimizing the join order and join operators of a distributed database table. Background Technology

[0002] Query optimization is a core aspect of query execution in relational databases. When handling complex queries, how the database system chooses the most efficient way to execute the query directly impacts query response time and system resource utilization. The goal of query optimization is to find the lowest-cost execution path among numerous possible query execution plans to reduce resource consumption and response time. The optimizer evaluates the cost of different query strategies, uses statistical information and cost models to predict the resource consumption of various execution strategies, and thus makes a decision to generate the optimal execution plan.

[0003] Determining the table join order and selecting join operators are crucial in multi-table query optimization. Join operations are among the most time-consuming operations in multi-table queries, and different join orders and join operators (such as nested loops, hash joins, and merge joins) can significantly impact query performance. A correct join order can significantly reduce the size of intermediate results, lower I / O costs and computational overhead, thereby accelerating query processing. Therefore, the query optimizer needs to comprehensively consider factors such as table size, the existence of indexes, and data distribution, and use algorithms to calculate the optimal table join order and suitable join operators.

[0004] However, current join order selection algorithms face several limitations. First, these algorithms often rely on the accuracy of cost models and statistical information, which may not be entirely accurate in practical applications, leading to optimization results deviating from the optimal solution. Second, for queries involving a large number of table joins, the space for searching the optimal join order is extremely large. Dynamic programming algorithms, by systematically exploring all possible join orders, can guarantee finding the lowest-cost query execution plan based on an accurate cost model, thus achieving optimal query performance. While heuristic-based pruning algorithms reduce optimization time, they may miss the optimal solution, resulting in a poor execution plan. Furthermore, with the growth of data scale and the development of distributed databases, data may be distributed across different physical locations. Query optimization needs to consider factors such as data distribution, network transmission costs, and the parallel processing capabilities between nodes, making query optimization more complex and the selection of join order and join operators more difficult. The limitations of existing algorithms in terms of efficiency and performance are becoming increasingly apparent, urgently requiring new technologies and methods to improve the optimization efficiency of multi-table join queries. Summary of the Invention

[0005] This application provides a method for optimizing the join order and join operators of distributed database tables based on reinforcement learning, in order to solve the problem of low query execution efficiency caused by inaccurate selection of join order and join operators in existing distributed query optimizers.

[0006] In a first aspect, embodiments of this application provide a method for optimizing the table join order and join operators in a distributed database, the method comprising:

[0007] The query statement is encoded to obtain a statement feature vector;

[0008] Generate a connection tree and its tree feature vectors based on the statement feature vectors;

[0009] Encode the data distribution of the tables to be joined to obtain the distribution feature vector;

[0010] Generate a connection forest and its forest feature vectors based on the statement feature vectors and tree feature vectors;

[0011] Generate the current state feature vector of the current connection state based on the statement feature vector, distribution feature vector, and forest feature vector;

[0012] Construct a deep reinforcement learning model based on Tree-LSTM based on the current state feature vector;

[0013] The Beam Search algorithm selects the N optimal join actions to execute, generating N corresponding next-time join states, where N is a positive integer not less than 2. This process is repeated until an execution plan is generated that connects all the tables to be joined. This execution plan includes the join order and join operators.

[0014] After the execution plan is imported into the distributed database and executed, the execution time is used as a feedback signal to optimize the deep reinforcement learning model.

[0015] In conjunction with the first aspect, in one implementation, encoding the query statement to obtain a statement feature vector specifically includes the following steps:

[0016] The SELECT, WHERE, and JOIN clauses in the query statement are analyzed to extract the data table to be joined, the target column, and the corresponding predicate conditions. The predicate conditions include the attribute selection predicate and the join conditions of the data table to be joined.

[0017] The column-related information of the target column is encoded to obtain the column feature vector. The column-related information includes the table name, selection predicate, and join condition of the data table to be joined associated with the target column.

[0018] Summarize the column feature vectors of all target columns to obtain the table feature vector of the data table to be joined;

[0019] The predicate condition and the connection condition are encoded separately to obtain the predicate feature vector and the condition feature vector.

[0020] In conjunction with the first aspect, in one implementation, the step of generating a connection tree and its tree feature vector based on the statement feature vector specifically includes the following steps:

[0021] The data structure of the connection tree is designed based on the column feature vector and the table feature vector. This data structure is a Tree-LSTM structure. An internal node in the connection tree corresponds to a connection operation. Each internal node includes four child nodes. Two child nodes represent two targets to be connected, which are either the connection tree or the data table to be connected. The other two child nodes represent the target columns in the two targets to be connected.

[0022] In conjunction with the first aspect, in one implementation, encoding the data distribution of the data tables to be joined to obtain a distribution feature vector specifically includes the following steps:

[0023] Extract histogram statistics, metadata, partition keys, and partitioning algorithms from the tables to be joined;

[0024] The histogram statistics, metadata, partition keys, and partitioning algorithms are encoded to obtain the distribution feature vector of the data distribution.

[0025] In conjunction with the first aspect, in one implementation, the connection forest includes multiple targets to be connected, which are connection trees or data tables to be connected;

[0026] The sets of data tables to be connected corresponding to each connection tree in the connection forest are all different.

[0027] In conjunction with the first aspect, in one implementation, the deep reinforcement learning model is trained using a deep Q-network algorithm.

[0028] In conjunction with the first aspect, in one implementation, the step of selecting the N optimal join actions according to the Beam Search algorithm, generating N corresponding next-time join states, and repeating this process until an execution plan for connecting all the data tables to be joined is generated, specifically includes the following steps:

[0029] After taking the preset next-time connection state as the current connection state, the feature vector of the current state is input into the deep reinforcement learning model to obtain candidate connection actions and their values.

[0030] According to the Beam Search algorithm, the N candidate actions with the highest value are selected as the best connection actions, where N is the preset Beam width;

[0031] Execute each optimal connection action to obtain N current connection states;

[0032] After repeatedly using the preset next-time connection state as the current connection state, the N optimal connection actions are selected and executed according to the Beam Search algorithm, generating N corresponding next-time connection states, until an execution plan for connecting all the data tables to be connected is generated.

[0033] Secondly, embodiments of this application provide an optimization system for the join order and join operators of a distributed database table, the system comprising:

[0034] The encoding module is used to encode the query statement to obtain the statement feature vector; it is also used to encode the data distribution of the tables to be joined to obtain the distribution feature vector.

[0035] The connection forest generation module is used to generate connection trees and their tree feature vectors based on statement feature vectors; it is also used to generate connection forests and their forest feature vectors based on statement feature vectors and tree feature vectors.

[0036] The model building module is used to construct a deep reinforcement learning model based on Tree-LSTM based on the current state feature vector; it is also used to select the N optimal connection actions to execute according to the Beam Search algorithm, generate N corresponding next-time connection states, where N is a positive integer not less than 2, and repeat this process until an execution plan is generated that connects all the data tables to be connected. This execution plan includes the connection order and connection operators; it is also used to import the execution plan into the distributed database for execution, and use the execution time as a feedback signal to optimize the deep reinforcement learning model.

[0037] In conjunction with the second aspect, in one implementation, the encoding module is used to analyze the SELECT, WHERE, and JOIN clauses in the query statement to extract the data table to be joined, the target column, and the corresponding predicate conditions, wherein the predicate conditions include the attribute selection predicate and the join conditions of the data table to be joined;

[0038] It is also used to encode the column-related information of the target column to obtain the column feature vector. The column-related information includes the table name, selection predicate, and join condition of the data table to be joined associated with the target column.

[0039] It is also used to summarize the column feature vectors of all target columns to obtain the table feature vector of the data table to be joined;

[0040] It is also used to encode predicate conditions and connection conditions respectively, to obtain predicate feature vectors and condition feature vectors.

[0041] In conjunction with the second aspect, in one implementation, the connection forest generation module is used to design a connection tree data structure based on column feature vectors and table feature vectors. This data structure is a Tree-LSTM structure. One internal node in the connection tree corresponds to one connection operation. Each internal node includes four child nodes, where two child nodes represent two targets to be connected, which are connection trees or data tables to be connected, and the other two child nodes represent target columns in the two targets to be connected.

[0042] The beneficial effects of the technical solutions provided in this application include:

[0043] First, encoding the query statement provides more accurate data features. Furthermore, encoding the data distribution of the tables to be joined provides more accurate data distribution features. By learning these data distribution features, the deep reinforcement learning model can effectively consider the complexity of data partitioning and network transmission in a distributed database environment, making action selection more accurate. Second, the Beam Search algorithm is used to expand the search path, thus avoiding the model getting trapped in local optima. Finally, a Tree-LSTM network is designed in the deep reinforcement learning model for action selection. The tree-structured neural network can better capture the tree structure information of the connection tree, while the LSTM network can better capture the hidden temporal information of the connection tree executed from bottom to top, thus more accurately predicting action value.

[0044] The optimal join order and join operator of the data table obtained by this invention result in lower query time and better query performance during actual queries. Attached Figure Description

[0045] Figure 1 Examples of table creation and query statements for distributed databases in existing technologies;

[0046] Figure 2 This is a schematic diagram illustrating two-stage optimization for distributed database queries in existing technologies.

[0047] Figure 3 This is a flowchart illustrating an embodiment of the method for optimizing the join order and join operators of a distributed database table in this application.

[0048] Figure 4 This is a flowchart illustrating a specific embodiment of the method for optimizing the order of table joins and join operators in a distributed database according to this application.

[0049] Figure 5 A flowchart illustrating a specific embodiment of the action selection and state update in this application;

[0050] Figure 6This is a functional module diagram of an embodiment of the system for optimizing the join order and join operators of a distributed database table in this application. Detailed Implementation

[0051] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0052] First, some technical terms in this application will be explained to facilitate understanding by those skilled in the art. Reinforcement learning (RL) is a machine learning paradigm that aims to enable an agent to learn how to make decisions to maximize long-term rewards through interaction with its environment. Deep Q-Network (DQN) is an algorithm that applies deep learning techniques to reinforcement learning. Beam Search is a heuristic search algorithm used to find optimal solutions in large search spaces; it is commonly used to solve optimization problems. Tree-LSTM is a deep learning model that is an extension of the Long Short-Term Memory (LSTM) model. The embodiments of this application are described below:

[0053] Querying is a crucial function in a Database Management System (DBMS), enabling users to retrieve information stored in the database and obtain the desired result set by executing query statements. Query performance directly impacts the overall efficiency of the database system; therefore, implementing query optimization—that is, reducing query response time by constructing the most cost-effective execution plan—is a key strategy for improving database performance. In multi-table queries, determining the optimal join order of the tables involved is particularly important. This is because different join orders result in different execution costs. As the number of tables involved in a query increases, the computational space for finding the optimal join order grows exponentially, necessitating efficient strategies to quickly determine the best solution.

[0054] Currently, most database systems rely on cost-based estimation methods to determine the order of table joins, typically combining dynamic programming algorithms and cost models. This strategy selects the lowest-cost execution path based on estimates of the costs of different join strategies. However, this estimation-based approach can encounter accuracy issues when dealing with large datasets or complex query conditions, sometimes leading to a far-from-optimal join order, thus impacting query performance and the efficient utilization of system resources. Furthermore, dynamic programming enumerates all possible join orders, and its computational complexity increases exponentially with the number of tables involved in the join; when the number of tables is excessive, its computation time and resource consumption become unacceptable.

[0055] Furthermore, this problem is even more complex and challenging in distributed databases and big data processing engines. Different data fragments may be distributed across different nodes, and improper join order and join operators can lead to a large amount of unnecessary data being transferred between nodes, severely impacting query performance. Since computing and storage resources are distributed across different nodes in the network, these methods need to further consider factors such as data distribution, node computing power, and network transmission costs, increasing the complexity of optimization.

[0056] Traditional distributed optimizers typically employ a two-stage optimization algorithm: The first stage assumes all tables are local and relies on existing single-machine query optimization capabilities to select a locally optimal execution plan. The second stage transforms this optimal single-machine execution plan into a distributed execution plan based on simple distributed rules. This two-stage optimization method does not consider data distribution information when generating join orders and join operators, potentially leading to the selection of suboptimal join orders and join operators.

[0057] In two-phase distributed query optimization, a common problem is that the first phase fails to adequately consider data partitioning information, leading to a suboptimal data join order. Figure 1 The query example shown, and Figure 2 Taking the corresponding local and distributed execution plans as examples: Tables R1 and R2 have the same partitioning method, and the partition key and join key are the same. R1, R2, and R3 represent different data tables, EX represents data transfer, and HJ represents hash join. (The text to the left of the dotted line is incomplete and likely refers to a different execution plan.) Figure 2 .a and Figure 2 The first execution plan represented by .b ​​adopts the connection order of ((R2,R3),R1), while the one to the right of the dashed line... Figure 2 The second plan, represented by .c, selects a connection order of ((R1,R2),R3). Without considering partition information, the first-stage optimizer might choose a connection order like ((R2,R3),R1), but this order, after the second-stage optimization, could result in higher network transmission costs, such as… Figure 2 As shown in .b, the connection results of tables R1 and R2, R3, and R2 and R3 all need to be transmitted over the network. A better sequential order might be ((R1,R2),R3), as shown in .b. Figure 2 As shown in .c, after the second stage of optimization, this connection sequence only needs to transmit the connection results of R3, R1, and R2. Since the partition keys and connection keys of R1 and R2 are the same, no network transmission is required; the connection can be performed on the local node.

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0059] In a first aspect, embodiments of this application provide a method for optimizing the table join order and join operators in a distributed database. In one embodiment, referring to... Figure 3 , Figure 3 This is a flowchart illustrating an embodiment of the method for optimizing the join order and join operators of a distributed database table according to this application. Figure 3 As shown, optimization methods for the table join order and join operators in distributed databases include:

[0060] Step S1: Encode the query statement to obtain the statement feature vector. Step S2: Generate a connection tree and its tree feature vector based on the statement feature vector. Step S3: Encode the data distribution of the tables to be joined to obtain the distribution feature vector. Step S4: Generate a connection forest and its forest feature vector based on the statement feature vector and the tree feature vector. Step S5: Generate the current state feature vector of the current connection state based on the statement feature vector, the distribution feature vector, and the forest feature vector. Step S6: Construct a deep reinforcement learning model based on Tree-LSTM based on the current state feature vector. Step S7: Select the N optimal connection actions according to the Beam Search algorithm and execute them, generating N corresponding next-time connection states. Repeat this process until an execution plan for connecting all tables to be joined is generated. The execution plan includes the connection order and connection operators. N is a positive integer not less than 2. Step S8: After importing the execution plan into the distributed database for execution, use the execution time as a feedback signal to optimize the deep reinforcement learning model. If the deep reinforcement learning model still fails to reach the ideal model state after step S8 optimization, return to step S7 to optimize the model again.

[0061] In this embodiment, the multiple data tables to be joined and the target columns are identified by parsing the query statement. Table feature vectors are extracted for each data table to be joined, and column feature vectors are extracted for each target column. Distribution feature vectors are generated based on the statistical information and partitioning information of the data tables to be joined. Based on these feature vectors, a reinforcement learning method based on Tree-LSTM is used to decompose the join order and join operator selection problem into a series of actions. The Beam Search algorithm is used to expand the search path, progressively selecting tables to join pairwise until a complete join tree containing the join order and join operators is generated. Subsequently, the execution plan corresponding to the generated join tree is applied to the distributed database for execution. Execution time is used as feedback to optimize the Tree-LSTM network to generate better join orders and join operators.

[0062] During the validation phase, the trained deep reinforcement learning model is applied to new datasets or real-world query scenarios to evaluate its performance in real-world environments, including metrics such as execution time and resource consumption. The execution plans generated by the model are compared with baseline execution plans to verify whether the model can generate better execution plans under different conditions.

[0063] Furthermore, in one embodiment, step S1 specifically includes the following steps:

[0064] The SELECT, WHERE, and JOIN clauses in the query statement are analyzed to extract the data tables to be joined, the target columns, and the corresponding predicate conditions. The predicate conditions include the attribute selection predicate and the join conditions of the data tables to be joined.

[0065] The column-related information of the target column is encoded to obtain the column feature vector. The column-related information includes the table name of the data table to be joined associated with the target column, the selection predicate, and the join condition.

[0066] Summarize the column feature vectors of all target columns to obtain the table feature vector of the data table to be joined.

[0067] The predicate condition and the connection condition are encoded separately to obtain the predicate feature vector and the condition feature vector.

[0068] In this embodiment, the query statement is encoded to generate column feature vectors for each target column, table feature vectors for the data tables to be joined, predicate feature vectors for the selection predicate, and condition feature vectors for the join conditions.

[0069] Attribute and predicate extraction: Analyze the SELECT, WHERE, and JOIN clauses in the SQL statement to extract the relevant tables, columns, and corresponding predicate conditions. Predicate conditions include attribute selection predicates and table join conditions. Each attribute is encoded based on its predicate type and frequency of use.

[0070] Column Feature Encoding: All columns extracted in the previous step are encoded and vectorized, including the table name associated with the column, join conditions, and selection predicate information. Here, join conditions cover information about joining with a specific column of a specific table, while selection predicates assign a unique numerical code to each type of predicate (e.g., >, <, =, BETWEEN, IN, etc.). Feature codes are generated for each column based on this information. Based on the encoding information from the previous step, the selectivity (i.e., filtering effect) of each predicate associated with the column is estimated, which can be based on historical query statistics, metadata, or cardinality estimation methods. Information such as the range of column attribute values, the number of unique values, and the correlation with other attributes is processed. Feature vectors are generated for each column based on this information.

[0071] Feature matrix construction: Based on the column feature vectors generated in the previous step, all column feature vectors are summarized to form a feature matrix that comprehensively describes the data table. This matrix provides a comprehensive feature view of all relevant columns in the data table, summarizing the feature information of all relevant columns and providing basic data for subsequent processing. Next, starting from the feature matrix, principal component analysis (PCA) is used to transform it into a simplified one-dimensional feature vector. This captures the core information of the data table while reducing the complexity of model processing, facilitating subsequent analysis and calculation.

[0072] Conditional Feature Vector Generation: Parses the inter-table join conditions specified in the WHERE clause of an SQL query, encodes these join relationships in detail, and thus numerically expresses how different data tables are connected to each other through specific conditions.

[0073] Through the steps described above, this method successfully transforms the core content of SQL queries into feature vectors during the data preprocessing stage, meeting the data requirements of machine learning models while comprehensively and accurately capturing key information in the query. This transformation not only improves the accuracy of information extraction but also covers the details of the SQL statement, providing comprehensive and accurate data support for optimizing the order of table joins. By transforming the data tables and their join relationships in the SQL query into a series of feature vectors, this method lays the foundation for using reinforcement learning to automatically optimize database query execution plans. These feature vectors meticulously reflect the key attributes of the data tables and the join patterns between them, providing the comprehensive information needed for query optimization and enabling the model to make optimization decisions from a more holistic perspective.

[0074] Furthermore, in one embodiment, step S2 specifically includes the following steps:

[0075] The data structure of the connection tree is designed based on the column feature vectors and table feature vectors. This data structure is a Tree-LSTM structure. One internal node in the connection tree corresponds to one connection operation. Each internal node includes four child nodes. Two child nodes represent two targets to be connected, which can be the connection tree or the data table to be connected. The other two child nodes represent the target columns in the two targets to be connected.

[0076] In this embodiment, the data structure and encoding method of the connection tree are designed based on the column feature vector, table feature vector, predicate feature vector, and condition feature vector to generate the tree feature vector of the connection tree.

[0077] Based on the column's feature vector R(c) and the data table's feature vector R(t), design a connection tree data structure that can represent the connection relationship between tables.

[0078] The connection tree is represented using a Tree-LSTM, where leaf nodes can be either a table or a column. In the connection tree, an internal node corresponds to a join operation and consists of four child nodes (t0, c0, t1, c1). Nodes t0 and t1 represent the two connection trees or tables to be joined. Nodes c0 and c1 correspond to the two columns to be joined in this connection. Through these steps, this method can generate feature vectors for the connection tree, which better reflect the result of joining two data tables with a connection relationship, providing basic elements for subsequently constructing partial join plans.

[0079] Furthermore, in one embodiment, step S3 specifically includes the following steps:

[0080] Extract histogram statistics, metadata, partition keys, and partitioning algorithms from the tables to be joined.

[0081] The histogram statistics, metadata, partition keys, and partitioning algorithms are encoded to obtain the distribution feature vector of the data distribution.

[0082] In this embodiment, the metadata information of the table (index, number of rows, number of distinct values, etc.) is encoded to generate a feature vector.

[0083] The partitioning information of the table is encoded, including the partition key (which column to create the partition on) and the partitioning method (such as range partitioning, hash partitioning, etc.).

[0084] The histogram statistics of the table are encoded to generate feature vectors.

[0085] Through the above steps, this method generates feature vectors for the table's metadata, partitioning, and statistical information, which can more accurately extract key information about the data distribution and provide accurate and comprehensive data features for subsequent steps.

[0086] Furthermore, in one embodiment, the connection forest includes multiple targets to be connected, which are connection trees or data tables to be connected.

[0087] The sets of data tables to be connected corresponding to each connection tree in the connection forest are all different.

[0088] In this embodiment, a forest feature vector for a connection forest is generated based on the feature vectors of multiple data tables to be joined, the column feature vector of the target column, and the feature vectors of the already joined connection trees. This connection forest consists of several connection trees or data tables to be joined, and the set of data tables to be joined corresponding to each tree is distinct.

[0089] Based on the feature vectors of the query statement, the data distribution information, and the connection forest, a state feature vector for the current connection state is generated.

[0090] Through steps S4 and S5, this method uses a connection forest model to generate feature vectors for the connection forest, incorporating its tree structure information. Combined with subsequent steps, the connection plan tree for the next time step can be continuously updated, allowing for dynamic expansion of its structure. This enables more accurate generation of feature vectors for the connection states at the next time step. These feature vectors contain key information about data distribution, key attributes of data tables, and connection relationships between tables, providing strong data support for the reinforcement learning model to select connection order and connection operators.

[0091] Furthermore, in one embodiment, the deep reinforcement learning model is trained using a deep Q-network algorithm.

[0092] Furthermore, in one embodiment, step S7 specifically includes the following steps:

[0093] After taking the preset next-time connection state as the current connection state, the feature vector of the current state is input into the deep reinforcement learning model to obtain candidate connection actions and their values.

[0094] According to the Beam Search algorithm, the N candidate actions with the highest value are selected as the best connection actions, where N is the preset Beam width.

[0095] Execute each optimal connection action to obtain N current connection states.

[0096] After repeatedly using the preset next-time connection state as the current connection state, the N optimal connection actions are selected and executed according to the Beam Search algorithm, generating N corresponding next-time connection states, until an execution plan for connecting all the data tables to be connected is generated.

[0097] In this embodiment, in step S6, the deep reinforcement learning model is trained using the Deep Q-Network algorithm (DQN). The model adopts a Tree-LSTM network structure and uses the Beam Search algorithm to expand the search path, so that the initial model learns how to select connection actions with lower costs to obtain a better data table connection order during the model training process.

[0098] The feature vector of the next connection state is used to replace the feature vector of the current connection state.

[0099] Action generation: N new feature vectors of the current connection states are input into the Tree-LSTM network. For a given state, all possible actions are identified. A connection action represents selecting two currently connected tables within the WHERE clause to join. The network also selects which distributed connection operator (e.g., broadcast, repartition) or local connection operator (e.g., hash join, nested loop join) to use. A reinforcement learning model is then used to predict the potential value of each action.

[0100] Action filtering: Based on the value predicted by the model, the top N actions with the highest value are generated and filtered out from all possible connection actions as candidates, where N is the preset beam width.

[0101] Path expansion: Execute the action for each candidate, resulting in the generation of N new states. These newly generated states, based on the executed actions and previous states, provide the foundation for the next round of beam search.

[0102] Repeated execution: Repeat the process of action generation, action filtering, and path expansion until all tables have been effectively joined, and then terminate to obtain the final join plan tree.

[0103] It's important to note that in each round of the search, regardless of whether we start from the initial state or any intermediate state, we only retain N optimal candidate solutions, rather than selecting N candidate actions for every newly generated state. This avoids the exponential growth of the search path. In this way, we gradually construct the complete data table join order and join operators.

[0104] Through the above steps, this method determines the feature vector of the current connection state based on the feature vector of the current connection plan, the data distribution vector, and the feature vector of the query statement in each round of connection action selection. The Tree-LSTM network then determines the candidate connection actions for this round based on the feature vector of the current connection state combined with the Beam Search algorithm, ensuring that the execution cost of each connection action is relatively small, thereby ensuring that the final target connection plan contains a better connection order and a better connection operator.

[0105] The main purpose of incorporating Beam Search into reinforcement learning is to consider multiple candidate actions in the decision-making process, rather than a single optimal action. Beam Search achieves this by maintaining a set of most promising actions (i.e., the beam) at each decision point, thus allowing for the exploration of a wider range of possibilities in determining the optimal policy.

[0106] Reinforcement learning neural networks can directly obtain connection actions, significantly reducing search costs compared to dynamic programming methods, thus shortening optimization time. Furthermore, after training, neural networks can learn poor connection plans and avoid repeating them – a learning experience lacking in traditional query optimizers. In the connection action selection step, this method introduces the Beam Search algorithm to broaden the search path, avoiding getting trapped in local optima and making the reinforcement learning model more likely to converge. More importantly, by learning the data distribution characteristics, deep reinforcement learning models, after training, can consider issues such as partitioned storage and network transmission in distributed databases, thus surpassing the performance of traditional query optimizers in connection order and join operator selection, thereby solving the problem of slow query execution speed.

[0107] In one specific embodiment, refer to Figure 4 , Figure 4 This is a schematic diagram illustrating a specific embodiment of the method for optimizing the order of table joins and join operators in a distributed database according to this application.

[0108] The method for selecting the order of data table joins includes both the model training phase and the model application phase.

[0109] During the model training phase, query statements in the sample data are analyzed to generate corresponding statement feature vectors. Table feature vectors are constructed for each table in the database, and column feature vectors are generated for each column of each table. Then, connection trees and connection forest vectors are generated based on the table feature vectors and column feature vectors. Next, we use the Deep Q-Network algorithm and the Beam Search algorithm to train the initial model, training an optimized reinforcement learning model based on the sample data.

[0110] In the application phase of the model, a new query statement is received, and the data tables to be joined, target columns, selection conditions, and join relationships are extracted from the query statement. This generates column feature vectors for the target columns and table feature vectors for the multiple data tables to be joined. Then, a join tree and join forest vectors are generated based on the table feature vectors and column feature vectors. Combining the join tree, join forest vectors, and feature vectors of the data distribution, the previously trained reinforcement learning model is used to determine the optimal table join order and join operator. The optimal join order refers to the best order in which the multiple data tables specified in the query statement are joined during query operation, thus considered the optimal solution selected from all possible table join orders. The optimal join operator refers to the join operator with the lowest execution cost for each join during query operation, thus considered the optimal solution selected from all possible table join operators.

[0111] This method specifically includes the following steps:

[0112] In step S1, the query statement is encoded to generate column feature vectors for each target column, table feature vectors for the tables to be joined, feature vectors for the selection predicate, and feature vectors for the join conditions. This step aims to successfully transform the core content of the SQL query into feature vectors during the data preprocessing stage, meeting the data requirements of the machine learning model while comprehensively and accurately capturing key information in the query. This provides comprehensive and accurate data support for optimizing the table join order and join operators.

[0113] Specifically, the target column and the data table are two key components of a query. After constructing the feature vector R(c) of column c, we can construct the feature vector R(t) of table t using the column's feature vector. The specific details of column encoding and feature vectors are as follows: In a query, columns can participate in join and selection operations, both of which need to be considered.

[0114] For a column c, the selection operation can be divided into five cases: >, <, =, ! =, and between and. Therefore, we encode c as a feature code of length 6.

[0115] in, This indicates that column c has a join predicate; otherwise, it is 0. If c == 1, it means that column c has a selection predicate with the operator =. Similarly, if c >= 1, it means that column c has a selection predicate with the operator =. If the selection predicate is between and, then c >= 1. caid The value is the cardinality of column c after being filtered by the selection predicate, i.e., the number of rows.

[0116] The column feature encoding is processed to generate a column feature vector R(c) = Sigmod(w×F(c)+b), where R(c) is the column feature vector, Sigmod() is the activation function, F(c) is the column feature encoding, and w and b are parameters.

[0117] For a table t with k columns, we use the feature vectors of each column to construct the table's feature vector R(t). Specifically, we use the feature vectors of the k columns R(c1), R(c2), ..., R(c... k The elements are combined into a single matrix M(t), with a size of (k, h). s Then, an average pooling layer is applied to obtain the feature vector R(t) = MaxPool(M(t)) of the table.

[0118] The steps for encoding the join relationships and generating feature vectors are as follows: We construct a representation R(q) of query q, which is a vector containing the join information of all tables in the query. Let the total number of tables in the database be n, and each table be labeled with a unique identifier from 0 to n-1. We define an n×n matrix m, whose elements m... i,j If the value is 1, it indicates a connection between the i-th and j-th tables; otherwise, it is 0. This matrix is ​​then transformed into a one-dimensional vector v, such that v... i×n+j =m i,j And the dimension of vector v is n×n.

[0119] Furthermore, to obtain the specific feature vector of the query, we apply a fully connected layer (FC) to the vector v, R(q) = FC(v) = σ(Wv + b), where represents the activation function used in the neural network, such as Sigmoid or ReLU. W is an n×n matrix, and b is a matrix of shape (1, h). s The vectors (h) are the parameters of the fully connected layer and must be learned from training samples. s This represents the size of the hidden layer. R(q) is the vector representing the query q, with dimensions (1, h). s ).

[0120] The above steps transform the data tables and join relationships involved in the SQL query into a series of feature vectors, laying the foundation for using reinforcement learning to optimize the database query execution plan. These feature vectors meticulously reflect the key attributes of the data tables and the join patterns between them, providing the comprehensive information needed for query optimization and enabling the model to make optimization decisions from a more holistic perspective.

[0121] In step S2, based on the column feature vector R(c) and the data table feature vector R(t), a connection tree data structure is designed, which can represent the connection relationship between tables.

[0122] Specifically, the join tree is represented using a Tree-LSTM, where a leaf node can be a table or a column. In the join tree, an internal node corresponds to a join operation and contains four child nodes (t0, c0, t1, c1). Nodes t0 and t1 represent the two join trees or tables to be joined, while nodes c0 and c1 correspond to the two columns to be joined. For example, a join tree T... l It can be formed by two join tables t to be joined. l and t r Composition, t l and t r There is a connection relationship, t l,c and t r,c t l and t r The columns to be joined are the join tree T. l The root node has four child nodes t. l , t r , t l,c and t r,c Similarly, a connection tree T can be formed by a connection tree t. l and table t j Composition, where t j and connection tree T l A table such as t r There is a connection relationship, t j,c and t r,c2 t j and t r The columns to be joined are the four child nodes of the root node of the join tree T. l , t j , t r,c2 and t j,c .

[0123] This method uses Tree-LSTM to represent the feature vectors of the connection tree. In the Tree-LSTM structure, each node j has two important vector representations: h j and m j h j This is the hidden state, which summarizes information about a node and all its child nodes. This state propagates across different levels of the tree, used to capture and store the characteristic information of a node in a specific context. j The memory cell plays a crucial role in long short-term memory networks, maintaining and transmitting long-term dependent information. In Tree-LSTM, this memory cell helps the model maintain long-distance information flow across tree-structured data.

[0124] For a node j in the join tree structure: if node j is a leaf node representing a single table or a leaf node representing a single column, then h j =R(j), m j It will be initialized to a zero vector. For an intermediate node j representing a join operation, it has four child nodes t0, c0, t1, and c1. We process the representation of these four nodes in a Tree-LSTM to obtain h. j and m j By traversing the entire connection tree from bottom to top, the feature vector representation h of the root node is finally generated. j The eigenvector representation of the root node is considered to be the eigenvector R(T) = h of the connection tree. j .

[0125] In step S3, the table's metadata information (indexes, number of rows, number of distinct values, etc.) is encoded to generate feature vectors. The table's partitioning information is encoded, including the partition key (which column the partition is on) and the partitioning method (e.g., range partitioning, hash partitioning, etc.). The histogram statistics of each node are encoded to generate feature vectors.

[0126] Specifically, the encoding method for the table's metadata information is as follows: For a table t with k columns, we construct a vector F(ts) of shape 3×k. In this vector, the first row encodes index information, the second row encodes row count information, and the third row encodes the number of distinct values. For each column in table t, we use a binary bit to indicate whether the column has an index: if an index is created, the bit is set to 1; otherwise, it is set to 0. For the row count and the number of distinct values, since the magnitudes of these features can vary greatly, we use a logarithmic transformation to reduce the magnitude difference: x' = log(1+x), where x represents the number of rows or distinct values ​​in each column, and x' is the result of the logarithmic transformation. x' is then input into the corresponding position in the vector F(ts).

[0127] Similarly, for encoding the partition information, we construct a vector F(p) of size 2×k. For each column in the table, there may be three cases: no partition, hash partition, and range partition, with corresponding codes of (0, 0), (0, 1), and (1, 1), respectively. Combining the two-dimensional vectors of the k columns together yields the vector F(p) of size 2×k.

[0128] To encode the statistical information of the table, assuming each column has n buckets, we construct a vector F(stat) of size 3×n×k. Each bucket corresponds to a vector b of length 3. i =(max i',min i ', count i '), max i ',min i 'Represents the maximum boundary value of the bucket, respectively. i The max can be obtained by normalizing using the following formula (1). i ':

[0129]

[0130] Here, max and min are the maximum and minimum values ​​in the entire dataset, respectively. Similarly, for min... i Normalization will give the min i In this way, all boundary values ​​will be scaled to between 0 and 1. i The number of records in this bucket is counted using the following formula (2). i The result obtained after logarithmic transformation:

[0131] count′ i =log(count) i +1) (2)

[0132] The three-dimensional vector (b1, b2, ... b) of n buckets in a column. n The combination yields a vector h of size 3×n. ci Then, the vector (h) of k columns c1 h c2 ,...h cn The vector F(stat) is obtained by combining the two vectors.

[0133] Through the above steps, we use the following formulas (3) to (7) to calculate the source data information feature vector F(ts), partition information vector F(p), and statistical information vector F(stat) of the table. Then, we adjust the three vectors to the same dimension through a fully connected layer and merge them into a single data distribution feature vector R(D):

[0134] F(ts)′=ReLU(w1×F(ts)+b1) (3)

[0135] F(p)′=ReLU(w2×F(p)+b2) (4)

[0136] F(stat)′=ReLU(w3×F(stat)+b3) (5)

[0137] F(D)=concat(F(ts)′, F(p)′, F(stat)′) (6)

[0138] R(D)=tanh(F(D)) (7)

[0139] Where w1, w2, w3, b1, b2, b3 are parameters, concat() is the aggregation function, and ReLU() and tanh() are activation functions.

[0140] In step S4, a forest feature vector for the connection forest is generated based on the table feature vectors of multiple data tables to be connected, the column feature vector of the target column, and the tree feature vectors of the connected trees. The connection forest consists of several connection trees or data tables to be connected, and the set of data tables to be connected corresponding to each tree is different.

[0141] Specifically, for a query q involving joins of m tables, after n join operations, we have mn unjoined join trees. These join trees can also be single unjoined tables. Since these join trees T are unordered, any two trees can be joined in the next operation. To represent the combination of these unjoined join trees, we use a root node to represent the entire forest, and the child nodes in the forest are all these mn join trees T. i The forest feature vector R(F) connecting the forests is calculated using the following formula (8):

[0142]

[0143] Where tanh() is the activation function, w and b are parameters, and R(T) = ... i ) is a connection tree T i The tree feature vector.

[0144] In step S5, the state feature vector of the current connection state is generated based on the statement feature vector of the query statement, the distribution feature vector of the data distribution information, and the forest feature vector of the connected forest.

[0145] After obtaining the forest feature vector R(F) of the connected forest, the statement feature vector R(q) of the query statement q, and the distribution feature vector R(D) of the data distribution information, we can combine them to obtain the state feature vector R(s) of the current connected state s = R(F)⊕R(q)⊕R(D).

[0146] In step S6, a deep reinforcement learning model is constructed based on the state feature vector of the current connection state. The best N candidate actions are selected and executed according to the Beam Search algorithm to generate N corresponding connection states for the next time step. This process is repeated to gradually generate the complete connection order and connection operators of the data table.

[0147] In the process of selecting connection actions and generating new connection states, we use the Beam Search algorithm to expand the search path. Based on the value predicted by the model, the top N actions with the highest value are generated and selected from all possible candidate connection actions of all current states as the best, where N is the preset beam width. Each candidate action is executed, resulting in the generation of N new states. These newly generated states, based on the executed actions and previous states, provide the basis for the next round of Beam Search.

[0148] like Figure 5 As shown, suppose there are four tables A, B, C, and D that need to be joined, beam size = 2, and the initial state s1 has no joins.

[0149] First round of selection: In the initial state, based on all possible connections (e.g., Each connection can also select from multiple connection operators. The connection operations and connection operators together form the search space of actions. Assume the algorithm evaluates and selects two optimal actions. and Because beam size = 2.

[0150] Second round of choices: Execute action and Afterwards, there will be two new states. In the new state s2a, the remaining possible connections include... in Connect The result is connected to A; similarly, in the new state s2b, the remaining connections may include These actions are evaluated, and beam search is used again to select the top two actions. Executing these two actions will generate two new states, s3a and s3b. Since the beam size remains at 2, we will again select only the two optimal actions from all possible actions generated by these two new states (considering all possible connections originating from these two states). This means that the total number of candidate solutions is always N and will not grow exponentially with the number of layers.

[0151] Each round of model training involves inputting the feature vector of the current connection state into the initial model and obtaining the connection actions and their values ​​from the model output. Here, the current connection state refers to the connection state corresponding to the connection forest formed by multiple determined connected or unconnected connection trees. For example, if the data tables to be joined are A, B, C, D, and E, and some connection actions have been determined in previous model training, such as A connecting to B, and then B connecting to D, then the current connection state can be represented as A connecting to B and B connecting to D, while C and E are not yet connected. In this case, the connections between A and B, and B and D, can be considered as one connection tree in the connection forest, while C and E, since they are not connected to other tables, can each be considered as a separate connection tree. Similarly, for example, if the data tables to be joined are A, B, C, D, and E, and some connection actions have been determined in previous model training, such as A connecting to B, and then C connecting to D, then A and B form one connection tree, C and D form another connection tree, and E is a separate connection tree. Therefore, each connection tree in the connection forest represents a multi-table join relationship or a single unconnected table. The initial model outputs a connection action that selects two connection trees in the forest and determines which connection operator to use to connect the data tables corresponding to these two trees.

[0152] We can obtain the connection actions output by the initial model and their values. The value of the initial model, also known as the Q-value, is used to adjust the parameters of the initial model. In a certain round of model training, suppose the current connection state is E. s The initial model outputs a connection action, and the initial model's value is Q. The Q value can be considered as a function of E. s The function value of 'a' represents the current connection state E. s Next, we determine the initial model value when selecting action a. We obtain the result for each action from the Tree-LSTM and select the TOP K actions with the smallest values ​​using the following formula (9):

[0153]

[0154] In each round of model training, the initial model outputs multiple join actions. After multiple rounds of training, these join actions can construct an optimized join order and join operators for multiple tables to be joined. After completing model training for a specific query statement, we can choose a new query statement to continue model training. Furthermore, the reward value obtained by the initial model from taking a series of join actions is determined based on the cost of executing the query operation. This reward value is negatively correlated with the cost of executing the query. In other words, the reward value that the optimizer can calculate is the negative of the cost of executing the query operation or the reciprocal of the cost.

[0155] After estimating the value, we will calculate the loss between the initial model and the target model based on the set loss function, and adjust the parameters of the initial model accordingly to minimize the difference between the initial model and the target model value. The loss function is expressed by the following formula (10):

[0156] Loss(μ)=(Q(Es,a;p)-y)(10)

[0157] Here, μ represents all the parameters to be updated in the model, and y represents the target model value. Using the mean squared error loss function, we train the valuation neural network, continuously adjusting the weights of the parameters in the network through gradient backpropagation to minimize the error between the network output and the actual value. Furthermore, according to a predetermined time period, these updated weights are copied to the target valuation network to update the weights of the target network. By repeating this training process until the loss value reaches a satisfactory condition, the trained initial model becomes the optimized reinforcement learning model. This reinforcement learning model can then be used to determine the optimal join order and best join operator among multiple tables to be joined when performing query operations in a database.

[0158] Secondly, embodiments of this application also provide a system for optimizing the join order and join operators of a distributed database table. In one embodiment, referring to... Figure 6 , Figure 6 This is a functional module diagram of an embodiment of the system for optimizing the join order and join operators of a distributed database table according to this application. Figure 6 As shown, the optimization system for the join order and join operators of a distributed database includes: an encoding module 1, which encodes the query statement to obtain a statement feature vector. It also encodes the data distribution of the tables to be joined to obtain a distribution feature vector. A join forest generation module 2 generates a join tree and its tree feature vector based on the statement feature vector. It also generates a join forest and its forest feature vector based on the statement feature vector and the tree feature vector. A model building module 3 constructs a deep reinforcement learning model based on Tree-LSTM based on the current state feature vector. It also selects the N optimal join actions according to the Beam Search algorithm, generates N corresponding next-time join states, where N is a positive integer not less than 2, and repeats this process until an execution plan is generated that joins all the tables to be joined. This execution plan includes the join order and join operators. Finally, after importing the execution plan into the distributed database for execution, the execution time is used as a feedback signal to optimize the deep reinforcement learning model.

[0159] The functions of each module in the above-mentioned system for optimizing the order of distributed database table joins and join operators correspond to the steps in the above-mentioned method embodiment for optimizing the order of distributed database table joins and join operators. Their functions and implementation processes will not be described in detail here.

[0160] In some processes described in the embodiments of this application, multiple operations or steps appear in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined. Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0161] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for optimizing join order and join operators of distributed database tables, characterized in that, The method includes: The query statement is encoded to obtain a statement feature vector; Generate a connection tree and its tree feature vectors based on the statement feature vectors; Encode the data distribution of the tables to be joined to obtain the distribution feature vector; Generate a connection forest and its forest feature vectors based on the statement feature vectors and tree feature vectors; Generate the current state feature vector of the current connection state based on the statement feature vector, distribution feature vector, and forest feature vector; Construct a deep reinforcement learning model based on Tree-LSTM based on the current state feature vector; The Beam Search algorithm selects the N optimal join actions to execute, generating N corresponding next-time join states, where N is a positive integer not less than 2. This process is repeated until an execution plan is generated that connects all the tables to be joined. This execution plan includes the join order and join operators. After the execution plan is imported into the distributed database for execution, the execution time is used as a feedback signal to optimize the deep reinforcement learning model. Encoding the query statement to obtain a statement feature vector specifically includes the following steps: The SELECT, WHERE, and JOIN clauses in the query statement are analyzed to extract the data table to be joined, the target column, and the corresponding predicate conditions. The predicate conditions include the attribute selection predicate and the join conditions of the data table to be joined. The column-related information of the target column is encoded to obtain the column feature vector. The column-related information includes the table name, selection predicate, and join condition of the data table to be joined associated with the target column. Summarize the column feature vectors of all target columns to obtain the table feature vector of the data table to be joined; Encode the predicate condition and the connection condition separately to obtain the predicate feature vector and the condition feature vector; The process of generating a connection tree and its tree feature vector based on the statement feature vector specifically includes the following steps: The data structure of the connection tree is designed based on the column feature vector and the table feature vector. This data structure is a Tree-LSTM structure. An internal node in the connection tree corresponds to a connection operation. Each internal node includes four child nodes. Two child nodes represent two targets to be connected, which are either the connection tree or the data table to be connected. The other two child nodes represent the target columns in the two targets to be connected.

2. The method for optimizing the join order and join operators of a distributed database table as described in claim 1, characterized in that, Encoding the data distribution of the tables to be joined to obtain a distribution feature vector specifically includes the following steps: Extract histogram statistics, metadata, partition keys, and partitioning algorithms from the tables to be joined; The histogram statistics, metadata, partition keys, and partitioning algorithms are encoded to obtain the distribution feature vector of the data distribution.

3. The method for optimizing the table join order and join operators in a distributed database as described in claim 1, characterized in that, The connection forest includes multiple targets to be connected, which are connection trees or data tables to be connected. The sets of data tables to be connected corresponding to each connection tree in the connection forest are all different.

4. The method for optimizing the table join order and join operators in a distributed database as described in claim 1, characterized in that, The deep reinforcement learning model is trained using the deep Q-network algorithm.

5. The method for optimizing the table join order and join operator in a distributed database as described in claim 1, characterized in that, The step of selecting the N optimal join actions according to the Beam Search algorithm, generating N corresponding next-time join states, and repeating this process until an execution plan for joining all the data tables to be joined is generated includes the following steps: After taking the preset next-time connection state as the current connection state, the feature vector of the current state is input into the deep reinforcement learning model to obtain candidate connection actions and their values. According to the Beam Search algorithm, the N candidate actions with the highest value are selected as the best connection actions, where N is the preset Beam width; Execute each optimal connection action to obtain N current connection states; After repeatedly using the preset next-time connection state as the current connection state, the N optimal connection actions are selected and executed according to the Beam Search algorithm, generating N corresponding next-time connection states, until an execution plan for connecting all the data tables to be connected is generated.

6. A system for optimizing the join order and join operators of a distributed database table, characterized in that, A method for optimizing the table join order and join operators of a distributed database as described in any one of claims 1-5; the system comprises: The encoding module is used to encode the query statement to obtain the statement feature vector; it is also used to encode the data distribution of the tables to be joined to obtain the distribution feature vector. The connection forest generation module is used to generate connection trees and their tree feature vectors based on statement feature vectors; it is also used to generate connection forests and their forest feature vectors based on statement feature vectors and tree feature vectors. The model building module is used to construct a deep reinforcement learning model based on Tree-LSTM based on the current state feature vector; it is also used to select the N optimal connection actions to execute according to the Beam Search algorithm, generate N corresponding next-time connection states, where N is a positive integer not less than 2, and repeat this process until an execution plan is generated that connects all the data tables to be connected. This execution plan includes the connection order and connection operators; it is also used to import the execution plan into the distributed database for execution, and use the execution time as a feedback signal to optimize the deep reinforcement learning model.

7. The system for optimizing the order of table joins and join operators in a distributed database as described in claim 6, characterized in that, The encoding module is used to analyze the SELECT, WHERE, and JOIN clauses in the query statement to extract the data table to be joined, the target column, and the corresponding predicate conditions. The predicate conditions include the attribute selection predicate and the join conditions of the data table to be joined. It is also used to encode the column-related information of the target column to obtain the column feature vector. The column-related information includes the table name, selection predicate, and join condition of the data table to be joined associated with the target column. It is also used to summarize the column feature vectors of all target columns to obtain the table feature vector of the data table to be joined; It is also used to encode predicate conditions and connection conditions respectively, to obtain predicate feature vectors and condition feature vectors.

8. The system for optimizing the order of table joins and join operators in a distributed database as described in claim 6, characterized in that, The connection forest generation module is used to design the data structure of the connection tree based on the column feature vector and the table feature vector. The data structure is a Tree-LSTM structure. An internal node in the connection tree corresponds to a connection operation. Each internal node includes four child nodes, two of which represent two targets to be connected. The targets to be connected are the connection tree or the data table to be connected. The other two child nodes represent the target columns in the two targets to be connected.

Images