Database query feature vector generation method

Abstract

The invention relates to the technical field of query characterization, and discloses a database query feature vector generation method, which comprises the following steps of: obtaining a connection bitmap and a database mode graph of a database queried by a target query statement; determining a database mode sub-graph related to the target query statement according to the database mode graph; encoding the database mode sub-graph through a preset gating graph neural network to generate a mode item vector; and determining a query feature vector corresponding to the target query statement according to the connection bitmap and the mode item vector. According to the method, by obtaining the database mode graph and the connection bitmap, the structure association information between the table and the column in the database and the predicate condition features of the query can be synthesized, the semantic association represented by the features is enhanced based on the coding capability of the graph neural network, and the expression capability of the query feature vector for query intention and data association is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of query characterization, in particular to a database query feature vector generation method. BACKGROUND

[0002] The existing query-driven cardinality estimation mainly adopts One-Hot encoding to characterize the query, ignoring important schema information such as database table association and column attributes, which is easy to miss data features and cause loss of query semantic information, affecting the accuracy of cardinality estimation. SUMMARY

[0003] The main purpose of the present application is to provide a database query feature vector generation method, which aims to solve the technical problem that the existing query-driven cardinality estimation ignores important schema information such as database table association and column attributes, causing loss of query semantic information.

[0004] To achieve the above purpose, the present application provides a database query feature vector generation method, which comprises:

[0005] Obtain the connection bitmap and the database schema graph of the database queried by the target query statement;

[0006] Determine the database schema subgraph related to the target query statement according to the database schema graph;

[0007] Encode the database schema subgraph through a pre-set gate graph neural network to generate a schema item vector;

[0008] Determine the query feature vector corresponding to the target query statement according to the connection bitmap and the schema item vector.

[0009] Optionally, the step of obtaining the connection bitmap and the database schema graph of the database queried by the target query statement comprises:

[0010] Determine the database queried by the target query statement, and obtain the tables and columns in the database;

[0011] Construct a database schema graph according to the tables and the columns;

[0012] Create an association sample based on the primary-foreign key relationship between the tables, and process the association sample according to the predicate conditions in the target query statement to generate a connection bitmap.

[0013] Optionally, the step of constructing a database schema graph according to the tables and the columns comprises:

[0014] Set each table in the database and the columns contained in each table as an independent node respectively to obtain a node set;

[0015] determine a first type of edge according to the belonging relationship between the column and the table;

[0016] determine a second type of edge and a third type of edge based on the association relationship between the foreign key column and the primary key column between the table and another table;

[0017] obtain an edge set according to the first type of edge, the second type of edge and the third type of edge;

[0018] construct a database schema graph through the node set and the edge set.

[0019] Optionally, the step of encoding the database schema subgraph through the preset gated graph neural network to generate a schema item vector includes:

[0020] assign an initial feature representation to each node in the database schema subgraph, the initial feature representation being determined based on an embedding representation of a schema item and a relevance score;

[0021] iteratively optimize the initial feature representation in the database schema subgraph through a message passing mechanism of the gated graph neural network to obtain a schema item vector of each schema item node.

[0022] Optionally, the gated graph neural network includes a message passing function and a gated recurrent unit.

[0023] The step of iteratively optimizing the initial feature representation in the database schema subgraph through the message passing mechanism of the gated graph neural network to obtain a schema item vector of each schema item node includes:

[0024] determine neighborhood nodes and corresponding edge types of each node in the database schema subgraph;

[0025] process feature representations of the neighborhood nodes through the message passing function based on the edge types to generate neighborhood messages;

[0026] aggregate the neighborhood messages and the initial feature representations of the nodes to obtain intermediate feature representations;

[0027] update and iterate the intermediate feature representations through the gated recurrent unit to obtain node hidden states;

[0028] when the number of iterations is less than a preset fixed propagation step number, take the node hidden state as the initial feature representation of the node at the current iteration, and return to the step of aggregating the neighborhood messages and the initial feature representations of the nodes to obtain intermediate feature representations until the number of iterations is equal to the preset fixed propagation step number;

[0029] The node hidden state of the last iteration is taken as a pattern item vector of each of the pattern items.

[0030] Optionally, the step of determining the database pattern subgraph related to the target query statement according to the database pattern graph comprises:

[0031] The association probability of a query word in the target query statement and each pattern item in the database pattern graph is calculated.

[0032] The relevance score between the target query statement and each of the pattern items is determined according to the association probability.

[0033] The nodes in the database pattern graph that satisfy a preset condition in terms of the relevance score are retained, and the nodes that do not satisfy the preset condition are removed, to obtain a database pattern subgraph.

[0034] In addition, to achieve the above object, the present application further provides a database query feature vector generation device, which comprises:

[0035] A pattern graph acquisition module is configured to acquire a connection bitmap and a database pattern graph of a database queried by a target query statement.

[0036] A pattern subgraph determination module is configured to determine a database pattern subgraph related to the target query statement according to the database pattern graph.

[0037] A pattern item representation module is configured to encode the database pattern subgraph by using a preset gated graph neural network to generate a pattern item vector.

[0038] A query featureization module is configured to determine a query feature vector corresponding to the target query statement according to the connection bitmap and the pattern item vector.

[0039] In addition, to achieve the above object, the present application further provides a database query feature vector generation device, which comprises: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the database query feature vector generation method as described above.

[0040] In addition, to achieve the above object, the present application further provides a storage medium, which is a computer-readable storage medium, and the storage medium stores a computer program, the computer program being executable by a processor to implement the steps of the database query feature vector generation method as described above.

[0041] In addition, to achieve the above object, the present application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the database query feature vector generation method as described above.

[0042] In the present application, a connection bitmap and a database schema graph of a database queried by a target query statement are obtained; a database schema subgraph related to the target query statement is determined according to the database schema graph; a schema item vector is generated by encoding the database schema subgraph through a preset gating graph neural network; and a query feature vector corresponding to the target query statement is determined according to the connection bitmap and the schema item vector. By obtaining the database schema graph and the connection bitmap, the present application can comprehensively integrate the structural association information between tables and columns in the database and the predicate condition features of the query, and can enhance the semantic association of the feature representation based on the encoding capability of the graph neural network, thereby effectively improving the expression capability of the query feature vector for the query intent and the data association. BRIEF DESCRIPTION OF DRAWINGS

[0043] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

[0045] Figure 1 Flowchart of the first embodiment of the database query feature vector generation method of the present application;

[0046] Figure 2 Principle diagram of hard pruning and soft pruning;

[0047] Figure 3 Flowchart of the query feature enhancement method of the present application;

[0048] Figure 4 Framework diagram of the GNMSCN model of the present application;

[0049] Figure 5 Flowchart of the second embodiment of the database query feature vector generation method of the present application;

[0050] Figure 6 Database schema graph and schema subgraph corresponding to the query statement of the present application;

[0051] Figure 7 Example diagram of an m-bit bitmap;

[0052] Figure 8 for the sample bitmap corresponding to the query statement and the connection bitmap;

[0053] Figure 9 a flowchart of the third embodiment of the database query feature vector generation method of the present application;

[0054] Figure 10 a module structure diagram of the database query feature vector generation device of the embodiment of the present application;

[0055] Figure 11 a device structure diagram of the hardware running environment involved in the database query feature vector generation method in the embodiment of the present application.

[0056] The object implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0057] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application, and are not used to limit the present application.

[0058] In order to better understand the technical solutions of the present application, the following will be described in detail in combination with the drawings and specific embodiments of the specification.

[0059] The query-driven cardinality estimation method converts the SQL query statement into a query feature vector, and then maps the vector to the result of cardinality estimation through a cardinality estimation model. Among them, the query feature stage is crucial, which establishes the mapping relationship between the query statement and the cardinality estimation result. The existing query-driven cardinality estimation method mainly uses One-Hot encoding to process the query features. In the face of complex query scenarios in large-scale databases, One-Hot encoding is easy to cause the dimension of features to increase dramatically, causing dimension disaster, not only increasing the calculation overhead, but also easily leading to data sparseness; at the same time, this encoding method ignores the potential complex semantic information and association between tables and columns in the database. This encoding method ignores important schema information such as association between database tables and column attributes, which not only easily misses data features, causes loss of query semantic information, but also easily causes encoding redundancy and dimension disaster, affecting the accuracy of cardinality estimation.

[0060] Therefore, the application provides a new query feature enhancement technology based on a graph neural network, and a GGNN (Gated Graph Neural Networks) is introduced to encode a database schema graph structure. The GGNN aggregates neighborhood information and edge features of nodes to mine schema information such as types and primary-foreign key relationships in the database schema, and finally obtains a schema item representation in a node-level task. In addition, a connection bitmap is integrated into query statement encoding to effectively capture data distribution characteristics and the association between primary and foreign keys.

[0061] It should be noted that the execution subject of the embodiment can be a computing service device with database processing, feature fusion, and program running functions, such as a computer or the like, or an electronic device capable of implementing the above functions. The following describes the embodiment and each of the following embodiments by taking a query feature enhancement device as an example.

[0062] Based on this, the application embodiment provides a database query feature vector generation method, which is described below with reference to Figure 1 , Figure 1 FIG. 1 is a flowchart of a first embodiment of a database query feature vector generation method according to the application.

[0063] In the embodiment, the database query feature vector generation method comprises the following steps.

[0064] In step S10, a connection bitmap and a database schema graph of a database queried by a target query statement are obtained.

[0065] It should be noted that the database schema graph is a graph structure taking tables and columns in a database as nodes, taking the ownership relationship (such as a table containing a column) and the foreign key association relationship (such as the correspondence relationship between the foreign key column of one table and the primary key column of another table) between tables as edges, and is used to intuitively present the logical structure and semantic association of the database. The connection bitmap is a binary vector generated based on a primary-foreign key association sample, wherein each bit corresponds to a sample tuple, and 1 or 0 is used to identify whether the sample tuple satisfies the predicate condition in the target query statement, and is used to represent the data distribution characteristics.

[0066] In an example, given a database D=(S,Σ), the database schema S includes a table set T={t1,…,t |T|}, a column set C={C t} t∈T and a foreign key-primary key column pair set F. Wherein |T| represents the number of tables in the database, |C t | represents the number of columns in the table t, represents the set of columns in the database table t, and each foreign key-primary key column pair (c f ,c p)∈T represents a foreign key c from a table f To the primary key c in another table p The relationship is as follows. In the database, some symbolic conventions are defined as follows:

[0067] Table 1 Symbol Conventions

[0068] Symbol Definition D Database S Database schema Σ Database statistics T Set of database tables C Set of database columns F Set of foreign-key-primary-key column pairs V Database schema item (all tables, column sets in the database) Q Workload q i ]]> i-th query statement in the workload Y Query feature vector

[0069] By using the notation conventions in the table, the query feature representation problem is defined as f: Q×V×Σ→Y, that is, with workload Q={q1,q2,…,q n}、Pattern term V=T∪{C t} t∈T The database statistics Σ are used as inputs to the query featureization stage. Different query feature encoding methods are used to obtain the mapping of Q, V, and Σ to the query feature vector Y. Finally, Y is used as the input to the cardinality estimation model.

[0070] Step S20: Determine the database schema subgraph related to the target query statement based on the database schema diagram.

[0071] It should be noted that a database schema subgraph refers to a portion of the database schema graph that is directly semantically related to the target query statement. It contains nodes corresponding to the tables and columns involved in the target query, as well as the edges connecting these nodes. It is used to focus on the database structure information related to the target query and reduce the interference of redundant data on subsequent feature encoding.

[0072] Understandably, determining the database schema subgraph can be done by taking query q i As a selection criterion, a subgraph search is performed on the database schema graph. Based on q... i The tables and columns involved are soft-pruned in the database schema diagram, retaining the table and column nodes that are directly related to the query and removing redundant nodes that are not directly semantically related to the query, thus generating a database schema subgraph.

[0073] Understandably, determining the database schema subgraph can also involve parsing the target query statement, identifying the directly involved tables and columns, using these directly involved schema items as initial nodes, calculating the association path length between the initial nodes and other nodes in the schema graph, determining nodes with a path length ≤ 2 as related nodes, i.e., directly involved nodes and their adjacent foreign key association nodes, retaining these related nodes and their corresponding edges to form the database schema subgraph.

[0074] It should be understood that determining the database schema subgraph can also be syntax parsing of the target query statement, generating a set of schema items involved in the query, presetting a basic weight for each node in the schema graph, diffusing the basic weight to adjacent nodes through a graph propagation algorithm, iteratively updating the weight value of each node, and selecting the nodes with the top-ranked weight values and the connected edges to constitute the database schema subgraph.

[0075] Further, in order to accurately extract the database schema subgraph by calculating the association probability of the query word and the schema item, determining the correlation score, and screening the nodes, and ensure that the subgraph focuses on the query-related schema items. The step S20 can include:

[0076] calculating the association probability of the query word in the target query statement and each schema item in the database schema graph; determining the correlation score between the target query statement and each schema item according to the association probability; retaining the nodes in the database schema graph that satisfy the preset condition, and eliminating the nodes that do not satisfy the preset condition, to obtain a database schema subgraph.

[0077] It should be noted that the association probability refers to the possibility of the existence of the association between the query word in the target query statement and each schema item (table, column) in the database schema graph, which is obtained by normalizing the similarity score of the query word and the schema item; the correlation score is a quantitative index determined based on the association probability, which is used to represent the close degree of association between the target query statement and each schema item.

[0078] In an example, referring to Figure 2 , Figure 2 is a schematic diagram of the principle of hard pruning and soft pruning. Soft pruning is a widely used technique in machine learning and deep learning models, mainly used to improve the generalization ability of the model and reduce the computational complexity. As shown in the figure, hard pruning (Hard Pruning) compresses the model by permanently removing redundant filters. Although this can significantly reduce the computational complexity, the pruned filters will completely lose the ability to update, which may lose important information and cause the model performance to decline. In contrast, soft pruning temporarily deactivates unimportant filters, but retains their recovery ability and can be reactivated when needed. This way, the model capacity is maintained, and the model's ability to adapt to different data distributions is improved.

[0079] Since each query is associated with different nodes in the schema graph, in order to link the query word x i to the schema item in the graph, first introduce the existing entity linking method, through s link (v,x i ) represents the similarity score between the query statement word x i and the schema item v. If denotes that the query is not linked to any schema item. To enrich the context representation and further define the link score p i inject the semantic information of its associated schema item, enrich the context representation, and further define the link score p link (v|x i ), which represents the query word x i and the probability of the schema item v, which is shown as follows:

[0080]

[0081] where V τ is all schema items of type τ, including tables, string columns, numerical columns, etc.

[0082] Meanwhile, based on the input query statement, soft pruning is performed using p link (·) to reduce the number of parameters, avoid the interference of invalid information, and improve the generalization ability and efficiency of the model. Finally, a relevance score ρ v of the schema item v is defined based on the query statement, which is shown as follows:

[0083] ρ v = max i p link (v|x i )

[0084] Each query involves different database schema items. If static global modeling is used, the model needs to process a large amount of redundant information. When the scale of the schema items is large, not only will it lead to low computational efficiency, but also will make the semantic representation fuzzy, and even cause dimension disaster due to the explosion of the number of parameters, and increase the risk of model overfitting. Therefore, the relevance of the schema items should change dynamically with the query, and a database schema subgraph containing only relevant schema items should be generated according to the current query.

[0085] In step S30, the database schema subgraph is encoded by a preset gated graph neural network to generate a schema item vector.

[0086] It should be noted that the information transmission intensity is controlled by the gated graph neural network, which can effectively process graph structure data and capture the associated semantics between nodes. The schema item vector is a vector obtained after encoding, which is used to represent each schema item (table or column) in the database schema subgraph, and contains the features of the schema item itself and the associated information with other relevant schema items.

[0087] It can be understood that the mode item vector generated by the gated graph neural network encoding can effectively capture the deep semantic association between the mode items in the database mode subgraph, such as the ownership of tables and columns, the primary-foreign key dependency between tables, and the generated mode item vector has both the attributes of the mode item itself and the associated context information, providing a semantic-rich basic feature for the subsequent fusion with the connection bitmap, thereby enhancing the expression ability of the query feature vector to the query intent and data structure, and providing higher quality input for the subsequent tasks based on the vector (such as cardinality estimation).

[0088] Step S40, determining the query feature vector corresponding to the target query statement according to the connection bitmap and the mode item vector.

[0089] It can be understood that determining the query feature vector according to the connection bitmap and the mode item vector can be feature extraction on the connection bitmap, converting it into a bitmap feature vector with fixed dimensions to retain the distribution information of the sample satisfying the predicate condition; then performing dimension alignment processing on the mode item vector and the bitmap feature vector to ensure that they have the same feature dimension for fusion; then using a pre-set feature fusion strategy (such as feature splicing, element-level product, or weighted sum), the distribution features of the bitmap feature vector and the semantic association features of the mode item vector are fused, so that the fused vector contains both data structure association information and query condition satisfaction information; finally, the fused vector is taken as the query feature vector corresponding to the target query statement, providing comprehensive and accurate feature input for subsequent tasks based on the vector (such as cardinality estimation).

[0090] In an example, referring to Figure 3 and Figure 4 , Figure 3 is a flowchart of the query feature enhancement method of the present application, Figure 4 is a framework diagram of the GNMSCN model of the present application. The query feature enhancement method based on GGNN optimizes the query feature representation by fusing database schema, statistical features, etc. The optimized query feature vector is input into the cardinality estimation model, and the cardinality estimation result is output. The core process mainly includes four stages.

[0091] (1) Constructing a database schema graph. The tables and columns in the database are abstracted as nodes, and the table-column relationship and the inter-table foreign key relationship are abstracted as edges to construct a graph corresponding to the database schema, providing structured input for the subsequent GGNN and providing the structure and semantic information of the database.

[0092] (2) Constructing a query-related database schema subgraph. The query q i is selected as the condition, and the subgraph search is performed on the database schema graph. According to q iThe tables and columns involved, soft pruning the database schema graph, retaining the table, column nodes directly associated with the query, and eliminating redundant nodes not directly semantically associated with the query.

[0093] (3) Database schema subgraph embedding representation based on GGNN. Through the message passing mechanism of GGNN, multiple rounds of neighborhood information aggregation are performed in the pruned subgraph, and finally a low-dimensional vector representation of the schema item containing the association information between tables and columns and rich semantics is generated, replacing the original One-Hot vector in MSCN.

[0094] (4) Integrated connection bitmap encoding. The connection bitmap is encoded into the query feature vector as domain knowledge, ultimately improving the accuracy of cardinality estimation.

[0095] GNMSCN introduces GGNN to encode the database schema graph in the framework of MSCN, obtains the representation of the schema item node in the node-level task, and then replaces the original One-Hot vector of the table and column in MSCN with the obtained model item representation vector. In addition, the connection bitmap replaces the original sample bitmap in the query feature encoding, thereby capturing the data distribution information in the database and the association between the primary and foreign keys. As shown in the figure, the architecture faces the query cardinality estimation task, first inputs the database schema into the gated graph neural network to generate the table and column schema item vector; at the same time, the target query statement is parsed into table, connection relationship, predicate (including column, operator, and value) components. Among them, the table component directly calls the table feature vector output by the GGNN, the connection component constructs the connection feature vector based on the association semantics of the schema subgraph, and in the predicate component, the column reuses the column vector output by the GGNN, the operator is converted into a One-Hot vector, and the value is normalized. Subsequently, the features of the table, connection, and predicate are respectively transformed in dimension and features are extracted through multiple layer perceptrons (MLP), and then concatenated and input into the final MLP fusion, and finally the cardinality estimation value of the query result is output.

[0096] In this embodiment, by obtaining the database schema graph and the connection bitmap, the structural association information between tables and columns in the database and the predicate condition features of the query can be integrated, and based on the coding ability of the graph neural network, the semantic association of the feature representation is enhanced, effectively improving the expression ability of the query feature vector for the query intent and data association.

[0097] Reference Figure 5 , Figure 5 The flowchart of the second embodiment of the database query feature vector generation method of the present application is shown in the figure, based on the first embodiment described above, the second embodiment of the database query feature vector generation method of the present application is proposed.

[0098] In the second embodiment, the step S10 comprises:

[0099] Step S101, determine the database queried by the target query statement, and obtain the tables and columns in the database.

[0100] It can be understood that the target query statement is an instruction initiated by a user for retrieving data, such as an SQL statement. When determining the database queried by the target query statement, the database pointed to by the query can be located by parsing the syntax structure of the query statement (such as the USE database name statement in SQL, the prefix of the table name in the FROM clause, and the database table name) in combination with the context configuration of the query execution, such as the default database. When obtaining the tables in the database, all table names under the database are filtered out by querying the data system table of the database. When obtaining the columns in the table, for each table, the metadata system table is queried to filter out the column name and column attribute information of the corresponding table, thereby obtaining the complete set of tables and columns in the database.

[0101] Step S102, constructing a database schema graph according to the tables and the columns.

[0102] Further, in order to make the database schema graph more comprehensively reflect the internal structure of the data. The step S102 can include:

[0103] Each table in the database and each column contained in each table is set as an independent node to obtain a node set; a first type of edge is determined according to the ownership relationship between the columns and the tables; a second type of edge and a third type of edge are determined based on the association relationship between the foreign key column and the primary key column of the tables and another table; an edge set is obtained according to the first type of edge, the second type of edge and the third type of edge; and a database schema graph is constructed through the node set and the edge set.

[0104] It should be noted that the nodes in the database schema graph represent the tables and columns in the database, and the edges represent the association relationship between the tables and the columns and between the tables; the node set is a set composed of all table nodes and column nodes in the database, and each table and each column in the table is an independent node; the edge set is a set composed of edges corresponding to the ownership relationship between the tables and the columns and the primary-foreign key association relationship between the tables, wherein the first type of edge corresponds to the ownership relationship between the columns and the tables, the second type of edge corresponds to the positive association of the foreign key column of one table pointing to the primary key column of another table, and the third type of edge corresponds to the reverse association of the primary key column pointing to the foreign key column.

[0105] It can be understood that when constructing the database schema graph, first, each table (such as a user table, an order table) in the database and each column included in each table (such as a user ID column of the user table, an order number column of the order table) are respectively set as independent nodes to form a node set; second, a first type of edge is determined according to the relationship that a column belongs to a table (such as the user ID column belonging to the user table); third, based on the primary-foreign key association between tables (such as the user ID foreign key column of the order table associating the user ID primary key column of the user table), a second type of edge from the foreign key column to the primary key column and a third type of edge from the primary key column to the foreign key column are determined, and three types of edges are combined to form an edge set; finally, a complete database schema graph is constructed through the combination of the node set and the edge set.

[0106] In an example, referring to Figure 6 , Figure 6 is a schema subgraph corresponding to the database schema graph and the query statement of the present application. Given a database schema S = <T, C, F>, it is first necessary to convert it into a schema graph G = {V, E}, where V represents the nodes of the graph, that is, the database schema item set, and E represents the edges of the graph. The schema subgraph corresponding to the database schema and the query q is shown in the figure.

[0107] Taking the query q as an example, the query involves the title and movie_info_idx tables, and the filtering conditions include “title.kind_id = 7” and “movie_info_idx.info_type_id > 99”. The two tables are connected through title.id = movie_info_idx.movie_id. When constructing the database schema graph, the nodes cover all tables and columns such as the id and kind_id of the title, the movie_id and info_type_id of the movie_info_idx, etc. The edges include the first type of edge, the second type of edge, and the third type of edge. The first type of edge is the table-column attribution, such as the connection of the id and kind_id of the title. The second type of edge is the foreign key association, such as the title.id pointing to the movie_info_idx.movie_id. The schema subgraph focuses on the query-related elements: retaining the title table and its id and kind_id columns, the movie_info_idx table and its movie_id and info_type_id columns, and their attribution edges and foreign key association edges, and eliminating irrelevant column nodes such as episode_id and episode_nr, etc. Finally, a simplified subgraph containing only the tables, columns, and associations involved in the query is formed, which intuitively presents the core of the query and the database schema.

[0108] To enable the GGNN to more effectively capture different ways of interrelation between tables and columns, three types of edges can be distinguished and added to different edge sets: for each column c in a table t t , an edge (c t(t, c t ) to edge set E1 with type e1. In the figure, this kind of edge corresponds to solid line; for each primary foreign key column pair add edge (t1, t2) to edge set E2 with type e2. In the figure, this kind of edge corresponds to dashed line; correspondingly, add edge (t2, t1) to edge set E3 with type e3. In the figure, this kind of edge corresponds to dashed line.

[0109] By distinguishing different types of association between tables and tables, columns and columns, and columns and tables, GGNN can differentiate the aggregation of neighborhood information in the message passing process, thereby more accurately capturing complex semantic dependencies.

[0110] In step S103, an association sample is created based on the primary-foreign key relationship between the tables, and the association sample is processed according to the predicate condition in the target query statement to generate a join bitmap.

[0111] It should be noted that the association sample is a sample set formed by legally pairing the tuples of related tables according to the primary-foreign key association relationship between the database tables, and each pair of matched tuples constitutes an association sample, such as the tuple of the title table associated with the movie_id of the movie_info_idx table through id.

[0112] In an example, referring to Figure 7 and Figure 8 , Figure 7 is an example of an m-bit bitmap, Figure 8 is a sample bitmap and a join bitmap corresponding to the query statement. In the research of cardinality estimation, the statistical information of data distribution (such as histogram, bitmap, etc.) is usually encoded as domain knowledge input into the model. For each predicate in the query, an m-bit bitmap can be used to reflect the satisfaction of the tuples in the table, where m is the total number of tuples in the table. Figure 7 An example of an m-bit bitmap corresponding to a database table T is shown, which is composed of m binary bits, each bit corresponding to a tuple, and 1 or 0 indicating whether the tuple satisfies the predicate condition in the query statement, 1 means satisfaction, and 0 means non-satisfaction.

[0113] Since the cost of maintaining a vector for all tuples is high, most cardinality estimation models use sample bitmaps as input. Sample bitmaps randomly sample sample tuples from each table, and by recording whether the sample tuples satisfy the predicate condition, they can intuitively reflect the percentage of tuples that satisfy the condition and effectively describe the data distribution of the underlying table. For example Figure 8The left side shows a sample bitmap example corresponding to the query statement. For the two tables with primary-foreign key association in the query statement, sample bitmaps are independently generated for the two tables, and the filters of different tables are only applied to the corresponding independent samples. If the query predicate does not filter the table, the sample bitmap of the table is set to 1.

[0114] However, there is a key problem with the sample bitmap: the selected join key values in the sample extracted from one table may not exist in the sample of another table. If the join operation is performed on the independent samples of the two tables, no matching data record can be found, and the sample quality will decrease again. To solve this problem, a join bitmap is used for optimization. As shown on the right side, Figure 8 The right side shows that the associated sample is first created according to the primary key and the foreign key, that is, each primary key corresponds to a bitmap containing foreign key sample information. On this basis, the corresponding query predicate is applied to the associated sample to generate the corresponding join bitmap, thereby avoiding the problem of sample invalidation during the join operation.

[0115] In this embodiment, the database schema graph and the join bitmap are obtained through the tables and columns in the database, which ensures that the database schema graph can accurately reflect the data structure relationship, and the join bitmap can accurately represent the satisfaction of the sample to the query condition, thereby providing high-quality input for subsequent subgraph determination and feature coding.

[0116] Referring to Figure 9 , Figure 9 The flowchart of the third embodiment of the database query feature vector generation method of the present application is shown, and the third embodiment of the database query feature vector generation method of the present application is proposed based on the second embodiment.

[0117] In the third embodiment, the step S30 comprises:

[0118] Step S301: An initial feature representation is assigned to each node in the database schema subgraph, and the initial feature representation is determined based on the embedding representation of the schema item and the correlation score.

[0119] It should be noted that the initial feature representation is an initial vector assigned to each node (table or column) in the database schema subgraph, which combines the embedding representation of the schema item (the semantic vector of the table / column name generated by the pre-trained model, which carries general structure semantics, such as the type identifier attribute of kind_id) and the correlation score (the association probability of the target query and the schema item, which quantifies the attention degree of the query to the node, such as the high score of kind_id obtained by the query predicate kind_id = 7), so that the initial vector contains both the inherent semantics of the schema item and the query association information.

[0120] It can be understood that the initial feature representation can be converted into a low-dimensional semantic vector by using a pre-trained word embedding model, and the general semantic association of the name is captured. The association probability of the query word and the pattern item is reused to represent the association degree of the query to the node.

[0121] In step S302, the initial feature representation is iteratively optimized in the database schema subgraph through a message passing mechanism of a gated graph neural network, to obtain a pattern item vector of each pattern item node.

[0122] It should be noted that the message passing mechanism refers to the process of exchanging feature information by nodes according to adjacency relationships. The pattern item vector is the final feature vector that contains the node's own attributes, neighborhood association and query context after iterative optimization by the GGNN, which not only retains the inherent semantics of the pattern item, but also captures the structural information of the table-column ownership and the primary-foreign key association in the subgraph.

[0123] Specifically, first, adjacency relationships are constructed according to the edge types of the database schema subgraph, and the message passing weights of different edges are defined, such as the ownership edge weight being higher than the weak association edge. In each iteration, the node first aggregates the initial iteration features of the neighborhood nodes to generate neighborhood messages through an aggregation function; then the fusion proportion of the current feature and the neighborhood message is updated through the gating logic of the GRU, the effective information is filtered through the gate, and the hidden state is updated; multiple iterations are repeated, so that the node feature gradually absorbs the predicate information of the columns in the table and the association information of the primary-foreign key between tables, and finally outputs a stable pattern item vector.

[0124] Further, in order to make the information aggregation more targeted and avoid the problems of over-aggregation or under-aggregation, the gated graph neural network includes a message passing function and a gated recurrent unit; and the step S302 can include:

[0125] The neighborhood nodes and corresponding edge types of each node in the database schema subgraph are determined; based on the edge types, the feature representation of the neighborhood nodes is processed through the message passing function to generate neighborhood messages; the neighborhood messages and the initial feature representation of each node are aggregated to obtain an intermediate feature representation; the intermediate feature representation is updated and iterated through the gated recurrent unit to obtain a node hidden state; when the number of iterations is less than a preset fixed propagation step, the node hidden state is taken as the initial feature representation of the node at the current iteration, and the step of aggregating the neighborhood messages and the initial feature representation of each node to obtain an intermediate feature representation is returned until the number of iterations is equal to the preset fixed propagation step; and the node hidden state of the last iteration is taken as the pattern item vector of each pattern item node.

[0126] It should be noted that the neighborhood node refers to the node directly connected to the current node through the edge, such as the neighborhood of the table node being the column node contained therein. The edge type includes table-column ownership edge, primary-foreign key association edge and the like, and is used to distinguish the association properties between nodes. The message passing function is a feature processing function defined based on the edge type, and is used to convert the neighborhood node features into transmissible neighborhood messages. The intermediate feature is represented as a transition vector after aggregation of the node itself feature and the neighborhood message. The fixed propagation step number is a preset iteration number, which ensures that the information is sufficiently transmitted.

[0127] Specifically, first, the database schema subgraph is traversed, and the neighborhood nodes of each node (such as the neighborhood of the title table being the id, kind_id column) and the corresponding edge type (such as the ownership edge) are determined; based on the edge type, the matched message passing function (such as the high weight function for the ownership edge) is called to process the initial feature representation of the neighborhood node (such as linear transformation, activation function), and the neighborhood message is generated; the neighborhood message and the initial feature representation of the current node are combined through the aggregation function to obtain the intermediate feature representation; the intermediate feature representation is input into the GRU, and the valid information is filtered through the update gate and the reset gate, such as retaining the predicate association information of the kind_id column, and updating the node hidden state; if the iteration number does not reach the fixed propagation step number, the current hidden state is taken as the new initial feature representation of the node, and the neighborhood message generation, aggregation and GRU update steps are repeated; when the iteration number is equal to the fixed propagation step number, the iteration is stopped, and the last updated node hidden state is taken as the schema item vector of the schema item node.

[0128] In an example, in order to capture the local relevance of the schema item while preserving the global structure information of the database schema, an initial feature representation h (0) v ·ρ v is assigned to each node v in the graph G, where r v is the embedding representation of the schema item. On this basis, the GGNN is introduced to learn the feature representation of the schema item node in the schema graph.

[0129] For a predefined fixed propagation step number L, the hidden state of the node v is updated at each step through two processes of aggregating neighbor information and updating the GRU (Gate Recurrent Unit) state. Specifically, first, the information is transmitted between nodes and the features are updated according to the in-edge, out-edge and edge type of the node in the schema graph, wherein the parameters of the information aggregation are determined by the edge type and the direction, so that each node recalculates its own state based on the representation of the neighbor node in the previous step. As shown below, the neighbor feature vector is obtained after recalculating the representation of the node v

[0130] ​

[0131] where E type represents the set of edges of type e type W type and b type are learnable parameters related to edge types.

[0132] Then, the hidden state of each node is iteratively optimized by a standard GRU, which combines the aggregated information of neighbor nodes with the previous state of the node itself. Finally, as shown in equation (3.4), the vector representation of the schema item node is obtained

[0133]

[0134] After L-step propagation, the final hidden state of each schema item node v is obtained The vector encodes the structural context information of the node in the schema graph, such as column constraints of the table it belongs to, inter-table relationships of foreign key association, etc. Finally, the final representation vector of the schema item node related to the query statement can replace the original One-Hot vector in MSCN.

[0135] In this embodiment, the initial feature representation is determined based on the embedding representation of the schema item and the correlation score, and the message passing mechanism of the gated graph neural network is used for iterative optimization to generate the schema item vector, so that the initial feature can not only retain the basic information of the schema item itself, but also integrate its relevance with the query; and the iterative optimization process can aggregate the information of the neighbor nodes, so that the schema item vector not only contains its own features, but also reflects the associated semantics with other schema items.

[0136] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the database query feature vector generation method of the present application. More forms of simple transformation based on this technical concept are within the protection scope of the present application.

[0137] The present application also provides a database query feature vector generation device, which is described in detail with reference to Figure 10 The database query feature vector generation device comprises:

[0138] A schema graph acquisition module 10 is configured to acquire a connection bitmap and a database schema graph of a database queried by a target query statement.

[0139] A schema subgraph determination module 20 is configured to determine a database schema subgraph related to the target query statement according to the database schema graph.

[0140] A schema item representation module 30 is configured to encode the database schema subgraph by a preset gated graph neural network to generate a schema item vector.

[0141] The query featureization module 40 is configured to determine a query feature vector corresponding to the target query statement according to the connection bitmap and the pattern item vector.

[0142] The database query feature vector generation apparatus provided by the present application adopts the database query feature vector generation method in the above embodiments, and can solve the technical problem that the existing query-driven cardinality estimation ignores important schema information such as inter-table association and column attributes, resulting in loss of query semantic information. Compared with the prior art, the database query feature vector generation apparatus provided by the present application has the same beneficial effects as the database query feature vector generation method provided by the above embodiments, and other technical features in the database query feature vector generation apparatus are the same as the features disclosed in the above embodiments, which will not be repeated here.

[0143] The present application provides a database query feature vector generation device, which comprises at least one processor and a memory connected with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the database query feature vector generation method in the above embodiment one.

[0144] Reference will now be made to the drawings, and specific language will be used to describe the same. Figure 11 The database query feature vector generation device shown in the drawing is only an example, and should not impose any limitation on the functions and use range of the embodiments of the present application. Figure 11 The database query feature vector generation device shown in the drawing is only an example, and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0145] As Figure 11As shown, the database query feature vector generation device can include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a read-only memory 1002 or loaded from a storage device 1003 into a random access memory 1004. In the random access memory 1004, various programs and data required for the operation of the database query feature vector generation device are also stored. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other by a bus 1005. An input / output interface 1006 is also connected to the bus. Generally, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the database query feature vector generation device to communicate with other devices wirelessly or by wire to exchange data. Although the database query feature vector generation device with various systems is shown in the figure, it should be understood that all the systems shown are not required to be implemented or possessed. More or fewer systems can be alternatively implemented or possessed.

[0146] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by a communication device, or installed from the storage device 1003, or installed from the read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0147] The database query feature vector generation device provided by the present application adopts the database query feature vector generation method in the above-mentioned embodiments, and can solve the technical problem that the existing query-driven cardinality estimation ignores important schema information such as inter-table association and column attributes, resulting in loss of query semantic information. Compared with the prior art, the database query feature vector generation device provided by the present application has the same beneficial effects as the database query feature vector generation method provided by the above-mentioned embodiments, and other technical features in the database query feature vector generation device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0148] It should be understood that various parts of the present application can be realized by hardware, software, firmware or a combination thereof. In the above description of embodiments, specific features, structures, materials or characteristics can be combined in any appropriate manner in any one or more embodiments or examples.

[0149] The above description is merely illustrative of the application and is not intended to limit the scope of the application. Any variations and modifications that can be made by any person skilled in the art within the spirit and scope of the application are intended to be encompassed by the application. Therefore, the scope of the application should be determined by the appended claims.

[0150] The present application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e., a computer program) for performing the database query feature vector generation method in the above-described embodiments.

[0151] The computer readable storage medium provided by the present application may, for example, be a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, system or device, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system or device. The program code contained on the computer readable storage medium can be transmitted by any appropriate medium, including but not limited to electrical wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0152] The above computer readable storage medium can be contained in the database query feature vector generation device; or can exist separately and not be assembled into the database query feature vector generation device.

[0153] The computer readable storage medium described above bears one or more programs, when the one or more programs are executed by the database query feature vector generation device, the database query feature vector generation device executes the database query feature vector generation method described above.

[0154] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0155] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in a different order than that noted in the figures. For example, two blocks noted in succession can in fact be executed substantially concurrently or in the opposite order, depending on the functionality involved. It will also be noted that each block in the block diagrams and / or flow diagrams, and combinations of blocks in the block diagrams and / or flow diagrams, can be implemented by dedicated hardware-based systems that carry out specified functions or operations, or combinations of dedicated hardware and computer instructions.

[0156] The modules involved in the embodiments of the present application can be implemented in software or in hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0157] The readable storage medium provided by the application is a computer readable storage medium, and the computer readable storage medium stores computer readable program instructions (namely, a computer program) for executing the database query feature vector generation method described above, and can solve the technical problem that the existing query-driven cardinality estimation ignores important mode information such as inter-table association and column attribute of a database, and causes loss of query semantic information. Compared with the prior art, the computer readable storage medium provided by the application has the same beneficial effects as the database query feature vector generation method provided by the above-described embodiments, and will not be described here.

[0158] The application further provides a computer program product comprising a computer program, which, when executed by a processor, implements the steps of the database query feature vector generation method as described above.

[0159] The computer program product provided by the application can solve the technical problem that the existing query-driven cardinality estimation ignores important mode information such as inter-table association and column attribute of a database, and causes loss of query semantic information. Compared with the prior art, the computer program product provided by the application has the same beneficial effects as the database query feature vector generation method provided by the above-described embodiments, and will not be described here.

[0160] The above only describes some embodiments of the application, and does not limit the scope of the application, and any equivalent structural transformation made by using the content of the specification and drawings, or direct / indirect application in other related technical fields within the technical concept of the application is included in the protection scope of the application.

Claims

1. A database query feature vector generation method, characterized by, The method comprises: obtaining a connection bitmap and a database schema graph of a database queried by a target query statement; determining a database schema subgraph related to the target query statement according to the database schema graph; encoding the database schema subgraph through a preset gated graph neural network to generate a schema item vector; determining a query feature vector corresponding to the target query statement according to the connection bitmap and the schema item vector.

2. The database query feature vector generation method of claim 1, wherein, The step of obtaining the connection bitmap and the database schema graph of the database queried by the target query statement comprises: determining a database queried by a target query statement, and obtaining tables and columns in the database; constructing a database schema graph according to the tables and the columns; creating an association sample based on the primary-foreign key relationship between the tables, and processing the association sample according to the predicate condition in the target query statement to generate a connection bitmap.

3. The database query feature vector generation method of claim 2, wherein, The step of constructing a database schema graph according to the tables and the columns comprises: setting each table in the database and each column contained in each table as an independent node respectively to obtain a node set; determining a first type of edge according to the ownership relationship between the columns and the tables; determining a second type of edge and a third type of edge based on the association relationship between the foreign key column and the primary key column of another table; obtaining an edge set according to the first type of edge, the second type of edge and the third type of edge; constructing a database schema graph through the node set and the edge set.

4. The database query feature vector generation method of claim 1, wherein, The step of encoding the database schema subgraph through a preset gated graph neural network to generate a schema item vector comprises: assigning an initial feature representation to each node in the database schema subgraph, the initial feature representation being determined based on the embedding representation and the correlation score of the schema item; iteratively optimizing the initial feature representation in the database schema subgraph through the message passing mechanism of the gated graph neural network to obtain a schema item vector of each schema item node.

5. The database query feature vector generation method of claim 4, wherein, The gated graph neural network comprises a message passing function and a gated recurrent unit. The step of iteratively optimizing the initial feature representation in the database schema subgraph through the message passing mechanism of the gated graph neural network to obtain a schema item vector of each schema item node comprises: determining the neighborhood nodes and the corresponding edge types of each node in the database schema subgraph; processing the feature representation of the neighborhood nodes through the message passing function based on the edge types to generate neighborhood messages; aggregating the neighborhood messages with the initial feature representation of each node to obtain an intermediate feature representation; updating and iterating the intermediate feature representation through the gated recurrent unit to obtain a node hidden state; when the number of iterations is less than a preset fixed propagation step number, taking the node hidden state as the initial feature representation of the node at the current iteration, and returning to the step of aggregating the neighborhood messages with the initial feature representation of each node to obtain an intermediate feature representation, until the number of iterations is equal to the preset fixed propagation step number; taking the node hidden state of the last iteration as the schema item vector of each schema item node.

6. The database query feature vector generation method of any one of claims 1 to 5, wherein, The step of determining the database schema subgraph related to the target query statement according to the database schema graph comprises: calculating the association probability of the query words in the target query statement and each schema item in the database schema graph; determining the correlation score between the target query statement and each schema item according to the association probability; retaining the nodes in the database schema graph that satisfy the preset condition and eliminating the nodes that do not satisfy the preset condition to obtain the database schema subgraph.

7. A database query feature vector generation apparatus characterized by comprising: The device comprises: a schema graph acquisition module configured to acquire the connection bitmap and the database schema graph of the database queried by the target query statement; a schema subgraph determination module configured to determine the database schema subgraph related to the target query statement according to the database schema graph; a schema item representation module configured to encode the database schema subgraph by using a preset gated graph neural network to generate a schema item vector; a query featureization module configured to determine the query feature vector corresponding to the target query statement according to the connection bitmap and the schema item vector.

8. A database query feature vector generation device characterized by comprising: The device comprises a memory, a processor and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the database query feature vector generation method according to any one of claims 1 to 6.

9. A storage medium, characterized by The storage medium is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the database query feature vector generation method according to any one of claims 1 to 6.

10. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program is executed by the processor to implement the steps of the database query feature vector generation method according to any one of claims 1 to 6.

Images