A project data query method and system

Abstract

The present invention relates to the field of data processing technology, and in particular to a project data query method and system, the method comprising: obtaining a question statement and database information input by a user; encoding the question statement and database information based on a pre-trained BERT model and an attention mechanism, respectively obtaining a question feature vector matrix, a table column feature vector matrix and a comprehensive feature vector matrix; generating an SQL clause according to the extracted feature vector matrix; extracting the table name to be queried based on the mapping relationship between the table name and the column name in the database, obtaining the association conditions between the table names to be queried through a preset breadth-first search algorithm; combining the SQL clauses to obtain a target SQL query statement, and sending the target SQL query statement to the database to obtain a query result. The present invention can generate structured query statements more efficiently and accurately, making the data query results more accurate.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a project data query method and system. Background Art

[0002] Traditional project data query design is mainly based on SQL database query technology, which pre-abstracts query elements of the analysis scenario to form structured query conditions, such as by project industry, scale, region, construction entity, stage, nature, etc., to guide users to conduct multi-condition joint data query. When users select conditions to locate data according to business management needs, the system translates the page query input into SQL to achieve single or joint data screening in multiple tables and databases.

[0003] Traditional methods use SQL technology to achieve dynamic data screening at the technical level, but their service capabilities in business are extremely limited. Faced with different work tasks, different levels, departments, and personnel focus on different projects, and the data perspective is changeable. The preset condition mode cannot exhaustively enumerate the query elements and build a full query condition to support the changing analysis needs. In addition, too many query conditions will also bring about operational complexity, increase the learning cost of personnel, and reduce the operational experience of personnel. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a project data query method and system.

[0005] The present invention adopts the following technical solution: a project data query method, the method comprising:

[0006] Obtaining a question statement input by a user and database information of a database to be queried, wherein the database information includes table information and a database schema diagram;

[0007] Encode the question sentence and the table information based on the pre-trained BERT model to obtain a question feature vector matrix and a table feature vector matrix respectively, and comprehensively encode the question feature vector matrix and the table feature vector matrix based on the attention mechanism to obtain a comprehensive feature vector matrix;

[0008] Generate SQL clauses according to the problem feature vector matrix, the table column feature vector matrix and the comprehensive feature vector matrix, wherein the SQL clauses include a SELECT clause, a WHERE clause, a GROUP BY clause, an ORDER BY clause and a LIMIT clause; the SELECT clause includes a column name to be queried, and the WHERE clause includes a condition column name;

[0009] Based on the mapping relationship between table names and column names in the database, extract the table names to be queried corresponding to the column names to be queried and the condition column names, obtain the association conditions between the table names to be queried in the database schema through a preset breadth-first search algorithm, and add the association conditions to the WHERE clause;

[0010] The SELECT clause, the WHERE clause, the GROUP BY clause, the ORDER BY clause and the LIMIT clause are combined to obtain a target SQL query statement, and the target SQL query statement is sent to a database to obtain a query result.

[0011] The project data query method of an embodiment of the present invention extracts features from the query text and database information input by the user, and then generates five SQL clauses, namely, SELECT clause, WHERE clause, GROUP BY clause, ORDER BY clause and LIMIT clause, according to the extracted feature vector matrix, so that it can generate structured query statements more efficiently and accurately, and show excellent effects in both performance and accuracy; then, based on the mapping relationship between the table name and the column name in the database, the table name to be queried corresponding to the column name to be queried and the condition column name is extracted, and the association conditions between the table names to be queried are obtained, so as to ensure that the generated target SQL query statement is highly matched with the database structure, improve the efficiency and stability of the database query, and make the data query result more accurate; and the present invention is based on the generation of SQL clauses, and focuses more on the database table structure and query rules, improves the utilization rate of data, reduces the demand for training data, and reduces the training cost of the model.

[0012] Furthermore, the pre-trained BERT model is used to encode the question statement and the table information to obtain a question feature vector matrix and a table feature vector matrix respectively, and the question feature vector matrix and the table feature vector matrix are comprehensively encoded based on the attention mechanism to obtain a comprehensive feature vector matrix. The steps specifically include:

[0013] The question sentence is encoded based on the pre-trained BERT model to obtain the question feature vector matrix, which is recorded as ;

[0014] Define the table information as ,in, For the In the table column names, is the number of tables in the database, For the The number of column names in the table, is the start character of the table information, is the terminator of the table information;

[0015] The table information is encoded based on the pre-trained BERT model to obtain the table feature vector matrix, which is recorded as , ,in, The start character The characteristic vector of The terminator The characteristic vector of The information in the table is In the table A character vector of column names, for The column names in the table information correspond one to one with the eigenvectors in the table eigenvector matrix;

[0016] The problem feature vector matrix and the table feature vector matrix are comprehensively encoded through the attention mechanism to obtain a comprehensive feature vector matrix :

[0017]

[0018] in, is the comprehensive coding learning factor, is the weight parameter of the eigenvector matrix of the problem, is the weight parameter of the tabular eigenvector matrix.

[0019] Furthermore, the SELECT clause includes a column name to be queried and an aggregate function corresponding to the column name to be queried, and the steps of generating the column name to be queried and the aggregate function specifically include:

[0020] The comprehensive feature vector matrix is ​​further encoded based on the preset LSTM neural network to obtain the hidden state of the last time step ;

[0021] The start character As the input of the initial decoding, it is recorded as , the hidden state As the initial state of decoding, The decoded vector is recorded as , the decoded vector The feature vector matrix of the table is calculated through the attention mechanism to obtain The probability set of selecting each column name in the database in a time step:

[0022]

[0023] in, For the The probability set of selecting each column name in the database in a time step, The number of elements in is the same as the number of column names in the table column information and they correspond one to one. For calculation The attention weight parameter when ;

[0024] Select The element with the largest probability value is denoted as , and The column name in the corresponding table column information is used as the first The column name to be queried in the time step, and the eigenvector corresponding to the column name to be queried in the table column eigenvector matrix is ​​used as the input for decoding in the next time step, recorded as ;

[0025] Until the column name in the database is selected in a time step as the terminator So far, based on the obtained order of the column names to be queried, all the column names to be queried are generated into a set of column names to be queried ;

[0026] Associating the category of the aggregation function with the preset first fully connected layer; defining the first The columns to be queried are named , then remember The eigenvectors in the tabulated eigenvector matrix are ;

[0027] The With the The first context vector is calculated through the attention mechanism:

[0028]

[0029] in, For the a first context vector corresponding to the column name to be queried, For calculation The attention weight parameter when ;

[0030] The first context vector Input to the first fully connected layer to obtain the The probability set of the aggregate function categories corresponding to the column names to be queried:

[0031]

[0032] in, For the The probability set of the aggregate function categories corresponding to the column names to be queried, and are respectively the first learning parameter and the second learning parameter of the first fully connected layer;

[0033] Select The aggregation function corresponding to the element with the highest probability is taken as the first The aggregate function corresponding to the column name to be queried.

[0034] Furthermore, the WHERE clause includes a condition column name, a condition operator and a condition value, and a triple consisting of the condition column name, the condition operator and the condition value is defined as an SQL query condition. The steps of generating the WHERE clause specifically include:

[0035] Repeat the step of generating the to-be-queried column name set by using independent learning parameters to obtain a conditional column name set, wherein the conditional column name set includes a plurality of the conditional column names;

[0036] Associating the category of the conditional operator with the preset second fully connected layer; defining the first The conditions are listed as , then remember The eigenvectors in the tabulated eigenvector matrix are ;

[0037] The With the The second context vector is calculated through the attention mechanism:

[0038]

[0039] in, For the The second context vector corresponding to the condition column name, For calculation The attention weight parameter when ;

[0040] The second context vector Input to the second fully connected layer to get the The probability set of the condition column names corresponding to the condition operator categories:

[0041]

[0042] in, For the The probability set of the condition column names corresponding to the condition operator categories, and a first learning parameter and a second learning parameter of the second fully connected layer, respectively;

[0043] Select The conditional operator corresponding to the element with the highest probability is used as the first The condition operator corresponding to each of the condition column names;

[0044] The With the Calculate the third context vector through the attention mechanism :

[0045]

[0046] in, For the The third context vector corresponding to the condition column name, For calculation The attention weight parameter when ;

[0047] Based on the preset LSTM neural network As the encoding input, get the hidden state of the last time step , the hidden state As the initial state of decoding, the start character As the initial decoding input, the conditional tag sequence is obtained by step-by-step decoding until the termination symbol is generated. So far, the condition value can be obtained according to the condition tag sequence.

[0048] Furthermore, the steps of generating the GROUP BY clause specifically include:

[0049] The With the The fourth context vector is calculated through the attention mechanism:

[0050]

[0051] in, For the a fourth context vector corresponding to the column name to be queried, For calculation The attention weight parameter when ;

[0052] Calculate the The probability that the column name to be queried is stored in the GROUP BY clause:

[0053]

[0054] in, For the The probability that the column name to be queried is stored in the GROUP BY clause, For the The weight parameter, For the The weight parameter of

[0055] like is greater than the preset first probability threshold, then The column names to be queried are stored in the GROUP BY clause.

[0056] Furthermore, the steps of generating the ORDER BY clause specifically include:

[0057] Associating the sorting category of the column name to be queried with the preset third fully connected layer; wherein the sorting category of the column name to be queried includes 'NONE', 'ASC', 'DESC';

[0058] The With the Calculate the fifth context vector through the attention mechanism :

[0059]

[0060] in, For the a fifth context vector corresponding to the column name to be queried, For calculation The attention weight parameter when ;

[0061] The Input to the third fully connected layer to obtain the The probability set of the sorting method categories corresponding to the column names to be queried:

[0062]

[0063] in, For the The probability set of the sorting method categories corresponding to the column names to be queried, and are respectively the first learning parameter and the second learning parameter of the third fully connected layer;

[0064] Select The sorting method corresponding to the element with the highest probability is taken as the first The sorting method corresponding to the column name to be queried is used to obtain the ORDER BY clause;

[0065] Among them, if The sorting method of the column name to be queried is 'NONE', then The column name to be queried is not included in the ORDER BY clause. If the sorting method of the column name to be queried is 'ASC' or 'DESC', then Add the column names to be queried to the ORDER BY clause, and The column names to be queried are sorted in ascending order or descending order.

[0066] Furthermore, the step of generating the Limit clause specifically includes:

[0067] The fourth fully connected layer is used to Perform binary classification to determine whether the Limit clause is needed. The output probability of the fourth fully connected layer for:

[0068]

[0069] in, and are respectively the first learning parameter and the second learning parameter of the fourth fully connected layer;

[0070] If the If the probability is greater than a preset second probability threshold, it is determined that the Limit clause is needed;

[0071] based on Function determines the Limit value of the Limit clause :

[0072]

[0073] in, , , and They are respectively the first learning parameter, the second learning parameter, the third learning parameter and the fourth learning parameter of the Limit value.

[0074] The present invention also provides a project data query system, the system comprising:

[0075] An acquisition module, used to acquire the question statement input by the user and the database information of the database to be queried, wherein the database information includes table information and a database model diagram;

[0076] An encoding module is used to encode the question sentence and the table information based on a pre-trained BERT model to obtain a question feature vector matrix and a table feature vector matrix respectively, and to comprehensively encode the question feature vector matrix and the table feature vector matrix based on an attention mechanism to obtain a comprehensive feature vector matrix;

[0077] An SQL clause module, used for generating SQL clauses according to the problem feature vector matrix, the table column feature vector matrix and the comprehensive feature vector matrix, wherein the SQL clauses include a SELECT clause, a WHERE clause, a GROUP BY clause, an ORDER BY clause and a LIMIT clause;

[0078] A table association condition module, used for extracting the table name to be queried corresponding to the column name to be queried and the condition column name based on the mapping relationship between the table name and the column name in the database, obtaining the association condition between the table names to be queried in the database schema through a preset breadth-first search algorithm, and adding the association condition to the WHERE clause;

[0079] The query module is used to combine the SELECT clause, the WHERE clause, the GROUP BY clause, the ORDER BY clause and the LIMIT clause to obtain a target SQL query statement, and send the target SQL query statement to a database to obtain a query result.

[0080] The project data query system of an embodiment of the present invention extracts features from the query text and database information input by the user, and then generates five SQL clauses, namely, SELECT clause, WHERE clause, GROUP BY clause, ORDER BY clause and LIMIT clause, according to the extracted feature vector matrix, so that it can generate structured query statements more efficiently and accurately, and show excellent effects in both performance and accuracy; then, based on the mapping relationship between the table name and the column name in the database, the table name to be queried corresponding to the column name to be queried and the condition column name is extracted, and the association conditions between the table names to be queried are obtained, so as to ensure that the generated target SQL query statement is highly matched with the database structure, improve the efficiency and stability of the database query, and make the data query result more accurate; and the present invention is based on the generation of SQL clauses, and focuses more on the database table structure and query rules, so as to improve the utilization rate of data, reduce the demand for training data, and reduce the training cost of the model.

[0081] The present invention also provides a readable storage medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned project data query method is implemented.

[0082] The present invention also proposes a terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned project data query method when executing the computer program. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0084] Figure 1 It is a flow chart of the project data query method of the present invention;

[0085] Figure 2 It is a structural block diagram of the project data query system of the present invention. DETAILED DESCRIPTION

[0086] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the embodiments of the present invention, and should not be construed as limiting the present invention.

[0087] In the description of the embodiments of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0088] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0089] In the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present invention can be understood according to specific circumstances.

[0090] Embodiment 1

[0091] Reference Figure 1 The first embodiment of the present invention is a project data query method, the method comprising:

[0092] S1: Obtain the question statement input by the user and the database information of the database to be queried, where the database information includes table column information and database model diagram; the database, as the underlying storage and data processing engine, is responsible for accepting and executing SQL query statements, aggregating, filtering, sorting and other operations on the data to meet the query and analysis needs based on the data set.

[0093] S2: Encode the question sentence and the table information based on the pre-trained BERT model to obtain the question feature vector matrix and the table feature vector matrix respectively, and comprehensively encode the question feature vector matrix and the table feature vector matrix based on the attention mechanism to obtain the comprehensive feature vector matrix; further, the steps specifically include:

[0094] Encode the question sentence based on the pre-trained BERT model to obtain the question feature vector matrix, recorded as ;

[0095] Define the table column information as ,in, For the In the table column names, is the number of tables in the database, For the The number of column names in the table, It is the start character of the table information. It is the terminator of the table information;

[0096] Based on the pre-trained BERT model, the table information is encoded to obtain the table feature vector matrix, which is recorded as , ,in, Start character The characteristic vector of is the terminator The characteristic vector of For the information listed in the table In the table A character vector of column names, for The column names in the table information correspond one to one with the eigenvectors in the table eigenvector matrix;

[0097] The problem feature vector matrix and the table feature vector matrix are encoded comprehensively through the attention mechanism to obtain the comprehensive feature vector matrix :

[0098]

[0099] in, is the comprehensive coding learning factor, is the weight parameter of the problem eigenvector matrix, is the weight parameter of the tabulated eigenvector matrix.

[0100] S3: Generate SQL clauses according to the problem feature vector matrix, the table column feature vector matrix and the comprehensive feature vector matrix, wherein the SQL clauses include a SELECT clause, a WHERE clause, a GROUP BY clause, an ORDER BY clause and a LIMIT clause; the SELECT clause includes the name of the column to be queried, and the WHERE clause includes the name of the condition column;

[0101] Furthermore, the SELECT clause includes the column name to be queried and the aggregate function corresponding to the column name to be queried, and the steps of generating the column name to be queried and the aggregate function specifically include:

[0102] The comprehensive feature vector matrix is ​​further encoded based on the preset LSTM neural network to obtain the hidden state of the last time step ;

[0103] The start character As the input of the initial decoding, it is recorded as , hidden state As the initial state of decoding, The decoded vector is recorded as , the decoded vector The feature vector matrix of the table is calculated through the attention mechanism to obtain The probability set of selecting each column name in the database in a time step:

[0104]

[0105] in, For the The probability set of selecting each column name in the database in a time step, The number of elements in the table is the same as the number of column names in the table column information and they correspond one to one. For calculation The attention weight parameter when ;

[0106] Select The element with the largest probability value is denoted as , and The column name in the corresponding table column information is used as the first The column name to be queried in the time step, and the corresponding eigenvector of the column name to be queried in the table column eigenvector matrix is ​​used as the input for decoding in the next time step, denoted as ;

[0107] Until the column name in the database is selected in a time step, it is the terminator So far, based on the order of the obtained column names to be queried, all the column names to be queried are generated into a set of column names to be queried ;

[0108] Associate the category of the aggregation function with the preset first fully connected layer; define the first The columns to be queried are named ,but The corresponding eigenvector in the tabular eigenvector matrix is ;

[0109] Will and The first context vector is calculated through the attention mechanism:

[0110]

[0111] in, For the The first context vector corresponding to the column name to be queried, For calculation The attention weight parameter when ;

[0112] The first context vector Input to the first fully connected layer to get the The probability set of the aggregate function categories corresponding to the column names to be queried:

[0113]

[0114] in, For the The probability set of the aggregate function categories corresponding to the column names to be queried, and are the first learning parameters and the second learning parameters of the first fully connected layer respectively;

[0115] Select The aggregation function corresponding to the element with the highest probability is taken as the first Aggregate functions corresponding to the column names to be queried; in this embodiment, the aggregate functions include NONE aggregate function, MAX aggregate function, MIN aggregate function, COUNT aggregate function, SUM aggregate function, and AVG aggregate function.

[0116] Furthermore, the WHERE clause includes the condition column name, condition operator and condition value. The triple [COLUMN, OP, VALUE] consisting of the condition column name, condition operator and condition value constitutes the SQL query condition, COLUMN is the condition column name, OP is the condition operator , VALUE is the conditional value, and the steps for generating the WHERE clause include:

[0117] Repeat the step of generating a set of column names to be queried using independent learning parameters to obtain a set of conditional column names, wherein the set of conditional column names includes multiple conditional column names;

[0118] Associate the category of the conditional operator with the preset second fully connected layer; define the first The condition column is named , then remember The eigenvectors in the tabulated eigenvector matrix are ;

[0119] Will and The second context vector is calculated through the attention mechanism:

[0120]

[0121] in, For the The second context vector corresponding to the conditional column name, For calculation The attention weight parameter when ;

[0122] The second context vector Input to the second fully connected layer to get the The probability set of condition column names corresponding to condition operator categories:

[0123]

[0124] in, For the The probability set of condition column names corresponding to the condition operator categories, and the first learning parameters and the second learning parameters of the second fully connected layer respectively;

[0125] Select The conditional operator corresponding to the element with the highest probability is taken as the first The condition operator corresponding to the condition column name;

[0126] Will and Calculate the third context vector through the attention mechanism :

[0127]

[0128] in, For the The third context vector corresponding to the conditional column name, For calculation The attention weight parameter when ;

[0129] Based on the preset LSTM neural network As the encoding input, get the hidden state of the last time step , the hidden state As the initial state of decoding, the start character As the initial decoding input, the conditional tag sequence is obtained by step-by-step decoding until the terminal symbol is generated So far, the conditional value can be obtained according to the conditional tag sequence.

[0130] Furthermore, the steps for generating the GROUP BY clause specifically include:

[0131] Will and The fourth context vector is calculated through the attention mechanism:

[0132]

[0133] in, For the The fourth context vector corresponding to the column name to be queried, For calculation The attention weight parameter when ;

[0134] Calculate the The probability that the column names to be queried are stored in the GROUP BY clause:

[0135]

[0136] in, For the The probability that the column names to be queried are stored in the GROUP BY clause, for The weight parameter, for The weight parameter of

[0137] like is greater than the preset first probability threshold, then The names of the columns to be queried are stored in the GROUP BY clause; in this embodiment, the first probability threshold value is 0.6.

[0138] Furthermore, the steps for generating the ORDER BY clause specifically include:

[0139] Associating the sorting method category of the column name to be queried with the preset third fully connected layer; wherein the sorting method category of the column name to be queried includes 'NONE', 'ASC', 'DESC';

[0140] Will and Calculate the fifth context vector through the attention mechanism :

[0141]

[0142] in, For the The fifth context vector corresponding to the column name to be queried, For calculation The attention weight parameter when ;

[0143] Will Input to the third fully connected layer to get The probability set of sorting categories corresponding to the column names to be queried:

[0144]

[0145] in, For the The probability set of sorting categories corresponding to the column names to be queried, and are the first learning parameters and the second learning parameters of the third fully connected layer, respectively;

[0146] Select The sorting method corresponding to the element with the highest probability is taken as the first The sorting method corresponding to the column name to be queried, and thus the ORDER BY clause is obtained;

[0147] Among them, if If the sorting method of the first column name to be queried is 'NONE', then The column name to be queried is not included in the ORDER BY clause. If the sorting method of the first column name to be queried is 'ASC' or 'DESC', Add the column name to be queried to the ORDER BY clause, and Sort the column names to be queried in ascending or descending order.

[0148] Furthermore, the steps for generating the Limit clause specifically include:

[0149] Use the preset fourth fully connected layer to Perform binary classification and use this to determine whether a Limit clause is needed. The output probability of the fourth fully connected layer for:

[0150]

[0151] in, and are the first learning parameters and the second learning parameters of the fourth fully connected layer, respectively;

[0152] like If it is greater than a preset second probability threshold, it is determined that a Limit clause is required; in this embodiment, the value of the second probability threshold is 0.5;

[0153] based on Function determines the Limit value of the Limit clause :

[0154]

[0155] in, , , and They are the first learning parameter, the second learning parameter, the third learning parameter and the fourth learning parameter of the Limit value respectively.

[0156] S4: Based on the mapping relationship between the table name and the column name in the database, the table name to be queried corresponding to the column name to be queried and the condition column name is extracted, and the association conditions between the table names to be queried are obtained in the database schema diagram through the preset breadth-first search algorithm, and the association conditions are added to the WHERE clause; the database schema diagram describes the primary and foreign key relationships between the tables in the database, and the breadth-first search algorithm is well known to those skilled in the art, so it will not be described in detail here.

[0157] S5: Combine the SELECT clause, WHERE clause, GROUP BY clause, ORDER BY clause and LIMIT clause to obtain a target SQL query statement, send the target SQL query statement to the database to obtain query results, and organize the query results into table data to display to the user.

[0158] The project data query method of an embodiment of the present invention extracts features from the query text and database information input by the user, and then generates five SQL clauses, namely, SELECT clause, WHERE clause, GROUP BY clause, ORDER BY clause and LIMIT clause, according to the extracted feature vector matrix, so that it can generate structured query statements more efficiently and accurately, and show excellent effects in both performance and accuracy; then, based on the mapping relationship between the table name and the column name in the database, the table name to be queried corresponding to the column name to be queried and the condition column name is extracted, and the association conditions between the table names to be queried are obtained, so as to ensure that the generated target SQL query statement is highly matched with the database structure, improve the efficiency and stability of the database query, and make the data query result more accurate; and the present invention is based on the generation of SQL clauses, and focuses more on the database table structure and query rules, improves the utilization rate of data, reduces the demand for training data, and reduces the training cost of the model.

[0159] Embodiment 2

[0160] Reference Figure 2 The present invention also proposes a project data query system, the system comprising:

[0161] An acquisition module, used to acquire the question statement input by the user and the database information of the database to be queried, wherein the database information includes table information and a database model diagram;

[0162] The encoding module is used to encode the question sentence and the table information based on the pre-trained BERT model to obtain the question feature vector matrix and the table feature vector matrix respectively, and to comprehensively encode the question feature vector matrix and the table feature vector matrix based on the attention mechanism to obtain the comprehensive feature vector matrix;

[0163] An SQL clause module is used to generate SQL clauses according to the problem feature vector matrix, the table column feature vector matrix and the comprehensive feature vector matrix, wherein the SQL clauses include a SELECT clause, a WHERE clause, a GROUP BY clause, an ORDER BY clause and a LIMIT clause;

[0164] The table association condition module is used to extract the table name to be queried corresponding to the column name to be queried and the condition column name based on the mapping relationship between the table name and the column name in the database, obtain the association condition between the table names to be queried in the database schema through a preset breadth-first search algorithm, and add the association condition to the WHERE clause;

[0165] The query module is used to combine the SELECT clause, the WHERE clause, the GROUP BY clause, the ORDER BY clause and the LIMIT clause to obtain a target SQL query statement, and send the target SQL query statement to the database to obtain a query result.

[0166] Furthermore, the encoding module is specifically used for:

[0167] Encode the question sentence based on the pre-trained BERT model to obtain the question feature vector matrix, recorded as ;

[0168] Encode the question sentence based on the pre-trained BERT model to obtain the question feature vector matrix, recorded as ;

[0169] Define the table column information as ,in, For the In the table column names, is the number of tables in the database, For the The number of column names in the table, It is the start character of the table information. It is the terminator of the table information;

[0170] Based on the pre-trained BERT model, the table information is encoded to obtain the table feature vector matrix, which is recorded as , ,in, Start character The characteristic vector of is the terminator The characteristic vector of For the information listed in the table In the table A character vector of column names, for The column names in the table information correspond one to one with the eigenvectors in the table eigenvector matrix;

[0171] The problem feature vector matrix and the table feature vector matrix are encoded comprehensively through the attention mechanism to obtain the comprehensive feature vector matrix :

[0172]

[0173] in, is the comprehensive coding learning factor, is the weight parameter of the problem eigenvector matrix, is the weight parameter of the tabulated eigenvector matrix.

[0174] Furthermore, the SELECT clause includes the column name to be queried and the aggregate function corresponding to the column name to be queried, and the steps of generating the column name to be queried and the aggregate function specifically include:

[0175] The comprehensive feature vector matrix is ​​further encoded based on the preset LSTM neural network to obtain the hidden state of the last time step ;

[0176] The start character As the input of the initial decoding, it is recorded as , hidden state As the initial state of decoding, The decoded vector is recorded as , the decoded vector The feature vector matrix of the table is calculated through the attention mechanism to obtain The probability set of selecting each column name in the database in a time step:

[0177]

[0178] in, For the The probability set of selecting each column name in the database in a time step, The number of elements in the table is the same as the number of column names in the table column information and they correspond one to one. For calculation The attention weight parameter when ;

[0179] Select The element with the largest probability value is denoted as , and The column name in the corresponding table column information is used as the first The column name to be queried in the time step, and the corresponding eigenvector of the column name to be queried in the table column eigenvector matrix is ​​used as the input for decoding in the next time step, denoted as ;

[0180] Until the column name in the database is selected in a time step, it is the terminator So far, based on the order of the obtained column names to be queried, all the column names to be queried are generated into a set of column names to be queried ;

[0181] Associate the category of the aggregation function with the preset first fully connected layer; define the first The columns to be queried are named ,but The corresponding eigenvector in the tabular eigenvector matrix is ;

[0182] Will and The first context vector is calculated through the attention mechanism:

[0183]

[0184] in, For the The first context vector corresponding to the column name to be queried, For calculation The attention weight parameter when ;

[0185] The first context vector Input to the first fully connected layer to get the The probability set of the aggregate function categories corresponding to the column names to be queried:

[0186]

[0187] in, For the The probability set of the aggregate function categories corresponding to the column names to be queried, and are the first learning parameters and the second learning parameters of the first fully connected layer respectively;

[0188] Select The aggregation function corresponding to the element with the highest probability is taken as the first Aggregate functions corresponding to the column names to be queried; in this embodiment, the aggregate functions include NONE aggregate function, MAX aggregate function, MIN aggregate function, COUNT aggregate function, SUM aggregate function, and AVG aggregate function.

[0189] Furthermore, the WHERE clause includes a condition column name, a condition operator, and a condition value. The triple consisting of the condition column name, the condition operator, and the condition value is defined as an SQL query condition. The steps for generating the WHERE clause specifically include:

[0190] Repeat the step of generating a set of column names to be queried using independent learning parameters to obtain a set of conditional column names, wherein the set of conditional column names includes multiple conditional column names;

[0191] Associate the category of the conditional operator with the preset second fully connected layer; define the first The condition column is named , then remember The eigenvectors in the tabulated eigenvector matrix are ;

[0192] Will and The second context vector is calculated through the attention mechanism:

[0193]

[0194] in, For the The second context vector corresponding to the conditional column name, For calculation The attention weight parameter when ;

[0195] The second context vector Input to the second fully connected layer to get the The probability set of condition column names corresponding to condition operator categories:

[0196]

[0197] in, For the The probability set of condition column names corresponding to the condition operator categories, and the first learning parameters and the second learning parameters of the second fully connected layer respectively;

[0198] Select The conditional operator corresponding to the element with the highest probability is taken as the first The condition operator corresponding to the condition column name;

[0199] Will and Calculate the third context vector through the attention mechanism :

[0200]

[0201] in, For the The third context vector corresponding to the conditional column name, For calculation The attention weight parameter when ;

[0202] Based on the preset LSTM neural network As the encoding input, get the hidden state of the last time step , the hidden state As the initial state of decoding, the start character As the initial decoding input, the conditional tag sequence is obtained by step-by-step decoding until the terminal symbol is generated So far, the conditional value can be obtained according to the conditional tag sequence.

[0203] Furthermore, the steps for generating the GROUP BY clause specifically include:

[0204] Will and The fourth context vector is calculated through the attention mechanism:

[0205]

[0206] in, For the The fourth context vector corresponding to the column name to be queried, For calculation The attention weight parameter when ;

[0207] Calculate the The probability that the column names to be queried are stored in the GROUP BY clause:

[0208]

[0209] in, For the The probability that the column names to be queried are stored in the GROUP BY clause, for The weight parameter, for The weight parameter of

[0210] like is greater than the preset first probability threshold, then The names of the columns to be queried are stored in the GROUP BY clause; in this embodiment, the first probability threshold value is 0.6.

[0211] Furthermore, the steps for generating the ORDER BY clause specifically include:

[0212] Associating the sorting method category of the column name to be queried with the preset third fully connected layer; wherein the sorting method category of the column name to be queried includes 'NONE', 'ASC', 'DESC';

[0213] Will and Calculate the fifth context vector through the attention mechanism :

[0214]

[0215] in, For the The fifth context vector corresponding to the column name to be queried, For calculation The attention weight parameter when ;

[0216] Will Input to the third fully connected layer to get The probability set of sorting categories corresponding to the column names to be queried:

[0217]

[0218] in, For the The probability set of sorting categories corresponding to the column names to be queried, and are the first learning parameters and the second learning parameters of the third fully connected layer, respectively;

[0219] Select The sorting method corresponding to the element with the highest probability is taken as the first The sorting method corresponding to the column name to be queried, and the ORDER BY clause is obtained from it;

[0220] Among them, if If the sorting method of the first column name to be queried is 'NONE', then The column name to be queried is not included in the ORDER BY clause. If the sorting method of the first column name to be queried is 'ASC' or 'DESC', Add the column name to be queried to the ORDER BY clause, and Sort the column names to be queried in ascending or descending order.

[0221] Furthermore, the steps for generating the Limit clause specifically include:

[0222] Use the preset fourth fully connected layer to Perform binary classification and use this to determine whether a Limit clause is needed. The output probability of the fourth fully connected layer for:

[0223]

[0224] in, and are the first learning parameters and the second learning parameters of the fourth fully connected layer, respectively;

[0225] like If it is greater than a preset second probability threshold, it is determined that a Limit clause is required; in this embodiment, the value of the second probability threshold is 0.5;

[0226] based on Function determines the Limit value of the Limit clause :

[0227]

[0228] in, , , and They are the first learning parameter, the second learning parameter, the third learning parameter and the fourth learning parameter of the Limit value respectively.

[0229] The project data query system of an embodiment of the present invention extracts features from the query text and database information input by the user, and then generates five SQL clauses, namely, SELECT clause, WHERE clause, GROUP BY clause, ORDER BY clause and LIMIT clause, according to the extracted feature vector matrix, so that it can generate structured query statements more efficiently and accurately, and show excellent effects in both performance and accuracy; then, based on the mapping relationship between the table name and the column name in the database, the table name to be queried corresponding to the column name to be queried and the condition column name is extracted, and the association conditions between the table names to be queried are obtained, so as to ensure that the generated target SQL query statement is highly matched with the database structure, improve the efficiency and stability of the database query, and make the data query result more accurate; and the present invention is based on the generation of SQL clauses, and focuses more on the database table structure and query rules, so as to improve the utilization rate of data, reduce the demand for training data, and reduce the training cost of the model.

[0230] Embodiment 3

[0231] The third embodiment of the present invention is based on the same inventive concept and proposes a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of the project data query method of the above embodiment are implemented.

[0232] The logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution system, device or apparatus. For purposes of this specification, "computer-readable medium" may be any device that contains storage, communication, propagation or transmission of a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution system, device or apparatus.

[0233] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0234] Among them, the memory may include a large-capacity memory for data or instructions. By way of example and not limitation, the memory may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory may include a removable or non-removable (or fixed) medium. Where appropriate, the memory may be inside or outside the data processing device. In a specific embodiment, the memory is a non-volatile memory. In a specific embodiment, the memory includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0235] Embodiment 4

[0236] The fourth embodiment of the present invention is based on the same inventive concept. The present invention proposes a terminal, which includes: a processor and a memory; the processor and the memory communicate with each other; the memory is used to store instructions; the processor is used to execute the instructions in the memory to execute the project data query method of the above embodiment.

[0237] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0238] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0239] Under the premise that no conflict occurs, those skilled in the art may freely combine and superimpose the above-mentioned additional technical features.

[0240] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A project data query method, characterized in that: The method comprises: Obtaining a question statement input by a user and database information of a database to be queried, wherein the database information includes table information and a database schema diagram; Encode the question sentence and the table information based on the pre-trained BERT model to obtain a question feature vector matrix and a table feature vector matrix respectively, and comprehensively encode the question feature vector matrix and the table feature vector matrix based on the attention mechanism to obtain a comprehensive feature vector matrix; Generate SQL clauses according to the problem feature vector matrix, the table column feature vector matrix and the comprehensive feature vector matrix, wherein the SQL clauses include a SELECT clause, a WHERE clause, a GROUP BY clause, an ORDER BY clause and a LIMIT clause; the SELECT clause includes a column name to be queried, and the WHERE clause includes a condition column name; Based on the mapping relationship between table names and column names in the database, extract the table name to be queried corresponding to the column name to be queried and the condition column name, obtain the association conditions between multiple table names to be queried in the database schema through a preset breadth-first search algorithm, and add the association conditions to the WHERE clause; The SELECT clause, the WHERE clause, the GROUP BY clause, the ORDER BY clause and the LIMIT clause are combined to obtain a target SQL query statement, and the target SQL query statement is sent to a database to obtain a query result.

2. The project data query method according to claim 1, characterized in that: The step of encoding the question sentence and the table information based on the pre-trained BERT model to obtain a question feature vector matrix and a table feature vector matrix respectively, and comprehensively encoding the question feature vector matrix and the table feature vector matrix based on the attention mechanism to obtain a comprehensive feature vector matrix specifically includes: The question sentence is encoded based on the pre-trained BERT model to obtain the question feature vector matrix, which is recorded as ; Define the table information as ,in, For the In the table column names, is the number of tables in the database, For the The number of column names in the table, is the start character of the table information, is the terminator of the table information; The table information is encoded based on the pre-trained BERT model to obtain the table feature vector matrix, which is recorded as , ,in, The start character The characteristic vector of The terminator The characteristic vector of The information in the table is In the table A character vector of column names, for The column names in the table information correspond one to one with the eigenvectors in the table eigenvector matrix; The problem feature vector matrix and the table feature vector matrix are comprehensively encoded through the attention mechanism to obtain a comprehensive feature vector matrix : in, is the comprehensive coding learning factor, is the weight parameter of the eigenvector matrix of the problem, is the weight parameter of the tabular eigenvector matrix.

3. The project data query method according to claim 2, characterized in that: The SELECT clause includes a column name to be queried and an aggregate function corresponding to the column name to be queried, and the steps of generating the column name to be queried and the aggregate function specifically include: The comprehensive feature vector matrix is ​​further encoded based on the preset LSTM neural network to obtain the hidden state of the last time step ; The start character As the input of the initial decoding, it is recorded as , the hidden state As the initial state of decoding, The decoded vector is recorded as , the decoded vector The feature vector matrix of the table is calculated through the attention mechanism to obtain The probability set of selecting each column name in the database in a time step: in, For the The probability set of selecting each column name in the database in a time step, The number of elements in is the same as the number of column names in the table column information and they correspond one to one. For calculation The attention weight parameter when ; Select The element with the largest probability value is denoted as , and The column name in the corresponding table column information is used as the first The column name to be queried in the time step, and the eigenvector corresponding to the column name to be queried in the table column eigenvector matrix is ​​used as the input for decoding in the next time step, recorded as ; Until the column name in the database is selected in a time step, it is the terminator So far, based on the obtained order of the column names to be queried, all the column names to be queried are generated into a set of column names to be queried ; Associating the category of the aggregation function with the preset first fully connected layer; defining the first The columns to be queried are named , then remember The eigenvectors in the tabulated eigenvector matrix are ; The With the The first context vector is calculated through the attention mechanism: in, For the a first context vector corresponding to the column name to be queried, For calculation The attention weight parameter when ; The first context vector Input to the first fully connected layer to obtain the The probability set of the aggregate function categories corresponding to the column names to be queried: in, For the The probability set of the aggregate function categories corresponding to the column names to be queried, and are respectively the first learning parameter and the second learning parameter of the first fully connected layer; Select The aggregation function corresponding to the element with the highest probability is taken as the first The aggregate function corresponding to the column name to be queried.

4. The project data query method according to claim 3, characterized in that: The WHERE clause includes a condition column name, a condition operator and a condition value. A triple consisting of the condition column name, the condition operator and the condition value is defined as an SQL query condition. The steps of generating the WHERE clause specifically include: Repeat the step of generating the to-be-queried column name set by using independent learning parameters to obtain a conditional column name set, wherein the conditional column name set includes a plurality of the conditional column names; Associating the category of the conditional operator with the preset second fully connected layer; defining the first The conditions are listed as , then remember The eigenvectors in the tabulated eigenvector matrix are ; The With the The second context vector is calculated through the attention mechanism: in, For the The second context vector corresponding to the condition column name, For calculation The attention weight parameter when ; The second context vector Input to the second fully connected layer to get the The probability set of the condition column names corresponding to the condition operator categories: in, For the The probability set of the condition column names corresponding to the condition operator categories, and a first learning parameter and a second learning parameter of the second fully connected layer, respectively; Select The conditional operator corresponding to the element with the highest probability is used as the first The condition operator corresponding to each of the condition column names; The With the Calculate the third context vector through the attention mechanism : in, For the The third context vector corresponding to the condition column name, For calculation The attention weight parameter when ; Based on the preset LSTM neural network As the encoding input, get the hidden state of the last time step , the hidden state As the initial state of decoding, the start character As the initial decoding input, the conditional tag sequence is obtained by step-by-step decoding until the termination symbol is generated. So far, the condition value can be obtained according to the condition tag sequence.

5. The project data query method according to claim 3, characterized in that: The steps for generating the GROUP BY clause specifically include: The With the The fourth context vector is calculated through the attention mechanism: in, For the a fourth context vector corresponding to the column name to be queried, For calculation The attention weight parameter when ; Calculate the The probability that the column name to be queried is stored in the GROUP BY clause: in, For the The probability that the column name to be queried is stored in the GROUP BY clause, For the The weight parameter, For the The weight parameter of like is greater than the preset first probability threshold, then The column names to be queried are stored in the GROUP BY clause.

6. The project data query method according to claim 3, characterized in that: The steps for generating the ORDER BY clause specifically include: Associating the sorting category of the column name to be queried with the preset third fully connected layer; wherein the sorting category of the column name to be queried includes 'NONE', 'ASC', 'DESC'; The With the Calculate the fifth context vector through the attention mechanism : in, For the a fifth context vector corresponding to the column name to be queried, For calculation The attention weight parameter when ; The Input to the third fully connected layer to obtain the The probability set of the sorting method categories corresponding to the column names to be queried: in, For the The probability set of the sorting method categories corresponding to the column names to be queried, and are respectively the first learning parameter and the second learning parameter of the third fully connected layer; Select The sorting method corresponding to the element with the highest probability is taken as the first The sorting method corresponding to the column name to be queried is used to obtain the ORDER BY clause; Among them, if The sorting method of the column name to be queried is 'NONE', then The column name to be queried is not included in the ORDER BY clause. If the sorting method of the column name to be queried is 'ASC' or 'DESC', then Add the column names to be queried to the ORDER BY clause, and The column names to be queried are sorted in ascending order or descending order.

7. The project data query method according to claim 3, characterized in that: The steps of generating the Limit clause specifically include: The fourth fully connected layer is used to Perform binary classification to determine whether the Limit clause is needed. The output probability of the fourth fully connected layer for: in, and are respectively the first learning parameter and the second learning parameter of the fourth fully connected layer; If the If the probability is greater than a preset second probability threshold, it is determined that the Limit clause is needed; based on Function determines the Limit value of the Limit clause : in, , , and They are respectively the first learning parameter, the second learning parameter, the third learning parameter and the fourth learning parameter of the Limit value.

8. A project data query system, characterized in that: The system comprises: An acquisition module, used to acquire the question statement input by the user and the database information of the database to be queried, wherein the database information includes table information and a database model diagram; An encoding module is used to encode the question sentence and the table information based on a pre-trained BERT model to obtain a question feature vector matrix and a table feature vector matrix respectively, and to comprehensively encode the question feature vector matrix and the table feature vector matrix based on an attention mechanism to obtain a comprehensive feature vector matrix; An SQL clause module, used for generating SQL clauses according to the problem feature vector matrix, the table column feature vector matrix and the comprehensive feature vector matrix, wherein the SQL clauses include a SELECT clause, a WHERE clause, a GROUP BY clause, an ORDER BY clause and a LIMIT clause; the SELECT clause includes a column name to be queried, and the WHERE clause includes a condition column name; A table association condition module, used for extracting the table name to be queried corresponding to the column name to be queried and the condition column name based on the mapping relationship between the table name and the column name in the database, obtaining the association conditions between the multiple table names to be queried in the database schema through a preset breadth-first search algorithm, and adding the association conditions to the WHERE clause; The query module is used to combine the SELECT clause, the WHERE clause, the GROUP BY clause, the ORDER BY clause and the LIMIT clause to obtain a target SQL query statement, and send the target SQL query statement to a database to obtain a query result.

9. A readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the project data query method as described in any one of claims 1 to 7 is implemented.

10. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the project data query method according to any one of claims 1 to 7 is implemented.

Images