Open database-oriented method and device for converting natural language into database query language

Abstract

The invention discloses a method and a device for converting a natural language into a database query language for an open database, which are used for the technical field of information processing. The method comprises the following steps that user data are obtained, and the user data comprise question sentences and declarations introduced into a database structure; based on a pre-trained Schema Linking model, performing vectorization processing on the user question statement and the database structure to obtain a vectorized question and a vector database; based on the vectorized questions, retrieving in a vector database by using a minimum navigation algorithm to obtain statements of a database table building data definition language related to the vectorized questions; splicing the user question statement and a statement of a database table building data definition language to obtain a spliced statement; and inputting the assembled statement into a pre-trained SQLCode model to obtain a database query language. A user is helped to open a database, and statement mapping from a user question statement to a database table building data definition language is achieved.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to a method and device for converting natural language into database query language for an open database. Background Art

[0002] At present, NL2SQL / Text2SQL (natural language to database query language technology) is an important direction in the field of NLP (natural language processing). It aims to help people who do not understand the structured query language (SQL) and business database tables to quickly query structured data through natural language. At present, Schema-linking mostly uses keyword-based or fuzzy matching methods to anchor the relationship between fields and user questions. However, in the scenario of Chinese corpus, database tables are generally named in English, and the query language is generally in Chinese. Therefore, in most usage scenarios, users will be required to specify database tables for questions, and users need to filter table creation statements. In the case of open databases, it is difficult for users to specify specific database storage tables for questions without understanding the database structure and structured query language, so it is impossible to achieve the mapping of user questions (Query) to database data definition language (DDL) table creation statements, and it is difficult to achieve natural language to database query language. Summary of the invention

[0003] The present invention provides a method and device for converting natural language into database query language for an open database, aiming to help personnel who do not understand SQL language and business database tables to quickly perform structured data query through natural language.

[0004] The first aspect of the present invention provides a method for converting natural language into database query language for an open database, comprising the following steps:

[0005] Acquiring user data, wherein the user data includes a question statement and a statement for introducing a database structure;

[0006] Based on the pre-trained Schema Linking model, the user question statement and the database structure are vectorized to obtain a vectorized question and a vector database;

[0007] Based on the vectorized question, a minimum navigation algorithm is used to search the vector database to obtain a statement of a database table creation data definition language related to the vectorized question;

[0008] Assembling the user question statement with the statement of the database table creation data definition language to obtain an assembled statement;

[0009] The assembled statements are input into a pre-trained SQL Coder model to obtain a database query language.

[0010] In some embodiments of the first aspect, the pre-training process of the Schema Linking model is specifically:

[0011] Organizing the data to be trained in a first preset data format;

[0012] Based on the BERT model, the input sentence is vector-encoded and mapped to obtain a vector data group, wherein the vector data group includes a plurality of vectors and vector modules;

[0013] Inputting the data set into a classification target function for classification processing;

[0014] The weights of the BERT model are updated by optimizing the update using the cross entropy loss function, and the training is iterated until the preset conditions are met to obtain a pre-trained Schema Linking model.

[0015] In some embodiments of the first aspect, in the step of organizing the data to be trained in the first preset data format, specifically:

[0016] According to user questions, database query language and database structure as benchmark data pairs;

[0017] Converting the database structure in the reference data pair into a data definition language to form a data triple of a user question, a data definition language, and a database query language;

[0018] The data triples are split to form a preset data form of the user question and the data definition language.

[0019] In some embodiments of the first aspect, based on the BERT model, vector encoding mapping is performed on the input sentence to obtain a vector data group, where the vector data group includes multiple vectors and vector modules. In this step, specifically:

[0020] Input the first sentence and the second sentence into the BERT model respectively, and obtain the vector representation of the first classification mark and the vector representation of the second classification mark respectively;

[0021] Inputting the vector representation of the first classification mark and the vector representation of the second classification mark into a pooling pool respectively to obtain a first vector and a second vector;

[0022] Based on the first vector and the second vector, obtain a magnitude of a difference vector between the first vector and the second vector;

[0023] The modules of the first vector, the second vector and the difference vector are concatenated to obtain the data group.

[0024] In some embodiments of the first aspect, the expression of the classification objective function is:

[0025] o=softmax(W t (u,v,|uv|))

[0026] In the above formula, o is the classification target, W t is the trained model weight, u is the first vector, v is the second vector, |uv| is the modulus of the difference vector;

[0027] The expression of the cross entropy loss function is:

[0028]

[0029] In the above formula, p is the sample in the training set, q is the generated sample, and p k is the unique hot encoding corresponding to sample p in the kth training set, q k Generate the unique hot encoding of the kth sample q.

[0030] In some embodiments of the first aspect, the pre-training process of the SQL Coder model is specifically:

[0031] Organizing the data to be trained in a second preset data format;

[0032] Based on the large language model, LoRa training is fine-tuned to obtain the trained SQL Coder model.

[0033] In some embodiments of the first aspect, in the step of organizing the data to be trained in the second preset data form, specifically: taking user questions, database query language and database structure as reference data pairs;

[0034] The database structure in the reference data pair is converted into a data definition language to form a data triple of a user question, a data definition language and a database query language, wherein the data triple is a second preset data format.

[0035] In some embodiments of the first aspect, the update objective function of the LoRa training fine-tuning is specifically:

[0036]

[0037] Φ=Φ+△Φ(Θ)

[0038] In the above formula, P Φ (y|x) is the response y given by the PLM with parameter matrix Φ to question x, and Θ is the training parameter set of LoRa.

[0039] In some embodiments of the first aspect, the step of obtaining the assembled sentence further includes the following steps:

[0040] Inputting the user question sentence into a preset user question rewriting model to generate a pseudo database query logic language;

[0041] The pseudo database query logic language and the database table creation data definition language are assembled to obtain the assembled statement.

[0042] The second aspect of the present invention provides a natural language to database query language device for an open database, which applies the natural language to database query language method described in the first aspect, including:

[0043] An acquisition module, used for acquiring user data, wherein the user data includes a question statement and a statement for introducing a database structure;

[0044] A vector module, used to vectorize the user question statement and the database structure based on a pre-trained Schema Linking model to obtain a vectorized question and a vector database;

[0045] A retrieval module, used to search the vector database using a minimum navigation algorithm based on the vectorized question to obtain a statement of a database table creation data definition language related to the vectorized question;

[0046] An assembling module, used for assembling the user question statement with the statement of the database table creation data definition language to obtain an assembled statement;

[0047] The processing module inputs the assembled statement into a pre-trained SQL Coder model to obtain a database query language.

[0048] It can be seen from the above technical solutions that the present invention has at least the following advantages:

[0049] The embodiment of the present invention provides a natural language to database query language method for an open database. By using a pre-trained Schema Linking model, the user question statement and the database structure are vectorized to obtain a vectorized question and a vector database. Therefore, the user question statement and the statement of the database table creation data definition language are both represented as a vector. By calculating the similarity of two equal-length vectors, N statements of the table creation data definition language that are most relevant to the current user question statement are queried. The user does not need to screen the table creation statements. This can help the user to achieve mapping of the user question statement to the statement of the database table creation data definition language without understanding the database structure and the structured query language, and help personnel who do not understand the structured query language and business library tables to quickly perform structured data queries through natural language. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0051] Figure 1 A schematic diagram of the overall steps of a method for converting natural language to database query language for an open database provided by an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of the overall processing flow of a method for converting natural language to database query language for an open database provided by an embodiment of the present invention;

[0053] Figure 3 A schematic diagram of a processing flow of a method for converting natural language to database query language for an open database provided by an embodiment of the present invention;

[0054] Figure 4 The original data set pattern before training provided by the embodiment of the present invention;

[0055] Figure 5 The original data triplet pattern used for training provided in the embodiment of the present invention;

[0056] Figure 6 A schematic diagram of the pre-training process of the Schema Linking model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The embodiment of the present invention provides a method and device for converting natural language into database query language for an open database, aiming to solve the problem that people who do not understand SQL language and business database tables cannot quickly perform structured data query through natural language.

[0058] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] See also Figure 1 and Figure 2 , Figure 1 A schematic diagram of the overall steps of the method for converting natural language to database query language for an open database provided by an embodiment of the present invention, Figure 2 A schematic diagram of the overall processing flow of a method for converting natural language to database query language for an open database provided by an embodiment of the present invention.

[0060] S1, such as Figures 1 to 3 As shown, user data is obtained, wherein the user data includes a question statement Query and a declaration of introducing a database structure Schema. In specific implementation, when the system is introduced, the user declares to the system to introduce the database structure Schema; when the user uses the system, the system obtains the user's question Query statement Query, and the system determines that the user's question Query enters the natural language to database query language DQL process.

[0061] S2, such as Figures 1 to 3 As shown in the figure, based on the pre-trained Schema Linking model, the user's query statement and the database structure Schema are vectorized to obtain vectorized questions and vector databases. In the specific implementation, Figures 1 to 3 As shown, the database table created by the designed library table in the database is imported, and the table creation DDL statement is embedded and generated using the Schema Linking model and inserted into the vector database. The user's question Query is represented by the Schema Linking model and then embedded in the vector database for query.

[0062] In this embodiment, a Dual Encoder type embedding vector generation model is trained with the help of the Schema-Linking concept. The user's query and the database DDL table creation statement can be represented as a vector. By calculating the similarity of two vectors of equal length, the top N DDL table creation statements most relevant to the current query can be queried in subsequent steps without the need for the user to screen the table creation statements.

[0063] In this embodiment, the pre-training process of the Schema Linking model is as follows: Figure 6 As shown, specifically:

[0064] a1, organizing the data to be trained in a first preset data format.

[0065] In step a1, specifically:

[0066] a11, such as Figure 4 The original data set style shown is organized in the form of <user query, database query language DQL and database structure Schema> as the benchmark data pair.

[0067] a12, such as Figure 5 As shown, the database structure Schema in the benchmark data pair is converted into the data definition language DDL to form a data triplet of <user question Query, data definition language DDL and database query language DQL>.

[0068] a13, split the data triples to form the first preset data form of <user question Query and data definition language DDL>, that is, use the form of <user question Query and data definition language DDL>> as the training data form of the Schema-Linking model.

[0069] a2. Based on the BERT model, the input sentence is vector-encoded and mapped to obtain a vector data group, which contains multiple vectors and vector modules.

[0070] In this embodiment, in step a2, Figure 6 As shown, specifically:

[0071] a21, input the first sentence Sentence-A and the second sentence Sentence-B into the BERT model respectively, and obtain the vector representation of the first classification mark CLS token and the vector representation of the second classification mark CLS token respectively.

[0072] a21, the vector representation of the first classification mark and the vector representation of the second classification mark are respectively input into the pooling pool for pooling to obtain a first vector u and a second vector v of equal length.

[0073] a21, based on the first vector and the second vector, obtains the modulus |uv| of the difference vector between the first vector and the second vector.

[0074] a21, concatenate the first vector u, the second vector v and the modulus |uv| of the difference vector to obtain a data group.

[0075] a3, input the data group into the classification objective function for classification processing;

[0076] In this embodiment, the expression of the classification objective function used for the model in the training phase is specifically:

[0077] o=softmax(W t (u,v,|uv|))

[0078] In the above formula, o is the classification target, W t is the trained model weight, u is the first vector, v is the second vector, |uv| is the modulus of the difference vector;

[0079] a4. Use the cross entropy loss function to optimize and update the weights of the BERT model, iterate the training until the preset conditions are met, and obtain the pre-trained Schema Linking model.

[0080] In this embodiment, the expression of the cross entropy loss function used for the model in the training phase is:

[0081]

[0082] In the above formula, p is the sample in the training set, q is the generated sample, and p k is the unique hot encoding corresponding to sample p in the kth training set, q k Generate the unique hot encoding of the kth sample q.

[0083] S3, such as Figures 1 to 3 As shown, based on vectorized questions, the minimum navigation algorithm is used to search in the vector database to obtain database table creation data definition language DDL statements related to the vectorized questions. In specific implementation, by calculating the similarity of two equal-length vectors, the Top N data definition language DDL table creation statements most relevant to the current query are queried. When facing an open database, users do not need to filter table creation statements.

[0084] It can be seen from steps S2 and S3 that by representing the user's query and the database DDL table creation statement as a vector, and calculating the similarity of two vectors of equal length, the Top N DDL table creation statements most relevant to the current query are queried, without the need for the user to screen the table creation statements. This can help the user to achieve mapping from the user's query to the database DDL table creation statement without understanding the database structure schema and SQL language, and help personnel who do not understand the SQL language and business database tables to quickly perform structured data query through natural language.

[0085] S4, assemble the Query statement asked by the user and the statement of the database table creation data definition language DDL by prompt to obtain an assembled statement;

[0086] In some embodiments, a user's query Query statement is input into a user question rewriting model to generate a pseudo database query logic language (FDQL statement), and then the pseudo database query logic language FDQL and the database table creation data definition language DDL statement are prompted and assembled to form an assembled statement for inputting a pre-trained SQL Coder model.

[0087] S5, such as Figures 1 to 3 As shown, the assembled statement is input into the pre-trained SQL Coder model to obtain the database query language DQL.

[0088] In this embodiment, the pre-training process of the SQL Coder model is specifically as follows:

[0089] b1, organizing the training data in a second preset data format;

[0090] In this embodiment, step b1 specifically includes:

[0091] b11, taking user query, database query language DQL and database structure Schema as the benchmark data pair;

[0092] b12, converting the database structure Schema in the benchmark data pair into a data definition language DDL to form a data triple of user question Query, data definition language DDL and database query language DQL, where the data triple is a second preset data format.

[0093] b2. Based on the large language model, LoRa training and fine-tuning are performed to obtain the trained SQL Coder model. That is, the SQL-Generation model is trained based on the LoRa method of LLM. Compared with the model after SFT, it occupies less resources and has the same or even better SQL generation accuracy.

[0094] In this embodiment, in step b2, the update objective function of LoRa training fine-tuning is specifically:

[0095] First, define P Φ (y|x) is the PLM with parameter matrix Φ that responds to question x with y, and the downstream dataset is defined as Z = {{x i ,y i}} i＝1,…,N , when the model is fine-tuned with all parameters, the objective function is defined as follows:

[0096]

[0097] A simple gradient update for training can be defined as Φ = Φ + △Φ (Θ), where Θ is the training parameter set of LoRA, so the updated objective function can be expressed as follows:

[0098]

[0099] In the above formula, P Φ (y|x) is the response y given by the PLM with parameter matrix Φ to question x, and Θ is the training parameter set of LoRa.

[0100] Specifically for each LoRA block, the parameter matrix W of PLM can be fixed unchanged, and the learnable matrices A and B can be updated through BackWard. The specific update can be expressed as the following formula:

[0101] h=(W+△W)X=WX+BAX

[0102] In the above formula, X is the input matrix, W is the parameter matrix of PLM, △W is the fine-tuning increment matrix (or the full parameter fine-tuning increment matrix) with a dimension of d*k, B is a matrix with a dimension of d*r, and A is a matrix with a dimension of r*k.

[0103] It can be seen that in step S5, the SQL-Generation model is trained by the LoRa method based on LLM. With the help of LLM's own generation capability, the SQL statement is generalized into a code generation problem. There is no need for traditional complex SQL statement presets and slot matching. It can effectively improve the efficiency and openness of SQL generation. Compared with the model after SFT, it occupies less resources and has the same or even better SQL generation accuracy.

[0104] S6, according to the database query language DQL, calls the database query interface to obtain the corresponding data results, and at the same time calls the knowledge base retrieval according to the user query to recall relevant knowledge, and transmits the queried data and knowledge content to the return summary model to generate the corresponding AI reply.

[0105] Based on the above embodiment, the present invention further provides a natural language to database query language DQL device for an open database, which applies the natural language to database query language DQL method of the above embodiment, including:

[0106] The acquisition module is used to acquire user data, wherein the user data includes a question statement and a declaration for introducing a database structure Schema;

[0107] The vector module is used to vectorize the user's query statements and database structure schema based on the pre-trained Schema Linking model to obtain vectorized questions and vector databases;

[0108] A retrieval module is used to search in a vector database based on a vectorized question using a minimum navigation algorithm to obtain statements in a database table creation data definition language related to the vectorized question;

[0109] An assembly module is used to assemble the Query statement asked by the user with the statement of the database table building data definition language to obtain an assembled statement;

[0110] The processing module inputs the assembled statements into the pre-trained SQL Coder model to obtain the database query language DQL.

[0111] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0112] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0113] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

Claims

1. A natural language to database query language method for an open database, characterized in that: The following steps are involved: Acquiring user data, wherein the user data includes a question statement and a statement for introducing a database structure; Based on the pre-trained Schema Linking model, the user question statement and the database structure are vectorized to obtain a vectorized question and a vector database; Based on the vectorized question, a minimum navigation algorithm is used to search the vector database to obtain a statement of a database table creation data definition language related to the vectorized question; Assembling the user question statement with the statement of the database table creation data definition language to obtain an assembled statement; The assembled statements are input into a pre-trained SQL Coder model to obtain a database query language.

2. The method for converting natural language to database query language according to claim 1, characterized in that: The pre-training process of the SchemaLinking model is specifically as follows: Organizing the data to be trained in a first preset data format; Based on the BERT model, the input sentence is vector-encoded and mapped to obtain a vector data group, wherein the vector data group includes a plurality of vectors and vector modules; Inputting the data set into a classification target function for classification processing; The weights of the BERT model are updated by optimizing the update using the cross entropy loss function, and the training is iterated until the preset conditions are met to obtain a pre-trained Schema Linking model.

3. The method for converting natural language to database query language according to claim 2, characterized in that: In the step of organizing the training data in the first preset data form, specifically: According to user questions, database query language and database structure as benchmark data pairs; Converting the database structure in the reference data pair into a data definition language to form a data triple of a user question, a data definition language, and a database query language; The data triples are split to form a first preset data form of the user question and the data definition language.

4. The method for converting natural language to database query language according to claim 2, characterized in that: Based on the BERT model, the input sentence is vector-encoded and mapped to obtain a vector data group, which includes multiple vectors and vector modules. In this step, specifically: Input the first sentence and the second sentence into the BERT model respectively, and obtain the vector representation of the first classification mark and the vector representation of the second classification mark respectively; Inputting the vector representation of the first classification mark and the vector representation of the second classification mark into a pooling pool respectively to obtain a first vector and a second vector; Based on the first vector and the second vector, obtain a magnitude of a difference vector between the first vector and the second vector; The modules of the first vector, the second vector and the difference vector are concatenated to obtain the data group.

5. The method for converting natural language to database query language according to claim 2, characterized in that: The expression of the classification objective function is: o=softmax(W t (u,v,|uv|)) In the above formula, o is the classification target, W t is the trained model weight, u is the first vector, v is the second vector, |uv| is the modulus of the difference vector; The expression of the cross entropy loss function is: In the above formula, p is the sample in the training set, q is the generated sample, and p k is the unique hot encoding corresponding to sample p in the kth training set, q k Generate the unique hot encoding of the kth sample q.

6. The method for converting natural language to database query language according to claim 1, characterized in that: The pre-training process of the SQLCoder model is specifically as follows: Organizing the data to be trained in a second preset data format; Based on the large language model, LoRa training is fine-tuned to obtain the trained SQL Coder model.

7. The method for converting natural language to database query language according to claim 6, characterized in that: In the step of organizing the training data in the second preset data form, specifically: According to user questions, database query language and database structure as benchmark data pairs; The database structure in the reference data pair is converted into a data definition language to form a data triple of a user question, a data definition language and a database query language, wherein the data triple is a second preset data format.

8. The method for converting natural language to database query language according to claim 6, characterized in that: The update objective function of the LoRa training fine-tuning is specifically: Φ=Φ+△Φ(Θ) In the above formula, P Φ (y|x) is the response y given by the PLM with parameter matrix Φ to question x, and Θ is the training parameter set of LoRa.

9. The method for converting natural language to database query language according to claim 1, characterized in that: The step of obtaining the assembled statement also includes the following steps: Inputting the user question sentence into a preset user question rewriting model to generate a pseudo database query logic language; The pseudo database query logic language and the database table creation data definition language are assembled to obtain the assembled statement.

10. A natural language to database query language device for an open database, characterized in that: The method for converting natural language to database query language according to any one of claims 1 to 9 is applied, comprising: An acquisition module, used for acquiring user data, wherein the user data includes a question statement and a statement for introducing a database structure; A vector module, used to vectorize the user question statement and the database structure based on a pre-trained Schema Linking model to obtain a vectorized question and a vector database; A retrieval module, used to search the vector database using a minimum navigation algorithm based on the vectorized question to obtain a statement of a database table creation data definition language related to the vectorized question; An assembling module, used for assembling the user question statement with the statement of the database table creation data definition language to obtain an assembled statement; The processing module inputs the assembled statement into a pre-trained SQL Coder model to obtain a database query language.

Images