Query generation method in Text2SQL (Structured Query Language) task based on large language model

Abstract

The invention discloses a query generation method in a Text2SQL (Structured Query Language) task based on a large language model, which comprises the following steps of: collecting statement information in a plurality of user interaction rounds, carrying out user dialogue intention identification according to a learning model and context tracking, and fusing the obtained dialogue intention information into a prompt; constructing a problem analysis model to obtain an analysis target result, wherein the analysis target result comprises user SQL query intention recognition and entity extraction; and identifying entities, conditions and sorting requirements involved in the question by utilizing the question analysis model, analyzing a mode of a required SQL structure, realizing SQL mode identification, obtaining a mapping relationship between the question and the SQL query to generate an SQL query statement meeting user requirements, and performing SQL query structure supervised fine adjustment on the generated SQL query statement. According to the method, the query generation method in the Text2SQL task emphasizing situation learning, new prompt engineering and supervised fine tuning can be realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer natural language processing and natural language processing-semantic parsing, and in particular to a query generation method and system in a natural language processing-semantic parsing subtask. Background Art

[0002] SQL (Structured Query Language) is a standardized programming language specifically used to communicate with databases. It is used to manage relational database systems, perform various database operations such as querying, inserting, updating, and deleting data, and manage the database itself, such as creating, modifying, and deleting database structures. For complex query requirements, writing SQL statements can become very complex and difficult. For example, in cases involving connections to multiple tables, subqueries, aggregate functions, etc., the writing of SQL statements can become cumbersome and difficult to understand. When processing data in different formats, SQL statements may face difficulties in data format conversion. For example, in cases where text data is converted to date format, or data is processed in different encodings, SQL statements may not be directly implemented. When there are a large number of complex data relationships and structures in the database, writing SQL query statements can become difficult. For large databases and complex data models, writing and maintaining SQL queries may require more skills and experience.

[0003] In order to solve the shortcomings of the high difficulty in writing SQL statements and the high threshold for users, attempts are made to combine natural language understanding in natural language processing (NLP) technology with query processing and optimization technology in the database field to achieve the conversion of natural language queries to SQL queries and lower the usage threshold.

[0004] Text2SQL (text to SQL) is an artificial intelligence technology and a subtask in the field of natural language processing-semantic parsing. It involves converting natural language text into commands of structured query language (SQL) so that users can interact with the database through natural language. This technology enables non-technical users to query and operate data in the database without understanding SQL syntax. The main purpose of Text2SQL technology is to convert natural language queries into structured query language (SQL), aiming to assist users to interact with the database in natural language, improve user experience and query efficiency. Through Text2SQL technology, users do not need to understand the cumbersome SQL syntax and database structure, but only need to describe the query requirements in natural language, and the system can convert them into accurate SQL query statements to achieve efficient database query and operation. This technology has broad application prospects in practical applications, which can assist non-technical personnel to query the database quickly and conveniently, reduce the workload of manually writing SQL statements, and improve work efficiency. At the same time, Text2SQL technology also helps to lower the threshold of database query and promote the popularization and application of database. By continuously optimizing and improving Text2SQL technology, we can achieve more accurate, efficient and intelligent conversion of natural language queries to SQL queries, providing users with a more convenient and intelligent database query experience.

[0005] The study found that the existing Text-to-SQL technology has problems in contextual learning. The existing technology has technical defects that ignore the mapping relationship between questions and queries and the trade-off between query quality and quantity. Summary of the invention

[0006] To solve the above-mentioned technical defects, the present invention proposes a query generation method based on a large language model in the Text2SQL task, and implements a query generation method that combines the Text2SQL task with contextual learning, prompt engineering and supervised fine-tuning design path based on the open source large language model (LLM) model architecture.

[0007] The present invention proposes a query generation method in a Text2SQL task based on a large language model, comprising:

[0008] Collect sentence information from multiple rounds of interactions with the user, and identify the user's conversation intention based on the learning model and context tracking, and fuse the acquired conversation intention information into a prompt;

[0009] A question analysis model is constructed, and the analysis target results including user SQL query intention recognition and entity extraction are obtained according to the prompt; the entity, condition and sorting requirements involved in the question are identified by using the question analysis model, the required SQL structure pattern is analyzed, SQL pattern recognition is realized, the mapping relationship between the question and the SQL query is obtained, the vector data table, column and query condition required for the SQL query are determined, so as to generate an SQL query statement that meets the user's needs, and the generated SQL query statement is fine-tuned in a supervised manner in the SQL query structure; this step includes constructing a pattern linking module to further enable the question analysis model to accurately understand the user's query intention and convert it into an effective SQL query structure; a query classification and decomposition module is constructed, and the query classification process includes: classifying each generated SQL query structure, supervising the detection of the table to be linked, independently generating sub-queries and merging them with the main query to achieve SQL query structure fine-tuning; the decomposition process includes decomposing the content in the vector database, and the same question is retrieved in multiple databases at the same time; secondly, irrelevant items are filtered out according to the data in the vector database by specifying keywords to accelerate the recall of content related to the user's question; a query generation module is constructed, and each query class is obtained according to the class label, and each query class uses different prompts to generate SQL queries.

[0010] In some embodiments, the user SQL query intention identification further includes the query purpose, target table and query conditions; the entity extraction further includes selecting the corresponding column and the table where the column is located from a given SQL database, and using the question analysis model to extract possible entities and values ​​in cells from the question.

[0011] In some implementations, the prompt is used to define tasks, instructions, and roles related to the user's SQL query intent to ensure that text that meets the user's needs is generated based on a large language model.

[0012] In some embodiments, in the question analysis model, the model takes a user query question in the form of natural language text as input, performs text preprocessing, word embedding, syntactic analysis, semantic analysis, and text generation, and outputs an analysis target result of the question in the form of a natural language text that is grammatically correct and matches the input semantics. The obtained analysis target result includes identifying the query purpose, target table, and query conditions proposed by the user.

[0013] In some embodiments, the use of the problem analysis model to identify the entities, conditions and sorting requirements involved in the problem, analyze the pattern of the required SQL structure, implement SQL pattern recognition, obtain the mapping relationship between the problem and the SQL query, and generate an SQL query statement that meets user needs further includes: calling the problem analysis model, identifying all column names in the problem, locating relevant tables and fields in the SQL database, selecting the corresponding column and the table where the column is located from a given SQL database, using the problem analysis model to extract possible entities and values ​​in cells from the problem, and generating more accurate SQL query statements by extracting the values ​​in the entities and cells.

[0014] In some implementations, the mode linking module performs the following steps:

[0015] The analysis results are obtained through SQL query structure analysis, and the elements related to the user's question in the vector database are identified, and irrelevant parts are filtered out to reduce noise and input complexity, and realize the processing of fine pattern links.

[0016] In some implementations, the query classification and decomposition module performs the following steps: classifying the generated SQL query structure, supervising and detecting the tables to be linked, independently generating sub-queries and merging them with the main query to achieve SQL query structure fine-tuning.

[0017] In some implementations, the query generation module is implemented using In-context learning.

[0018] In some embodiments, the mapping relationship between the question and the SQL query further includes an organizational strategy for improving token efficiency, including first masking domain-specific words in the target question and the query questions in the candidate set Q; then, ranking the candidate examples based on the Euclidean distance between the masked examples and the query embedding; at the same time, calculating the query similarity between the predicted SQL query and the query in the candidate set Q; finally, the selection criterion prioritizes the sorted candidate examples by question similarity, where the query similarity is greater than a predefined threshold, and then selects the top n queries that have good similarity in both the question and the query.

[0019] In some embodiments, the query classification is divided into one of three types of queries including simple class, non-nested complex class and nested complex class. Specifically: the simple class includes single-table queries that can be answered without linking or nesting; the non-nested complex class includes queries that require linking but no sub-queries, and queries in the nested complex class may require links, sub-queries and set operations.

[0020] Compared with the prior art, the beneficial technical effects and technical progress achieved by the present invention are as follows:

[0021] 1) Obtain the analysis target results including user SQL query intention recognition and entity extraction according to the prompt; In terms of supervised fine-tuning, supervised fine-tuning can alleviate the technical bottleneck that as the number of links in the query increases, the possibility of at least one pattern link not being correctly generated will also increase; Therefore, the present invention study demonstrates the potential of open source LLM in Text-to-SQL, emphasizes the importance of training corpus and model scale, and points out the decline of contextual learning ability after fine-tuning; In general, this study provides a comprehensive study of Text-to-SQL, emphasizing the importance of contextual learning, new prompt engineering and supervised fine-tuning, which can balance the mapping relationship between questions and queries and the quality and quantity of queries;

[0022] 2) and, speeding up retrieval by query decomposition, accelerating the recall of content related to the user's question. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flowchart of a query generation method based on a large language model in a Text2SQL task according to the present invention;

[0024] Figure 2 Detailed flowcharts; (a) detailed flowchart of step 1, (a) detailed flowchart of step 2;

[0025] Figure 3 This is a sample diagram of the target table in SQL database format;

[0026] Figure 4 for Figure 3 The final output result of the large language model of the example is generated as a flowchart. DETAILED DESCRIPTION

[0027] The technical solution of the present invention is described below in conjunction with specific embodiments and drawings.

[0028] like Figure 1 As shown, the query generation method based on accelerated multi-way recall in the Text2SQL task of the present invention specifically includes the following steps:

[0029] Step 1: gradually collect the sentence information in multiple rounds of interaction with the user, and make appropriate responses based on the learning model and context tracking. For example, use traditional machine learning models (such as SVM,) or deep learning models (such as RNN, LSTM, GRU) and a dialogue state tracker (DST) to track the dialogue state to identify the user intent of the dialogue, and fuse the acquired dialogue intent information into a prompt. Specifically, the prompt defines the three main elements of tasks, instructions, and roles related to the acquired user intent to ensure that text that meets user needs is generated based on a large language model; the user intent is specifically SQL query intent, which specifically includes the following processing:

[0030] Step 1.1, given a target table in the form of an SQL database, and using natural language processing (NLP) to build a question analysis model, the model takes a user query question in the form of a natural language text as input, performs text preprocessing, word embedding, syntactic analysis, semantic analysis, and text generation, and outputs an analysis target result of a question in the form of a natural language text that conforms to the grammar and matches the input semantics, and the obtained analysis target result includes key information such as identifying the query purpose, target table, and query conditions proposed by the user; wherein the question further includes information such as entities, condition descriptions, and sorting requirements, so as to generate a correct SQL query statement; the target table is used to present relational data, including tables and fields; the relational data (RDBMS) is composed of multiple interconnected two-dimensional tables, which are composed of rows and columns, and each table has a unique identifier (primary key); the purpose of the SQL query statement is specifically to determine which table in the target table the user uses to achieve the query purpose;

[0031] Step 1.2, call the question analysis model, identify all column names in the question, accurately locate the relevant tables and fields in the SQL database, select the corresponding column and the table where the column is located from the given SQL database, and lay the foundation for the subsequent SQL query generation; and use the question analysis model to extract possible entities and cell values ​​from the question to further improve the query conditions and limit the search scope; by extracting entities and cell values, the question analysis model can better understand the specific requirements of the user's query question, thereby generating more accurate SQL query statements; user intent recognition and entity extraction in this process are key steps for the question analysis model to understand user query intent and generate accurate SQL queries, providing important support for ensuring the accuracy and completeness of query results;

[0032] Step 2: Based on the user intent recognition and entity extraction obtained in step 1, identify the entities, conditions, and sorting requirements involved in the question, analyze the required SQL structure pattern, build a pattern linking module, generate SQL query statements, and perform supervised fine-tuning on the generated SQL query statements, including:

[0033] Step 2.1, using the problem analysis model constructed in step 1 to identify the entities, conditions, and sorting requirements involved in the problem, analyze the required SQL structure pattern, and implement SQL pattern recognition to generate SQL query statements that meet user needs; specifically, SQL statements have fixed templates and keywords, including common keywords such as SELECT and FROM, and keywords involving pattern structures such as Order BY and Group By. When processing natural language queries, the same type of questions usually have similar query structures. Therefore, first parse the user's intent based on the large language model and analyze the required SQL structure pattern; then, by analyzing the natural language query proposed by the user, use the problem analysis model to determine the required vector database tables, columns, and query conditions, so as to construct the correct SQL query statement;

[0034] Step 2.2, construct a pattern linking module to further enable the question analysis model to accurately understand the user's intention and convert it into an effective SQL query structure, that is, to obtain the analysis results through SQL query structure analysis, identify the elements related to the user's question in the vector database, and filter out irrelevant parts to reduce noise and input complexity, and achieve refined pattern linking. The large language model can better respond to different types of queries and improve the accuracy and efficiency of text-to-SQL query conversion. Therefore, the design and optimization of the pattern linking module is crucial to achieve accurate and efficient natural language to SQL query conversion. Specifically, it includes: before generating the SQL query structure, find the target table and the column name in the target table related to the user's question in advance, and then input it into the large language model (LLM) as a significantly simplified Database Schema, so as to reduce input noise and enhance SQL generation performance. It mainly uses algorithms based on similarity matching (such as: embedding model, n-gram) to select a series of content that is more relevant to the user's question from a large number of options;

[0035] Step 2.3, construct a query classification and decomposition module. The query classification process includes: classifying each SQL query structure generated by step 2.2, supervising the detection of tables to be linked, independently generating subqueries and merging them with the main query to achieve SQL query structure fine-tuning; for each pattern link, it is possible that the correct table or link condition is not detected. As the number of links in the query increases, the possibility that at least one pattern link cannot be generated correctly also increases. One way to alleviate this problem is to introduce a query classification and decomposition module to supervise the detection of tables to be linked. In addition, some queries have process components, such as unrelated subqueries, which can be generated independently and merged with the main query. To address these issues, a query classification and decomposition module is introduced, which divides each query into one of three categories of queries, including simple classes, non-nested complex classes, and nested complex classes. Specifically: the simple class includes single-table queries that can be answered without linking or nesting. Non-nested complex classes include queries that require links but no subqueries, while queries in nested complex classes may require links, subqueries, and set operations of the two. The corresponding simple classes, non-nested complex classes, and nested complex classes are respectively labeled; the decomposition process includes: decomposing the content in the vector database; searching the same question in multiple databases at the same time; secondly, specifying keywords for the data in the database to filter out irrelevant items. Accelerate the recall of content related to the user's question. For data with obvious structured characteristics (such as character relationships), the retrieval speed is accelerated by creating a knowledge graph and using the HNSW algorithm for retrieval;

[0036] In addition to the class labels, the query classification and decomposition module is used to detect the tables linked to non-nested SQL queries and nested SQL queries, as well as any subqueries that may be used in nested SQL queries. By introducing this module, complex query structures can be processed more effectively, improving the accuracy and efficiency of SQL queries;

[0037] Step 2.4, build a query generation module, obtain each query class according to the class label, and generate SQL queries for each query class using different prompts: In-context Learning infers how to complete new tasks based on example information in the context. In-context learning refers to focusing learning on a specific context or environment during the training and application of the problem analysis model to improve the model's understanding and application capabilities. This learning method can help large models better understand the context and background of input data, thereby improving their performance and effectiveness in fields such as natural language processing and computer vision. In-context learning can better understand the semantics and context of sentences or texts. This learning method enables the model to understand and generate natural language text more accurately based on contextual information, improving the quality of text understanding and generation. In general, applying In-context learning in large models can help the model better understand and apply the contextual information of the input data, thereby improving the performance and effectiveness of the model in various tasks. This learning method helps the model to understand the context and background of the data more deeply, and provides better support and performance for the application of the model in complex tasks.

[0038] In order to preserve the mapping information between questions and SQL queries and improve token efficiency, a new query organization strategy is proposed to balance the quality and quantity. Specifically, in terms of candidate query selection, the structural similarity of SQL query statements and the semantic similarity of user questions are considered, and the question and query are comprehensively considered to select candidate queries. Specifically, the domain-specific words in the target question and the query questions in the candidate set Q are first masked. Subsequently, the candidate examples are ranked according to the Euclidean distance between the masked examples and the query embedding. At the same time, the query similarity between the predicted SQL queries and the queries in the candidate set Q is calculated. Finally, the selection criterion prioritizes the ranked candidate examples by question similarity, where the query similarity is greater than a predefined threshold. In this way, the top n queries selected have good similarity in both questions and queries. This strategy aims to effectively manage example data and ensure efficiency and performance while maintaining information mapping accuracy. This query organization strategy can better balance the quality and quantity of examples and provide more effective example data support for model training and query processing.

[0039] In summary, the present invention systematically studies Text-to-SQL based on Large Language Model (LLM), including two aspects: hint engineering and supervised fine-tuning.

[0040] like Figure 3As shown in the figure, the user's question is "Who has a total performance rating of more than 85?" The entities obtained from the user's question are limited to "Performance," "Total Rating," and "85." The column names are "Age," "Total Rating." The cell values ​​are "80," "Zhang San," and "20." Fields are a general term for "entities," "column names," and "cell values." Figure 4 As shown, Figure 3 The SQL language model final output result generation process of the target table is shown.

[0041] The above is only a specific implementation method of the present invention. The above implementation steps are only used to help understand the specific methods and core ideas of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of several changes or equivalent substitutions, combinations and modifications made within the technical scope disclosed by the present invention and based on the ideas of the present invention without departing from the principles of the present invention. These changes or equivalent substitutions, combinations and modifications should be deemed to fall within the scope of protection of the present invention and should be covered by the scope of protection of the present invention.

Claims

1. A query generation method based on a large language model in a Text2SQL task, characterized in that: include: Collect sentence information from multiple rounds of interactions with the user, and identify the user's conversation intention based on the learning model and context tracking, and fuse the acquired conversation intention information into a prompt; Construct a problem analysis model, and obtain the analysis target results including user SQL query intention recognition and entity extraction according to the prompt; use the problem analysis model to identify the entities, conditions and sorting requirements involved in the problem, analyze the required SQL structure pattern, implement SQL pattern recognition, obtain the mapping relationship between the problem and the SQL query, determine the vector data table, column and query conditions required for the SQL query, so as to generate an SQL query statement that meets the user's needs, and perform supervised fine-tuning of the SQL query structure on the generated SQL query statement; this step includes constructing a pattern linking module to further enable the problem analysis model to accurately understand the user's query intention and convert it into an effective SQL query structure; Construct a query classification and decomposition module. The query classification process includes: classifying each generated SQL query structure, supervising the detection of tables to be linked, independently generating subqueries and merging them with the main query to achieve fine-tuning of the SQL query structure; the decomposition process includes decomposing the content in the vector database, and searching the same question in multiple databases at the same time; secondly, specifying keywords based on the data in the vector database to filter irrelevant items to speed up the recall of content related to the user's question; construct a query generation module, obtain each query class according to the class label, and use different prompts for each query class to generate SQL queries.

2. The query generation method based on a large language model in a Text2SQL task according to claim 1, characterized in that: The user SQL query intention recognition further includes the query purpose, target table and query conditions; the entity extraction further includes selecting the corresponding column and the table where the column is located from a given SQL database, and using the question analysis model to extract possible entities and values ​​in cells from the question.

3. The query generation method based on a large language model in a Text2SQL task according to claim 1, characterized in that: The prompt is used to define tasks, instructions, and roles related to the user's SQL query intention to ensure that text that meets user needs is generated based on a large language model.

4. The query generation method based on a large language model in a Text2SQL task according to claim 1, characterized in that: In the question analysis model, the model takes the user query question in the form of natural language text as input, performs text preprocessing, word embedding, syntactic analysis, semantic analysis, and text generation, and outputs the analysis target result of the question in the form of natural language text that conforms to the grammar and matches the input semantics. The obtained analysis target result includes identifying the query purpose, target table, and query conditions proposed by the user.

5. The query generation method based on a large language model in a Text2SQL task according to claim 1, characterized in that: The method of using the problem analysis model to identify the entities, conditions and sorting requirements involved in the problem, analyzing the pattern of the required SQL structure, realizing SQL pattern recognition, and obtaining the mapping relationship between the problem and the SQL query to generate an SQL query statement that meets user needs further includes: calling the problem analysis model, identifying all column names in the problem, locating relevant tables and fields in the SQL database, selecting the corresponding column and the table where the column is located from a given SQL database, extracting possible entities and values ​​in cells from the problem using the problem analysis model, and generating more accurate SQL query statements by extracting the values ​​in the entities and cells.

6. The query generation method based on a large language model in a Text2SQL task according to claim 1, characterized in that: The mode linking module performs the following steps: The analysis results are obtained through SQL query structure analysis, and the elements related to the user's question in the vector database are identified, and irrelevant parts are filtered out to reduce noise and input complexity, and realize the processing of fine pattern links.

7. The query generation method based on a large language model in a Text2SQL task according to claim 1, characterized in that: The query classification and decomposition module performs the following steps: classifying the generated SQL query structure, supervising the detection of tables to be linked, independently generating sub-queries and merging them with the main query to achieve SQL query structure fine-tuning.

8. The query generation method based on a large language model in a Text2SQL task according to claim 1, characterized in that: The query generation module is implemented using In-context learning.

9. The query generation method based on a large language model in a Text2SQL task according to claim 1, characterized in that: The mapping relationship between the question and the SQL query further includes an organizational strategy for improving token efficiency, including first masking domain-specific words in the target question and the query questions in the candidate set Q; then, ranking the candidate examples according to the Euclidean distance between the masked examples and the query embedding; at the same time, calculating the query similarity of the predicted SQL query and the queries in the candidate set Q; finally, the selection criterion prioritizes the ranked candidate examples by question similarity, where the query similarity is greater than a predefined threshold, and then selects the top n queries that have similarity in both the question and the query.

10. The query generation method based on a large language model in a Text2SQL task according to claim 1, characterized in that: The query classification is divided into one of three types of queries including simple class, non-nested complex class and nested complex class. Specifically: the simple class includes single-table SQL queries that can be answered without chaining or nesting; the non-nested complex class includes SQL queries that require chaining but no subqueries, and the queries in the nested complex class include those that require chaining, subqueries and set operations of both.

Images