Data synthesis method for text-to-structured query

Abstract

The invention relates to a data synthesis method for text-to-structured query. The method comprises the following steps: constructing a database structure based on Internet table data; generating a synthetic SQL query for the database structure through a first prompt word group; translating the synthesized SQL query into a natural language question with diversified semantic styles through a second prompt word group; generating a thinking chain for a triple comprising a corresponding database structure, a synthetic SQL query and a natural language problem through a third prompt word group; and outputting the triple and the corresponding thinking chain as a tetrad to obtain synthetic data. According to the method, a large amount of high-quality training data can be automatically generated, the cost and time of data annotation are remarkably reduced, and the accuracy and high quality of the generated data are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a data synthesis method for text-to-structured query. Background Art

[0002] The technology of converting natural language questions to structured queries (Text-to-SQL) aims to convert natural language questions into SQL queries that can be executed in a database, enabling non-professionals to easily retrieve the required data from the database. However, developing a powerful Text-to-SQL model requires a large amount of high-quality training data, that is, each training data is a <database, natural language question, SQL query> triple, to fine-tune a large language model (LLM).

[0003] In the current mainstream training sets, although some data synthesis methods have been proposed, they usually perform data augmentation on existing data sets, so that the diversity and quality of the synthesized data are limited. Specifically, most methods still use a small number of databases provided by the existing data sets, and the diversity of the databases has not increased; usually summarize SQL templates from the existing data sets, and then fill in tables, columns, and values, resulting in meaningless or syntactically incorrect generated SQL queries and low data quality; usually use a SQL-to-question translation model or predefined template pairs of rules to generate natural language questions, limiting the style diversity and semantic fluency of the questions; do not consider generating a chain-of-thought answer reasoning path during the synthesis process, resulting in low accuracy. Summary of the Invention

[0004] The present invention provides a data synthesis method for text-to-structured query to solve the defects of the prior art.

[0005] The present invention provides a data synthesis method for text-to-structured query, including:

[0006] S1: Construct a database structure based on Internet table data;

[0007] S2: Generate a synthetic SQL query for the database structure through a first prompt phrase;

[0008] S3: Translate the synthetic SQL query into natural language questions with diverse semantic styles through a second prompt phrase;

[0009] S4: Generate a chain of thought for a triple including the corresponding database structure, synthetic SQL query, and natural language question through a third prompt phrase;

[0010] S5: Output the triple and the corresponding chain of thought as a quadruple to obtain synthetic data.

[0011] A data synthesis method for text-to-structured query provided by the present invention, step S1 further includes:

[0012] S11: Collect Internet table data;

[0013] S12: Clean and filter the Internet table data to obtain preprocessed data;

[0014] S13: Analyze the data content in the preprocessed data through LLM, and add relevant extended columns corresponding to the business scenario to the database structure to construct the database structure.

[0015] A data synthesis method for text-to-structured query provided by the present invention, the database structure in step S13 includes:

[0016] Business scenario;

[0017] Structured information, the structured information includes relational tables, primary keys, foreign keys, and example data rows.

[0018] A data synthesis method for text-to-structured query provided by the present invention, step S2 further includes:

[0019] S21: Generate a basic SQL query through the first prompt phrase and LLM according to the information of the database structure;

[0020] S22: Filter the syntax error query and timeout error query in the basic SQL query to obtain a filtered SQL query;

[0021] S23: Extract a synthesized SQL query from the filtered SQL query, where the synthesized SQL query includes multiple combined SQL queries, and each combined SQL query includes an SQL template and a corresponding single SQL query.

[0022] A data synthesis method for text-to-structured query provided by the present invention, the types of the first prompt phrase in step S2 include:

[0023] First task instruction, database schema, advanced SQL function, database value, SQL complexity level, column quantity constraint.

[0024] A data synthesis method for text-to-structured query provided by the present invention, step S3 further includes:

[0025] S31: Generate natural language questions with diverse semantic styles for each SQL query in the synthesized SQL query through the second prompt phrase and the LLM, obtaining multiple candidate question groups corresponding to the multiple SQL queries respectively, where each candidate question group includes multiple candidate questions;

[0026] S32: For a single SQL query, filter out the optimal candidate question from the multiple candidate questions in the corresponding candidate question group, and output the single SQL query and the corresponding optimal candidate question.

[0027] According to a data synthesis method for text-to-structured query provided by the present invention, step S32 further includes:

[0028] S321: For a single candidate question group, input the multiple candidate questions included into a sentence encoding model to obtain multiple semantic vectors corresponding to the multiple candidate questions respectively;

[0029] S322: Calculate the average cosine similarity between the current candidate question and other candidate questions according to the semantic vectors;

[0030] S323: Select the candidate question corresponding to the maximum value of the average cosine similarity as the optimal candidate question.

[0031] According to a data synthesis method for text-to-structured query provided by the present invention, the types of the second prompt phrase in step S3 include:

[0032] Second task instruction, SQL query, SQL-related column information, and a language style randomly selected from multiple candidate language styles;

[0033] The candidate language styles include:

[0034] Formal style, colloquial style, imperative style, interrogative style, descriptive style, concise style, fuzzy style, metaphorical style, and conversational style.

[0035] According to a data synthesis method for text-to-structured query provided by the present invention, step S4 further includes:

[0036] S41: Generate a chain of thought for each triple through the third prompt phrase and the LLM, obtaining multiple candidate chain-of-thought groups corresponding to the multiple triples respectively, where each candidate chain-of-thought group includes multiple candidate chains of thought;

[0037] S42: For a single candidate chain-of-thought group, conduct a majority vote on the execution results of the SQL queries extracted from the multiple candidate chains of thought, and select the candidate chain of thought corresponding to the maximum value of the voting votes as the optimal chain of thought;

[0038] S43: Output a single triple and the corresponding optimal thought chain.

[0039] According to a data synthesis method for text-to-structured query provided by the present invention, the types of the third prompt phrases in step S4 include:

[0040] Third task instructions, database schema, natural language questions, and SQL query pairs.

[0041] A data synthesis method for text-to-structured query provided by the present invention realizes the full automation from database structure construction to SQL query generation, natural language question translation, and thought chain generation by combining Internet table data and large language models (LLMs), significantly reducing the need for manual intervention, greatly improving the efficiency of data synthesis, and being particularly suitable for large-scale data processing scenarios.

[0042] By using the second prompt phrases and LLMs, the present invention can generate natural language questions in various semantic styles, such as formal style, colloquial style, imperative style, etc., making the generated query data more diverse, meeting the needs of different users and application scenarios, and enhancing the practicality and adaptability of the data; during the process of generating SQL queries and natural language questions, the present invention ensures the accuracy and high quality of the generated data by filtering syntax errors, timeout errors, and screening the optimal candidate questions and thought chains, avoiding the negative impact of low-quality data on model training; in addition, by generating a thought chain for each triple, the present invention can clearly show the reasoning process from natural language questions to SQL queries, enhancing the interpretability of the model, helping users better understand and verify the generated queries, and at the same time providing strong support for model optimization. The prompt phrase design of the present invention is flexible, can be adjusted and extended according to different task requirements, supports multiple business scenarios and database structures, has strong generality and adaptability, and can meet the needs of complex queries and diverse data generation.

[0043] By automatically generating a large amount of high-quality training data, the present invention reduces the dependence on manually labeled data, significantly reducing the cost and time of data annotation, providing efficient and low-cost data support for the training of text-to-SQL models; in addition, the present invention can generate complex SQL queries, meeting the needs of complex operations such as multi-table association and nested queries in actual business, and enhancing the practical value of the generated data. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] To more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0045] Figure 1 Schematic flow diagram of a data synthesis method for text-to-structured query provided by an embodiment of the present invention;

[0046] Figure 2 Schematic flow diagram of a method for obtaining a database structure provided by an embodiment of the present invention;

[0047] Figure 3 Schematic flow diagram of a method for obtaining a synthesized SQL query provided by an embodiment of the present invention;

[0048] Figure 4 Schematic flow diagram of a method for obtaining a natural language question provided by an embodiment of the present invention;

[0049] Figure 5 Schematic flow diagram of a method for obtaining a chain of thought provided by an embodiment of the present invention. Detailed implementation manners

[0050] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention in conjunction with the drawings in the present invention. Obviously, the described embodiments are some, but not all, embodiments of the present invention, and they should not be construed as limitations to the present invention. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention. In the description of the present invention, it should be understood that the terms used are only for the purpose of description and cannot be construed as indicating or implying relative importance.

[0051] The following further illustrates the present invention with specific embodiments.

[0052] As Figure 1 shown, the present invention provides a data synthesis method for text-to-structured query, including:

[0053] S1: Construct a database structure based on Internet table data.

[0054] Step S1 aims to construct a complex and realistic database structure starting from Internet tabular data. Specifically, in Step S1, starting from Internet tabular data, the LLM is used to understand the tabular content and construct a complete database structure in the context of enterprise-level business scenarios. The generated database includes relational tables, primary keys, foreign keys, and example data rows. Subsequently, the LLM expands the database structure by adding additional relevant columns to enhance its complexity and realism, making it closer to real enterprise application scenarios while ensuring the integrity and consistency of the database.

[0055] As Figure 2 shown, Step S1 further includes:

[0056] S11: Collect Internet tabular data.

[0057] Currently, although there is a lack of large-scale, publicly available high-quality database resources, there are a large number of tabular data covering multiple fields and reflecting real scenarios of structured data storage. In the present invention, a large number of tabular data are collected from the Internet through technologies such as web crawlers. These tables exist in various forms such as HTML tables, CSV files, and Excel spreadsheets.

[0058] S12: Clean and filter the Internet tabular data to obtain preprocessed data.

[0059] In Step S12, the collected original tabular data are cleaned and filtered to remove low-quality, duplicate, or incomplete tables, and the remaining tables are standardized, mainly including removing empty tables, small tables with the number of rows less than the threshold, simple tables with the number of columns less than the threshold; removing duplicate tables; handling missing values; standardizing table headers; detecting and handling outliers; unifying data formats, etc.

[0060] S13: Analyze the data content in the preprocessed data through the LLM to construct a database structure.

[0061] Among them, the database structure in Step S13 includes: business scenarios; structured information, and the structured information includes relational tables, primary keys, foreign keys, and example data rows.

[0062] In Step S13, the large language model (LLM) is used to analyze the preprocessed tabular data, understand the semantics and relationships of the data, and construct a complete database structure. The LLM first understands the data content stored in the table, proposes an enterprise-level database business scenario, and then generates a complete database structure that meets this business scenario.

[0063] Despite the scarcity of database resources, there is a large amount of tabular data on the Internet, covering multiple fields and reflecting the real scenario of structured data storage. Inspired by this, the present invention proposes a novel method for synthesizing databases based on Internet tables, namely the method provided in step S1, which uses an LLM to understand the content of Internet tables and expand them into a database with a complete structure.

[0064] Specifically, given an Internet table, the LLM first understands the data content stored in the table and proposes an enterprise-level database business scenario, and then generates a complete database structure that meets this business scenario (Internet table - database business scenario + database structure). This structure includes all relational tables in the database and additional information such as primary key and foreign key relationships. Each relational table includes a table name, column names, column types, column descriptions, and two example data rows. Subsequently, to further increase the complexity of the synthesized database, the LLM adds more relevant data columns to each relational table (database structure - complex database structure). These newly added columns can ensure that the generated database is closer to the real enterprise-level application scenario while maintaining the integrity and consistency of the database by expanding the original database structure.

[0065] S2: Generate a synthetic SQL query for the database structure through a first prompt phrase.

[0066] Furthermore, in step S2, based on the synthesized database, the LLM generates an SQL query that meets the real analysis requirements according to the database information. After generation, the quality of the SQL query is ensured by filtering syntax errors and timeout queries, and deduplication is performed according to the SQL template to improve diversity.

[0067] Among them, the types of the first prompt phrase in step S2 include:

[0068] First task instruction, database schema, advanced SQL functions, database values, SQL complexity level, column quantity constraint.

[0069] Specifically, the prompt words for synthesizing SQL queries include the following parts: Task instruction: It instructs the LLM to generate an executable and meaningful SQL query based on the provided information; Database schema: It contains the CREATE TABLE statements of all relational tables in the database; Advanced SQL functions: Randomly select some advanced SQL functions supported by the database engine and allow the LLM to use these functions in the generated SQL query; Database values: Randomly select some values in the database to help the LLM generate meaningful predicates (i.e., WHERE conditions) in the SQL query; SQL complexity: Randomly select a complexity level from the predefined set [simple, medium, complex, highly complex]. Each level has a clear standard definition and is accompanied by an example SQL query; Column number constraint: Specify the number of columns to be returned in the generated SQL query. This value is sampled from a geometric distribution with a success probability of p = 0.6. This distribution is chosen because it naturally tends to smaller values, reflecting the characteristic that in the Text-to-SQL scenario, SQL queries usually select fewer columns.

[0070] As Figure 3 shown, step S2 further includes:

[0071] S21: Generate a basic SQL query through the first prompt phrase and the LLM according to the information of the database structure.

[0072] In step S21, a basic SQL query is generated through the first prompt phrase and the large language model (LLM) based on the information of the database structure. The first prompt phrase includes information such as task instructions, database schema, advanced SQL functions, database values, SQL complexity level, and column number constraint. These prompt words guide the LLM to generate an SQL query that matches the database structure. The generated SQL query includes simple queries (such as SELECT statements) and complex queries (such as multi-table joins, nested queries, etc.), covering a variety of query scenarios.

[0073] S22: Filter the syntax error queries and timeout error queries in the basic SQL query to obtain a filtered SQL query.

[0074] After generating the basic SQL query, there may be some queries that do not conform to the syntax rules or have low execution efficiency. In this embodiment, they are filtered by syntax errors or timeout errors. Through the filtering mechanism, these invalid or inefficient queries are eliminated, ensuring that the remaining SQL queries are syntactically correct and can be executed within a reasonable time, improving the quality of the generated data and avoiding the processing of invalid data caused by incorrect queries in subsequent steps.

[0075] S23: Obtain a synthetic SQL query by extracting from the filtered SQL query, where the synthetic SQL query includes multiple combined SQL queries, and each combined SQL query includes an SQL template and a corresponding single SQL query.

[0076] After step S22, extract the templates of all synthetic SQL queries. Specifically, obtain the templates by removing the values in the SQL queries. Subsequently, each SQL template only retains one SQL query. The SQL template is a general query structure that can adapt to different specific queries, while the single SQL query is a specific instance based on the template. This combination method makes the generated SQL queries have a certain degree of generality and can cover diverse query requirements, providing rich basic data for subsequent generation of natural language questions and thought chains.

[0077] S3: Translate the synthetic SQL query into natural language questions with diverse semantic styles through the second prompt phrase.

[0078] After synthesizing a new SQL query, step S3 aims to translate it into a natural language question with equivalent semantics. Previous studies usually only considered the semantic accuracy of the question in this step and often ignored the diversity of language styles. However, language diversity is crucial for developing robust text-to-SQL applications because real-world users often ask questions in various language styles. Therefore, both semantic accuracy and language style diversity are important when synthesizing natural language questions.

[0079] Among them, the types of the second prompt phrase in step S3 include:

[0080] The second task instruction, the SQL query, the SQL-related column information, and the language style randomly selected from multiple candidate language styles.

[0081] Furthermore, the prompt words for synthesizing natural language questions include the following components: Task instruction: Instruct the LLM to translate the given SQL query into a natural language question; SQL query: The SQL query to be translated; SQL-related column information: Includes the column names and descriptions of the columns used in the SQL query to help the LLM generate semantically accurate questions; Required language style: Randomly select one from the following nine predefined styles. Each style is accompanied by a detailed description and example questions for reference. For the formal, spoken, imperative, interrogative, descriptive, and concise styles, the LLM needs to generate stylized natural language questions. For the fuzzy and metaphorical styles, the LLM not only needs to generate natural language questions but also provide the external knowledge behind the questions. For the dialogue style, the LLM is instructed to generate natural language questions in the form of multiple-round dialogues between <user> and <assistant>.

[0082] The candidate language styles include:

[0083] Formal style, colloquial style, imperative style, interrogative style, descriptive style, concise style, vague style, metaphorical style, conversational style.

[0084] To promote language diversity in synthetic problems, the present invention defines nine language styles commonly used by real-world users: formal, colloquial, imperative, interrogative, descriptive, concise, vague, metaphorical, and conversational styles. The first six styles (formal, colloquial, imperative, interrogative, descriptive, and concise) reflect scenarios where users express their intentions clearly with slight variations in tone. In contrast, the vague and metaphorical styles represent situations where users use vague words or abstract language, requiring external knowledge to assist in interpreting their intentions. Finally, the conversational style is used to simulate scenarios where users need to clarify their intentions through multiple rounds of dialogue, which is particularly useful in practical applications because users may not express their intentions directly or completely, and the model needs to actively append questions to fully understand their needs.

[0085] As Figure 4 shown, step S3 further includes:

[0086] S31: Using the second prompt phrase and the LLM, generate natural language questions with diverse semantic styles for each SQL query in the synthetic SQL query, obtaining multiple candidate question groups corresponding to the multiple SQL queries, where each candidate question group includes multiple candidate questions.

[0087] In step S31, using the second prompt phrase and the large language model (LLM), generate diverse natural language questions for each synthetic SQL query. The LLM generates multiple candidate questions based on these prompt words to form a candidate question group. Each SQL query corresponds to a candidate question group, and the group contains multiple natural language questions with different semantic styles, thus ensuring that the generated questions are diverse and rich, covering different expressions and user needs.

[0088] S32: For a single SQL query, select the optimal candidate question from the multiple candidate questions in the corresponding candidate question group, and output the single SQL query and the corresponding optimal candidate question.

[0089] In step S32, for each SQL query, select the optimal natural language question from its corresponding candidate question group. The purpose of the selection is to choose a question that can most accurately express the semantics of the SQL query from multiple candidate questions.

[0090] Among them, step S32 further includes:

[0091] S321: For a single candidate question group, input the multiple candidate questions it contains into the sentence encoding model to obtain multiple semantic vectors corresponding to the multiple candidate questions respectively.

[0092] In step S321, first, each candidate question in the candidate question group needs to be input into the sentence encoding model (such as BERT, etc.) and converted into a semantic vector. The semantic vector is a numerical representation in a high-dimensional space that can capture the semantic information of the sentence and provide a basis for subsequent similarity calculation.

[0093] S322: Calculate the average value of the cosine similarities between the current candidate question and other candidate questions according to the semantic vectors.

[0094] Furthermore, for each candidate question, calculate the cosine similarity between its semantic vector and the semantic vectors of other candidate questions, and find the average value. The cosine similarity is used to measure the similarity degree between two semantic vectors. The closer the value is to 1, the more similar the semantics are. By calculating the average value, the overall consistency of the current candidate question with other questions in the group in terms of semantics can be evaluated.

[0095] S323: Select the candidate question corresponding to the maximum value of the average cosine similarity as the optimal candidate question.

[0096] In step S323, select the candidate question with the largest average cosine similarity as the optimal candidate question because the question with the largest average similarity is the most consistent with other questions in the group in terms of semantics, can best represent the semantics of the SQL query, and can also ensure that its language style meets the requirements. Finally, pair the optimal candidate question with the corresponding SQL query and output.

[0097] Furthermore, in step S32 and the corresponding steps S321 to S323, for each given SQL query, the LLM samples 8 candidate natural language questions. To identify the question that best matches the given SQL query, the present invention selects the most suitable question from the 8 candidate questions according to semantic consistency. Specifically, the present invention uses the Sentence Transformers to embed the 8 candidate natural language questions into semantic vectors. For each candidate question, calculate the average value of the cosine similarities with all other questions. Finally, select the question with the highest average similarity as the most representative question because it is closest to the semantic center of all candidate questions.

[0098] S4: Generate a chain of thought for the triple including the corresponding database structure, synthetic SQL query, and natural language question through the third prompt phrase.

[0099] Step-by-step chain reasoning performs excellently in solving complex tasks, including math problems at the Olympic competition level, code generation, and commonsense reasoning. By breaking down complex problems into smaller and more manageable steps, the chain of thought enables the model to systematically reason about each part of the task, thereby improving the model's accuracy and interpretability. However, due to the high cost of manual annotation, existing Text-to-SQL datasets lack detailed chains of thought as training labels, limiting the potential of Text-to-SQL models to utilize natural language intermediate reasoning steps.

[0100] To address this limitation, in addition to synthesizing <database, natural language question, SQL query> triples, the present invention proposes to further synthesize the chain-of-thought solution corresponding to each triple to clearly show how to gradually construct an SQL query from a natural language question.

[0101] Therefore, in step S4, in order to further improve the interpretability of the dataset, the LLM generates a step-by-step chain of thought for each <natural language question, SQL query> pair, showing the complete reasoning process from question understanding to SQL query construction. The chain of thought is completed by analyzing the question, identifying key information, and gradually constructing the query. In the post-processing stage, the most reliable chain of thought is selected through a majority voting mechanism.

[0102] Among them, the types of the third prompt phrases in step S4 include:

[0103] Third task instructions, database schema, natural language question, and SQL query pairs.

[0104] Furthermore, the prompt words used to synthesize the chain of thought include the following parts: Task instructions: instruct the LLM to generate a step-by-step chain-of-thought solution using the given database information, natural language question, and SQL query; Database schema: contains all CREATE TABLE statements for creating the database; Natural language question and SQL query pairs: natural language questions are paired with their corresponding SQL queries, where the SQL query is the reference answer to the natural language question.

[0105] As Figure 5 shown, step S4 further includes:

[0106] S41: Through the third prompt phrase and the LLM, generate a chain of thought for each triple to obtain multiple candidate chain-of-thought groups corresponding to the multiple triples, where each candidate chain-of-thought group includes multiple candidate chains of thought.

[0107] In step S41, the synthesis of the thought chain is first carried out. The synthesized thought chain first understands and analyzes the natural language problem to identify the key information required to generate the SQL query (such as the tables, columns, and filtering conditions required for the SQL query), and then gradually constructs the SQL query, combines the necessary joins, filters, aggregations, groupings, and other operators, and presents the complete SQL query as the final answer.

[0108] S42: For a single candidate thought chain group, perform a majority vote on the execution results of the SQL queries extracted from multiple candidate thought chains, and select the candidate thought chain corresponding to the maximum value of the vote count as the optimal thought chain.

[0109] S43: Output a single triple and the corresponding optimal thought chain.

[0110] Furthermore, in steps S42 to S43, for each candidate thought chain group, the optimal thought chain is screened through a majority voting mechanism. Specifically, first extract the generated SQL queries from each candidate thought chain, then execute these SQL queries in the database to obtain their respective execution results, then compare the execution results of all candidate thought chains, select the result that appears most frequently as the final correct result, and finally, among the candidate thought chains whose execution results are consistent with the final correct result, select the one with the most votes as the optimal thought chain, which can ensure that the screened thought chain is not only semantically correct but also generates a correct query result consistent with the majority.

[0111] S5: Output the triple and the corresponding thought chain as a quadruple to obtain synthetic data.

[0112] Through steps S1 to S4, the present invention can realize the automated synthesis of a large number of Text-to-SQL data using LLMs. Each piece of data is a complete quadruple, including a database, an SQL query, a natural language problem, and a thought chain. The synthesized dataset has four characteristics: databases from a wide range of fields; SQL queries ranging from simple to highly complex; natural language problems with diverse language styles; and reliable thought chain solutions.

[0113] To verify the feasibility of this method, the present invention initially sampled approximately 200,000 Internet tables from Tablib (tabular corpus) as seeds and synthesized 2.54 million high-quality and diverse Text-to-SQL data samples through the above process.

[0114] Using the synthesized data, the present invention trained the OmniSQL model based on the data synthesis method of the present invention on the basis of the Qwen2.5-Coder model for the Text-to-SQL task. OmniSQL has two parameter scales, 7B and 14B.

[0115] To comprehensively evaluate the Text-to-SQL ability of OmniSQL, the present invention uses 9 benchmark datasets, including Spider-dev, Spider-test, BIRD-dev, Spider2.0-SQLite, EHRSQL, ScienceBenchmark, Spider-DK, Spider-Syn, and Spider-Realistic. The baseline models for comparison include the closed-source GPT-4o, GPT-4-Turbo, GPT-4o-mini, and the open-source deepseek-coder series, Qwen2.5-Coder series, Qwen2.5 series, and Llama3.1 series and other models. The specific experimental results are shown in Table 1.

[0116] Table 1 Evaluation results of OmniSQL on 9 datasets. The evaluation metric is the execution accuracy (percentage value)

[0117]

[0118] As can be seen from Table 1, OmniSQL outperforms open-source LLMs of the same scale or even larger scale on most datasets. In addition, OmniSQL even outperforms the closed-source models GPT-4o and GPT-4-Turbo on some benchmark datasets, such as BIRD-dev and Spider-DK, demonstrating the effectiveness of the data synthesis method proposed in the present invention.

[0119] The technology of converting natural language questions to structured queries (Text-to-SQL) aims to convert natural language questions raised by users into SQL queries that can be executed in a database. To develop a powerful Text-to-SQL model, it is usually necessary to use a large amount of data to fine-tune large language models (LLMs). However, the existing mainstream training sets mainly rely on manual annotation and have the following limitations: high annotation cost; insufficient data diversity; lack of chain-of-thought information.

[0120] To solve these problems, the present invention provides a data synthesis method for text-to-structured query, and proposes a Text-to-SQL data synthesis framework based on LLM, aiming to automatically generate a large number of diverse Text-to-SQL data with chain-of-thought annotations. Each piece of data is a quadruple: <database, natural language question, SQL query, chain of thought>. The specific process is as follows: First, the present invention collects a large amount of tabular data from the Internet and performs cleaning and filtering, then expands them into a complete database structure. Subsequently, based on the synthesized database, high-quality and diverse SQL queries are generated, and these SQL queries are translated into natural language questions in various language styles. Finally, the chain-of-thought reasoning path is synthesized to better represent the step-by-step conversion process from natural language questions to SQL queries.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A data synthesis method for converting text into structured query, characterized in that: include: S1: Building database structure based on Internet table data; S2: Generate a synthetic SQL query for the database structure through the first prompt phrase; S3: translating the synthesized SQL query into a natural language question with diversified semantic styles through a second prompt phrase; S4: Generate a thought chain for a triple including a corresponding database structure, a synthetic SQL query, and a natural language question through a third prompt phrase; S5: Output the triples and the corresponding thought chains as quadruples to obtain synthetic data.

2. A data synthesis method for converting text into structured query according to claim 1, characterized in that: Step S1 further comprises: S11: Collect Internet form data; S12: Clean and filter the Internet table data to obtain pre-processed data; S13: Analyze the data content in the preprocessed data through LLM, and add relevant extended columns corresponding to the business scenario to the database structure to construct a database structure.

3. A data synthesis method for converting text into structured query according to claim 2, characterized in that: The database structure in step S13 includes: Business scenarios; The structured information includes a relational table, a primary key, a foreign key, and a sample data row.

4. The data synthesis method for converting text into structured query according to claim 1, characterized in that: Step S2 further comprises: S21: Generate a basic SQL query based on the first prompt phrase and LLM and the information of the database structure; S22: Filtering syntax error queries and timeout error queries in the basic SQL query to obtain a filtered SQL query; S23: extracting and obtaining a composite SQL query from the filtered SQL query, wherein the composite SQL query includes a plurality of combined SQL queries, each of which includes a SQL template and a corresponding single SQL query.

5. The data synthesis method for converting text into structured query according to claim 1, characterized in that: The types of the first prompt phrases in step S2 include: First task instructions, database schema, advanced SQL functions, database values, SQL complexity level, column number constraints.

6. A data synthesis method for converting text into structured query according to claim 1, characterized in that: Step S3 further comprises: S31: Generate a natural language question with diversified semantic styles for each SQL query in the synthesized SQL query through the second prompt phrase and the LLM, and obtain a plurality of candidate question groups corresponding to the plurality of SQL queries, wherein each candidate question group includes a plurality of candidate questions; S32: For a single SQL query, a plurality of candidate questions in the corresponding candidate question group are screened to obtain an optimal candidate question, and the single SQL query is output to the corresponding optimal candidate question.

7. A data synthesis method for converting text into structured query according to claim 6, characterized in that: Step S32 further includes: S321: For a single candidate question group, input the multiple candidate questions contained in the group into a sentence encoding model to obtain multiple semantic vectors corresponding to the multiple candidate questions respectively; S322: Calculate the average cosine similarity between the current candidate question and other candidate questions based on the semantic vector; S323: Select the candidate question corresponding to the maximum value of the average value of cosine similarity as the optimal candidate question.

8. A data synthesis method for converting text into structured query according to claim 1, characterized in that: The types of the second prompt phrase in step S3 include: second task instruction, SQL query, SQL related column information, and a language style randomly selected from a plurality of candidate language styles; The candidate language styles include: Formal style, colloquial style, imperative style, interrogative style, descriptive style, concise style, vague style, metaphorical style, conversational style.

9. A data synthesis method for converting text into structured query according to claim 1, characterized in that: Step S4 further comprises: S41: Generate a thought chain for each triple through the third prompt phrase and the LLM, and obtain a plurality of candidate thought chain groups corresponding to the plurality of triples, wherein each candidate thought chain group includes a plurality of candidate thought chains; S42: for a single candidate thinking chain group, a majority vote is performed on the execution results of the SQL queries extracted from multiple candidate thinking chains, and the candidate thinking chain corresponding to the maximum value of the votes is selected as the optimal thinking chain; S43: Output a single triple and the corresponding optimal thinking chain.

10. The data synthesis method for converting text into structured query according to claim 1, characterized in that: The types of the third prompt phrase in step S4 include: The third task is to instruct, database schema, natural language question and SQL query pairs.

Images