Large language model enhanced table question answering method based on chain-of-thought reasoning

Abstract

The application discloses a large language model enhancement table question answering method based on thought chain reasoning and belongs to the technical field of natural language processing.The application comprises the following steps: constructing a table question answering CoTSQL model and a framework extraction model based on a thought chain, screening examples by the framework extraction model, generating a SQL framework architecture of a natural language question, and selecting examples similar to a target question according to a framework category by using a screening algorithm; performing mode linking in a mode processing module, extracting key table information and mode relationships, and providing a context environment; integrating the screened examples and the processed mode information into a CoTSQL style prompt by a prompt construction module, inputting the CoTSQL style prompt into a large language model to generate a preliminary SQL query; and performing detection and repair on hallucination errors generated by the large language model by a SQL correction module through a SQL repair and consistency strategy, executing the corrected SQL query in a database, and finally obtaining an accurate table question answering result.The application effectively improves the accuracy and robustness of SQL query.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly relates to a large language model enhancement table question answering method based on thought chain reasoning. BACKGROUND

[0002] In recent years, large language models (LLMs) have made breakthroughs in natural language processing due to their strong semantic understanding and generation capabilities. In particular, in data-intensive tasks, LLMs exhibit unique advantages in converting natural language into structured query languages (such as SQL), which provides a new technical path for building intelligent database interaction systems. In the field of table question answering, text-to-SQL semantic parsing technology serves as a bridge connecting natural language and database operations, and its performance directly affects the efficiency and accuracy of human-machine collaboration. Existing research shows that text-to-SQL methods based on LLMs have gradually become a hot research direction, but in practical applications, they still face multiple challenges such as semantic alignment, logical reasoning, and knowledge hallucination.

[0003] Current mainstream large language model-based table question answering semantic parsing modules are mainly divided into two categories: one is the fine-tuning method based on parameter updating, which adjusts model parameters on a domain corpus to enable LLMs to generate SQL; the other is the prompt engineering method based on context learning, which constructs a prompt template containing task descriptions, database schemas, and examples to activate the intrinsic reasoning ability of the model. Compared with fine-tuning or training models from scratch, context learning has significant advantages, as it does not require a large amount of task-specific training data and can complete tasks through emergent capabilities. In addition, fine-tuning or training models often consume a large amount of computing resources, while context learning has been proven on multiple benchmark datasets that small-sample prompt-based methods can outperform previous state-of-the-art techniques and achieve excellent performance, even under limited training samples.

[0004] Currently, text-to-SQL methods based on LLMs usually enhance their understanding of user intent by inputting similar examples. However, the selection of examples and the construction of prompts have a direct impact on the accuracy of generated SQL. In addition, with the widespread application of LLMs, how to provide necessary logical composition knowledge without exceeding the model input length limit, and how to correct hallucination problems that may occur during generation, have become challenges that need to be addressed.

[0005] Meanwhile, to enhance the logical reasoning ability of LLMs, researchers proposed the Chain of Thought (CoT) prompting method, which guides the model to reason step by step through a series of intermediate reasoning steps, usually starting with the instruction "Let's think step by step." This prompting method has shown superior performance in complex logical tasks such as Text-to-SQL. However, how to effectively apply CoT prompting to text-to-SQL parsing is still an unsolved problem. SUMMARY

[0006] To solve the above technical problems, the present application provides a large language model enhanced table question answering method based on chain of thought reasoning, which combines framework extraction, pattern linking and semantic correction to generate target SQL.

[0007] The present application is implemented by using the following technical solutions:

[0008] The large language model enhanced table question answering method based on chain of thought reasoning includes the following steps:

[0009] Step S1: Construct a table question answering CoTSQL model based on chain of thought and a framework extraction model, the table question answering CoTSQL model based on chain of thought includes an example screening module, a pattern processing module, a prompt construction module and a structured query language SQL correction module;

[0010] Step S2: The example screening module generates the SQL framework architecture of the natural language question through the framework extraction model, and selects similar examples to the target question according to the category of the framework using a screening algorithm;

[0011] Step S3: In the pattern processing module, extract key table information and pattern relationships, provide a context environment, and obtain processed pattern information;

[0012] Step S4: The prompt construction module integrates the selected examples and processed pattern information into CoTSQL style prompts, inputs the LLM large language model, and generates a preliminary SQL query;

[0013] Step S5: The SQL correction module detects and repairs hallucination errors generated by the large language model through SQL repair and consistency strategies, executes the corrected SQL query in the database, and finally obtains accurate table question answering results.

[0014] Specifically, the construction of the framework extraction model includes:

[0015] Standardize the SQL in the data set according to the natural language question and the original SQL corresponding to the natural language question is normalized to obtain a normalized SQL; wherein the original SQL corresponding to the natural language question is represented as , and the normalized SQL is represented as ;

[0016] Based on the normalized SQL, a framework containing only SQL keywords, operators and slots is extracted, represented as .

[0017] Specifically, the example selection of the example screening module specifically includes:

[0018] The framework obtained based on the framework extraction model is converted into a discrete feature vector, which represents a test example and performs example retrieval, wherein each entry represents the existence of a certain syntax element in the query, and if a certain element appears in the SQL query, the corresponding position is assigned a value of 1.

[0019] A classification method based on difficulty level is used to divide the test example set into mutually exclusive partitions representing different categories, determine its category, and retrieve candidate examples in the relevant partition according to the category.

[0020] The k-means clustering algorithm is applied to the discrete vector of the candidate example, and k diversified examples are selected from each cluster center.

[0021] Specifically, the schema processing module includes schema linking and relationship extraction. The schema linking obtains a question-schema relevance score based on token matching and masking mechanism, and for each column appearing in the SQL query, finds the question sentence slice with the highest relevance score to form a schema link, which is used to construct a thinking chain-based prompt. The relationship extraction extracts and identifies key schema features from the database through data-driven, including primary keys, foreign keys and one-to-many relationships.

[0022] Specifically, the calculation of the question-schema relevance score includes:

[0023] Step A1: Connect the question and schema into a long sequence , replace the schema words of the database with a special mask token to obtain a masked sequence , put all the sequences into a pre-trained language model to obtain a deep context representation; the long sequence is represented as:

[0024] ;

[0025] The masked sequence is represented as:

[0026] ​ ;

[0027] wherein, and denote classification and sentence separation, respectively; , and denote the number of tokens for question, table and column, respectively; denotes a question sequence, denotes a schema table sequence, denotes a schema column sequence; denotes a schema item name;

[0028] Step A2: projection representation using hyperbolic space model, the hyperbolic representation is:

[0029] ;

[0030] wherein, denotes a hyperbolic space model, denotes a sequence to be projected, i.e., a sequence to be hyperbolically processed, denotes a sequence after hyperbolic processing;

[0031] Step A3: calculate the correlation between question tokens and schema items in the hyperbolic space, calculate the correlation on each pair of tokens , get the relationship matrix denoted as:

[0032] ;

[0033] wherein, denotes a long sequence before masking after hyperbolic processing, denotes a sequence after masking after hyperbolic processing, denotes a real number field.

[0034] Specifically, the prompt construction module adopts a thinking chain framework, integrates the selected examples, the database schema information processed by the schema processing module and the original user query, constructs the basic context information of the CoTSQL style prompt, and the prompt content includes question, database schema and thinking chain CoT. The synthesis of the prompt includes:

[0035] Guided by the question, the natural language question to be converted is clearly given;

[0036] The table structure in the database is listed in detail in the database schema, including table name, column name, primary key, foreign key and one-to-many relationship;

[0037] In the CoT, the beginning is preset, and the process of gradually showing the mode link is displayed through the template, and the natural language question is associated with the database column name;

[0038] A preliminary SQL framework containing only keywords is given, and a complete SQL query statement is generated by combining the results of the mode link and the preliminary SQL framework.

[0039] Specifically, the preliminary SQL query is specifically: constructing each example obtained from the example filtering module into a CoT SQL style prompt, and summarizing to form an example prompt set , test the database mode of the instance Organized in a compact code style, test the question Together constitute a complete prompt input, which is processed by the LLM large language model to generate a preliminary SQL query.

[0040] Specifically, the SQL repair in step S5 includes table name or column name mismatch, column name ambiguity, table missing, mode illusion and aggregation function illusion.

[0041] Specifically, the consistency strategy in step S5 is specifically:

[0042] In the SQL generation module, each time the large language model is called, it is prompted to generate multiple SQL queries, and SQL repair is performed on the generated invalid SQL;

[0043] Execute each executable SQL query in the database, and use a voting mechanism based on the execution results;

[0044] The first SQL query that produces a consensus execution result is selected as the output.

[0045] The beneficial effects of the present application are: the present application adopts a few-shot learning strategy, combines a SQL framework and a classification similarity filtering method, and filters the most representative similar samples from a large number of samples. By combining the database mode information in a compact code style with the prompts in a CoT style, the model can fully utilize the context learning ability of the large language model to generate accurate SQL queries. At the same time, the model also introduces a SQL correction module to automatically correct the generated queries, thereby further improving the accuracy and robustness of the SQL queries. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to the structures shown in these drawings without creative labor.

[0047] Figure 1 A flowchart of a large language model enhanced table question answering method based on thought chain reasoning in an embodiment of the present application is shown in Figure 1.

[0048] Figure 2 A diagram of SQL framework extraction in an embodiment of the present application is shown in Figure 2.

[0049] Figure 3 A diagram of an example screening algorithm in an embodiment of the present application is shown in Figure 3.

[0050] Figure 4 A diagram of pattern relevance calculation based on a mask mechanism in an embodiment of the present application is shown in Figure 4.

[0051] Figure 5 A diagram of a thought chain example in an embodiment of the present application is shown in Figure 5. DETAILED DESCRIPTION

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings of the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0053] It should be noted that similar reference numerals and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it need not be further defined and explained in subsequent drawings.

[0054] The following will be described in detail in connection with the drawings Figures 1-5 Some embodiments of the present application will be described in detail. The following embodiments and features in the embodiments can be combined with each other without conflict.

[0055] The present application proposes a large language model enhanced table question answering method based on thought chain reasoning, which generates a target SQL by combining framework extraction, pattern linking, and semantic correction. Specifically, first, a pre-trained language model is used to generate a SQL framework, and pattern linking is used to align the question and the database schema. Next, based on the generated SQL framework, the most similar examples are retrieved from the database. Due to the structured and discrete nature of SQL language, example retrieval based on the framework can better reflect semantic similarity than retrieval based on natural language questions. On this basis, through CoT style prompt construction, the "let's think step by step" instruction is set, and the SQL framework and pattern linking are combined to gradually guide the LLM to generate the target SQL. Finally, the generated SQL is optimized through semantic correction to solve possible illusion problems, thereby improving the accuracy and reliability of the generation.

[0056] In a preferred embodiment, the large language model based on thought chain reasoning enhances the table question answering method, as shown in Figure 1 The method comprises the following steps:

[0057] Step S1: Constructing a thought chain based table question answering CoTSQL model and a frame extraction model, the thought chain based table question answering CoTSQL model comprising an example screening module, a pattern processing module, a prompt construction module and an SQL correction module;

[0058] Step S2: The example screening module generates an SQL framework architecture of the natural language question through the frame extraction model, and selects examples similar to the target question according to the category of the framework using a screening algorithm;

[0059] Step S3: In the pattern processing module, the pattern is linked, the key table information and pattern relationship are extracted, and the context environment is provided;

[0060] Step S4: The prompt construction module integrates the screened examples and the processed pattern information into a CoTSQL style prompt, inputs the LLM large language model, and generates a preliminary SQL query;

[0061] Step S5: The SQL correction module detects and repairs hallucination errors generated by the large language model through SQL repair and consistency strategies, executes the corrected SQL query in the database, and finally obtains an accurate table question answering result.

[0062] The functions of each module of the model are described in detail below.

[0063] I. Example screening module and frame extraction model

[0064] (1) Framework extraction of the frame extraction model

[0065] Since the Text-to-SQL dataset is usually annotated manually by multiple annotators, their annotation habits are different, which leads to slight differences in the final annotated SQL query style, such as capitalization and lowercase. Although different styles have no effect on the execution result, the model needs additional effort to learn and adapt to them. In order to reduce the learning difficulty, first, the SQL in the dataset needs to be standardized, and Table 1 gives a specific example, wherein, represents the natural language question, represents the original SQL corresponding to the natural language question, represents the SQL after standardization, represents the final extracted SQL framework.

[0066] The SQL normalization operation is as follows:

[0067] 1. Unify SQL statement keywords to uppercase and schema items to lowercase;

[0068] 2. Add the ASC (Ascending) keyword to the default order of the ORDER BY (SQL keyword) clause;

[0069] On the basis of normalizing the SQL query, the SQL framework is further extracted. The framework only contains the keywords and slots (used to fill in tables, columns, and values) of SQL. For example, for a normalized SQL query, only the keywords and operators are retained, and the rest are replaced with slots. It should be noted that the ASC keyword is not retained during operation.

[0070] The examples shown in Table 1 show that although the original SQL query looks complex, the framework is relatively simple, and each keyword can be inferred from the problem. For example, the "ORDER BY" in the framework can be inferred from "sort in descending order of the date of employment" in the problem.

[0071] Table 1 SQL framework extraction

[0072]

[0073] Through the above processing, the framework extraction model of the present application is more diverse in the combination of SQL operators compared to directly predicting the SQL query. This method ensures a high recall rate of the required SQL operators, as the predicted framework serves as an intermediate output, providing a wider coverage of SQL operators compared to the final output of the pre-trained language model (PLM) method, thereby enhancing the comprehensiveness and accuracy of detection.

[0074] The proprietary framework prediction model, through fine-tuning training, has the ability to recognize and generate the necessary combination knowledge, and compared to the natural language to SQL method based on the pre-trained language model, this model exhibits a double advantage. For example, Figure 2As shown, first, the framework generation process decouples the complex SQL generation task into two stages, structure prediction and entity filling, by abstracting SQL implementation details (e.g., specific column names, table aliases, etc.), significantly reducing the complexity of model learning and enabling the model to focus on higher-level structure understanding rather than being limited to specific database details. Second, since the model ignores instances with the same structure framework but different database-specific tokens during training (e.g., queries with different column names but the same WHERE clause structure), the generated SQL framework exhibits more semantic diversity. This feature effectively avoids the problem of incomplete pattern coverage in traditional end-to-end methods due to overfitting to specific database patterns, thereby improving the model's generalization ability and adaptability, ensuring effective SQL query generation in a wider range of database environments. Figure 2 An example of SQL framework extraction for the input natural language query "Find out the countries that do not play animated films produced by the scriptwriter M" is shown in FIG. 2A, Figure 2 In FIG. 2B, the fields of the TV channel table include channel id, TV series name, country, language, and content; the fields of the TV episode table include episode id, episode number, rating, market share, and broadcast channel; and the fields of the animated film table include animated film id, title, director, scriptwriter, and broadcast channel. The final extracted SQL framework is "SELECT _ FROM _ EXCEPT SELECT _ FROM _ JOIN _ ON _=_WHERE _=_ ".

[0075] (2) Example filtering of the example filtering module

[0076] In this embodiment, the purpose of example filtering is to select a subset from a set of examples to provide the best context for solving the test problem. The simplest method is to randomly select examples from the dataset, but randomly selected examples may not provide a sufficiently effective learning sample for large models. To solve this problem, some researchers have proposed a k-NN enhanced example selection method (KATE), which retrieves k nearest neighbors from the dataset by comparing input examples. Specifically, all instances in the dataset are first converted to continuous vectors by a sentence encoder. During inference, the input of the test instance is also mapped to the latent space by the same encoder and compared with the vectors in the dataset by similarity metrics such as negative Euclidean distance or cosine similarity. Finally, the k most similar annotated examples are selected from the example set.

[0077] However, unlike many tasks where the output is a categorical label and the association with the problem structure is less, the Text-to-SQL task requires structured prediction, which contains more explicit information about the problem structure than what the input question provides. Moreover, unlike natural language questions that can only be converted into continuous semantic vectors, SQL queries can be easily converted into discrete feature vectors based on their syntax, making their comparison more efficient and transparent.

[0078] Therefore, the present application proposes a new example selection strategy. Unlike the k-NN enhanced example selection method which uses continuous value metrics to retrieve the most similar examples for each test instance, the example selection strategy of the present application converts the representation of a given example into a discrete value vector that captures the SQL framework, then classifies the example set by difficulty, and finally retrieves similar examples by SQL framework discrete vectors within the category to achieve the similarity and diversity of examples.

[0079] Specifically, first, the SQL framework of all examples is converted into a discrete syntax vector. This process directly uses the invented framework extraction model to remove the SQL framework dataset after the table, column and value. Each SQL framework is mapped to a "bag of syntax elements" feature vector, where each entry represents the presence of a certain syntax element in the query. If a certain element appears in the SQL query, the corresponding position is assigned a value of 1. In the reasoning process, first, the SQL framework of the question is generated using the framework extraction module. Then, the same method is used to convert the framework into a discrete feature vector for representing the test instance and performing example retrieval.

[0080] Next, in order to obtain examples similar to the given example, the example set is divided into mutually exclusive partitions representing different categories. Specifically, a classification method based on difficulty level is used, which is derived from the Spider dataset (a multi-database, multi-table, single-round query Text-to-SQL dataset), and the classification is strictly based on the syntax coverage and structure of the SQL query, ensuring that queries that meet the same conditions are classified into the same category. For example, there are four levels in total: Easy, Medium, Hard and Extra Hard, and the number of syntax coverage increases with difficulty. Using the previously obtained SQL framework of the test example, determine its category, and retrieve candidate examples in the relevant partition according to the category.

[0081] Finally, in order to select examples with diversity from the candidate partitions, a classification similarity screening method is used to apply the k-means clustering algorithm to the discrete vectors of examples, and k diverse examples are selected from each cluster center. The finally selected examples are not only similar to the test examples in category, but also maintain diversity in question structure. These selected examples will be used to build prompts subsequently. The specific code implementation process of the example selection strategy (i.e., the example screening algorithm) is as shown in Figure 3 , wherein the inputs of the example selection strategy include: example data SQL framework set , SQL framework set of the test set , required number of examples , category ; and the outputs include: prompt set , wherein, is the test example prompt. represents the object set to be processed, the function getCategory( ) is used to retrieve the category data; the function append( ) is used to add a list, and when adding a list, the reference address of the list is added; the function getSyntaxvectors( ) is used to generate discrete vectors under the SQL framework, i.e., to convert all examples of the SQL framework into discrete syntax vectors; the function K-MeansClustering( ) represents the K-Means clustering algorithm; the function getNearest( ) is used to obtain the nearest neighbor, the function frameExtraction( ) is used to extract the test example; and the function buildPrompt( ) is used to build a multi-modal input.

[0082] II. Pattern processing module

[0083] In the embodiment, the pattern processing module is divided into two parts: pattern linking and relation extraction. The pattern linking uses the question-pattern relevance score based on token matching and the mask mechanism to find the question sentence slice with the highest relevance score for each column appearing in the SQL query to form the pattern linking, which is then used to construct the prompt based on the thinking chain.

[0084] In the embodiment, the pattern linking uses the mask technology to identify the semantic relevance between the problem word and its corresponding database pattern, as shown in Figure 4 , specifically, there are three steps to calculate the similarity between the problem token and the pattern item:

[0085] 1. Masking: connect the problem and the pattern (i.e., table , column ) into a long sequence, as shown below:

[0086] ;

[0087] wherein, and denote classification and sentence separation, respectively, , and denote the number of tokens for question, table, and column, respectively. Then a special mask token is used to replace the schema words of the database to generate a series of sequences, as shown in the following formula:

[0088] ;

[0089] Next, all sequences are put into a pre-trained language model to obtain deep contextual representations. Here, we use to denote the representation before masking (i.e., the original long sequence), and to denote the representation after the schema tokens are masked.

[0090] 2. Representation transformation: Project the representation using the unique model of the standard hyperbolic space (Poincare ball).

[0091] Here, we use the following method to obtain the hyperbolic representation, denoted as :

[0092] ;

[0093] wherein, denotes the chosen hyperbolic space model, denotes the sequence to be hyperbolically processed, denotes the hyperbolic tangent function.

[0094] The reason for choosing hyperbolic space to project the representation is that hyperbolic space provides a suitable geometric shape for hierarchical structure modeling. Hyperbolic space has a feature called negative curvature, which can more effectively represent hierarchical structures, enabling the method of the present application to capture long-term dependencies and overall sentence structure. In addition, hyperbolic space has stronger representation ability than the low-dimensional space of Euclidean space, so it can more effectively represent sentences with semantic hierarchy.

[0095] 3. Correlation measurement: Calculate the correlation between the question tokens and the schema tokens in the hyperbolic space. Specifically, the correlation between the question tokens and the schema tokens is derived from the pre-trained language model based on the Poincare distance matrix. By calculating on each pair of tokens , a relationship matrix As follows:

[0096] ;

[0097] wherein is the Möbius ) addition, denotes the representation of the mask before hyperbolic representation, denotes the representation of the database schema mark after hyperbolic representation when being shielded. and are both hyperbolic representations, denotes the real number field.

[0098] The sequence formed by connecting all the tables and columns can be regarded as the representation of domain knowledge, which is mainly composed of vocabulary related to the domain or scenario. The method of matching specific marks in the sequence of the database schema is adopted to detect the relevance of these vocabularies and domain knowledge, and the core idea is to evaluate the importance of a certain mark by shielding it and observing the impact on the entire sequence vector representation. Specifically, if the vector representation of the sequence changes significantly after shielding a certain mark, and the change amplitude exceeds the preset threshold, it can be considered that the mark has an important role in the semantic expression of the sequence. This indicates that the mark is closely related to the domain knowledge conveyed by the sequence.

[0099] The matrix based on mark matching and the matrix based on the mask mechanism are combined to calculate the final problem-schema relevance score, which measures the probability that the schema item will be used to compose the SQL query, and the final relevance score matrix As follows:

[0100] ;

[0101] wherein, denotes the real number field, determines the relative influence of the two strategies, that is, the pre-configured coefficient. Finally, a threshold can be set to filter a series of tables ; the matrix represents the problem schema matching similarity:

[0102] ;

[0103] wherein, denotes the subsequence of the marks from the th to the th in the problem sequence, denotes the schema item name, A set of column values is represented.

[0104] In this embodiment, the design goal of relationship extraction is to extract and identify key schema features from the database in a data-driven manner. These features include table structure information, primary keys, foreign keys, and one-to-many relationships, which provide rich contextual support for subsequent processing modules. These extracted key information not only helps to understand the structure and data distribution of the database, but also provides a solid foundation for the SQL generation module. By accurately identifying the required schema components, the SQL generation module can improve accuracy when processing complex queries, especially when involving aggregation operations and complex joins. This relationship extraction module combines existing knowledge in database management systems and advanced technical methods, laying a solid foundation for generating high-quality, well-structured SQL query statements through in-depth analysis of database structure. In this way, the module can achieve more accurate and efficient query generation in the text-to-SQL conversion process, thereby improving the quality and performance of SQL queries.

[0105] Primary Key: The primary key in a database table is an element that uniquely identifies each record, usually composed of a single column or a combination of multiple columns, used to ensure the uniqueness of records. Primary keys play a crucial role in ensuring data integrity and consistency, especially when performing aggregation operations. In the text-to-SQL conversion process, accurately identifying and effectively utilizing primary key information is key to generating high-quality SQL queries. Primary keys are often the core elements when constructing aggregation queries (such as SUM, AVG, COUNT, MIN, MAX, etc. functions), especially when combined with the GROUPBY clause. In other words, if primary key information can be provided in advance during SQL generation, it will significantly improve the accuracy of text-to-SQL conversion, especially when dealing with queries involving aggregation operations, which can better ensure the correctness and efficiency of query results.

[0106] Foreign Key: A foreign key in a database is a field or field combination that references the primary key of another table, establishing an important association between two tables. This association is crucial for maintaining referential integrity and ensuring the coherence of cross-table queries. Understanding the role of foreign keys is particularly important for writing SQL queries involving multiple table joins. For example, if you need to associate a student table with a course table through the [Student ID] field, where [Student ID] is the primary key in the student table and used as a foreign key in the course table. This primary-foreign key relationship clearly defines the logical connection between tables, making foreign keys a key hub for executing JOIN operations. Providing foreign key information during SQL generation can significantly improve the ability of the generation tool to handle complex multi-table relationships, ensuring the accuracy of query conversion, especially when dealing with complex queries that require multiple table joins.

[0107] One-to-many relationships: One-to-many (1:N) relationships in a database represent a single record in one table being associated with multiple records in another table. For example, in one example, each student (the "one" side) may take multiple courses, which are stored in a table (the "many" side). While primary and foreign keys explicitly define the connections between tables, 1:N relationships are often implicit, yet they are crucial for understanding the database structure and constructing effective SQL queries. In text-to-SQL translation, accurately identifying these relationships is particularly important for executing aggregate queries involving summation, averaging, or counting. Identifying 1:N relationships can help the SQL generation module make reasonable column selections in aggregation operations and GROUP BY clauses. Methods for identifying these relationships include:

[0108] 1. Data value analysis: By carefully examining the actual data, one can find columns with unique values and those with repeated corresponding entries, which may indicate a 1:N relationship between columns.

[0109] 2. Machine learning methods: Machine learning algorithms can classify or cluster records in tables, revealing hidden data relationships and helping to identify 1:N associations. By combining these methods, the schema processing module can obtain a filtered set of database schemas, and effectively identify complex relationship networks in the database, significantly improving the accuracy and logic of SQL query translation.

[0110] III. Hint construction module

[0111] The thought chain hint construction module is the core component of the model, which integrates the filtered examples, database schema information processed by the schema processing module, and the original user query to build the basic context information of the hint. This module uses the thought chain framework to uniquely encapsulate these elements and passes the merged hint to the large language model to generate the corresponding SQL prediction.

[0112] Common methods for table question answering to provide database schema usually adopt three types of hints: one is code-centric hints, which strictly follow the database schema elements and build database hints through SQL CREATE statements; one is natural language style hints, which use natural language to describe the columns in the table and provide primary and foreign key relationships, favoring a more flexible but less code-oriented format; and one is a compact code-style format to represent SQL schema.

[0113] Through comparative experiments with different hint structures, the present invention proposes the hint style of CoTSQL, as shown in Figure 5 This style can represent SQL schema in a compact code-style format while still showing the thought chain process of generating SQL step by step in an easy-to-understand, conversational manner.

[0114] Specifically, the prompt content is mainly divided into three parts: Question, Schema, and CoT. First, the natural language question to be converted is explicitly given as the guide of Question. Then, in the Schema part, the table structure in the database is listed in detail, including table name, column name, PrimaryKey, ForeignKey, and One-to-Many relationship identified by the schema processing module. Finally, in the CoT part, Let's think step by step is used as the beginning, and through the template of According to [NLQ], allow the use of [table.column] (NLQ represents natural language query), the process of schema linking is gradually shown to establish the connection between natural language questions and database column names. Subsequently, a preliminary SQL framework containing only keywords is given. Finally, combined with the results of schema linking and the preliminary SQL framework, a complete SQL query statement is generated, as shown in Figure 5 . Figure 5 In the example shown in

[0115] In the CoT analysis, the step-by-step thinking process (Let's think step by step) is as follows: first, determine the concert noun field, i.e., according to [Concert Table.Concert Name] to determine; then determine the singer statistics field, according to [Singer Participation Table.Singer ID] to determine; finally, generate the corresponding SQL framework "SELECT _ COUNT(_)AS _ FROM _ JOIN _ ON _=_ GROUP BY_ ".

[0116] Next, the SQL framework is combined with the database schema. The resulting SQL statement is: SELECT T1.Concert Name, COUNT(T2.Singer ID) AS Singer Count FROM concert table T1, JOIN singer participation table T2 ON T1.Concert ID = T2.Concert ID GROUP BY T1.Concert ID; that is, select T1.Concert Name from the concert table (T1), COUNT(T2.Singer ID) as the singer count, to achieve the query purpose of querying the name of each concert and the corresponding number of participating singers.

[0117] Each example obtained from the example filtering module will be constructed into a CoTSQL-style prompt and aggregated to form an example prompt set . Also, test the database schema of the instance It will also be organized in a compact code-like format, culminating in a test question Together they form a complete prompt input for processing by the large language model to generate preliminary SQL queries.

[0118] 4. SQL Correction Module

[0119] Existing large language models commonly suffer from hallucinations, which can easily lead to invalid or erroneous SQL queries during the text-to-SQL conversion process. To address this issue, this paper conducts an in-depth analysis of the output of large language models, systematically classifies common errors, and proposes corresponding correction algorithms for each type of error to effectively correct deviations in the generation process. Furthermore, this paper introduces a consistent execution strategy to ensure that the resulting SQL query is a valid target query.

[0120] (1) SQL correction

[0121] In this embodiment, errors can be mainly divided into five categories: table name / column name mismatch, column name ambiguity, table missing, pattern illusion, and aggregate function illusion. For each type of error, the present invention designs a heuristic algorithm to correct SQL queries that will fail during execution, specifically: Table name / column name mismatch: The large language model performs reasoning based on statistical patterns, resulting in incorrect alignment of columns and tables. For example, the column name "name" belongs to the table "movie table", so "T2.name" is an error (here T2 is an alias for the TV channel information table). This type of error can be corrected by mapping the column to the correct table and adjusting the table identifier accordingly.

[0122] Column name ambiguity: If multiple tables contain columns with the same name, this may cause invalid SQL. Randomly assign the column to one of the possible tables to ensure its uniqueness.

[0123] Table absence: The column "Hire Date" belongs to the table "Employees Table", but is absent in SQL. This can be fixed by adding the table to the FROM clause based on the primary-foreign key relationship.

[0124] Schema hallucination: Large language models can generate SQL that references tables or columns that do not exist. For example, the column "Course ID" does not exist in any table. This can be addressed by identifying the column and replacing it with a column that has the smallest string edit distance to it.

[0125] Aggregation function hallucination: Aggregation functions in standard SQL can only accept a single column name or expression as a parameter. To fix this error, the COUNT function can be split into two separate counts, and the DISTINCT (de-duplication) keyword is kept for both columns.

[0126] In implementation, at most five attempts are made to correct the unexecutable SQL to avoid falling into a dead loop. The hallucination problem of large language models is an unavoidable challenge. Categorizing hallucination problems helps to fix these errors.

[0127] (2) Consistency strategy

[0128] In this embodiment, the consistency strategy is integrated into the model by increasing the cost of output labels. Specifically, in the SQL generation module, the large language model is prompted to generate several SQL queries each time it is called. For the generated invalid SQL, the SQL correction algorithm is applied for correction. Subsequently, each executable SQL query is executed in the database, and a voting mechanism is adopted based on the execution results. Finally, the first SQL query that produces a consensus execution result is selected as the output.

[0129] For example, first, the SQL queries are classified according to their execution results. Then, the results of each category are compared, and the category that contains the most valid SQL queries is selected, and the first SQL query in that category is selected as the final output. This method improves the reliability and flexibility of the model in generating SQL queries by utilizing the collective knowledge generated by multiple paths, resulting in more robust results.

[0130] The present application also provides a comparative analysis of the proposed table question and answer CoTSQL model based on thought chain and existing models, and the experimental results are shown in Table 2:

[0131] Table 2 Experimental results of the model on the Spider dataset

[0132]

[0133] As can be seen from Table 2, on the Spider dataset, the table question answering model (CoTSQL) based on the thought chain proposed by the application has certain performance improvement compared with the task-related use of the large language model version GPT-3.5-Turbo and CodexDavinci model. Compared with the ChatGPT-SQL benchmark also using GPT-3.5-Turbo but not using the CoT prompt style, the execution accuracy is improved by 12.1 percentage points. Compared with the ACT-SQL model also using the CoT prompt, the execution accuracy is improved by 1.8 percentage points, proving the effectiveness of the model. At the same time, compared with the DIN-SQL model which needs to call the API (application) 4 times for each generated SQL statement, the CoTSQL has lower cost and uses one API call of LLM.

[0134] In the table question answering task based on the large language model, although the context learning method can directly use the LLM to generate the SQL code, the generation result is highly dependent on the prompt quality and is easily affected by the illusion problem. Therefore, the application proposes an LLM enhanced table question answering model based on thought chain reasoning, which aims to improve the reliability of SQL generation through example screening, step-by-step reasoning and SQL correction strategy. Specifically:

[0135] (1) A classification similarity example screening algorithm is proposed: in the example pool, high-quality examples are screened according to the SQL framework structure and semantic similarity, ensuring that the LLM can learn from more relevant SQL query patterns when prompted;

[0136] (2) Construct a reasoning prompt based on CoTSQL: adopt the thought chain reasoning method, and follow the idea of two-stage SQL generation of framework-sentence, guide the LLM to gradually deduce the SQL query, and improve the reasoning ability in complex query scenarios;

[0137] (3) Design a SQL correction strategy: combine SQL syntax correction and consistency detection to automatically correct the SQL generated by the LLM, so as to reduce the illusion problem and improve the query accuracy.

[0138] The application proposes a table question answering model CoTSQL based on a large language model for a table question answering task. The model adopts a few-shot learning strategy, combines a SQL framework and a classification similarity screening method, and screens the most representative similar samples from a large number of samples. By combining the database schema information in the compact code style with the prompt in the thinking chain style, the model can fully utilize the context learning ability of the large language model to generate accurate SQL queries. Meanwhile, the model also introduces a SQL correction module to automatically correct the generated queries, thereby further improving the accuracy and robustness of the SQL queries. Experimental results show that the CoTSQL model exhibits excellent performance, verifying the effectiveness of the model in the table question answering task.

[0139] For the foregoing embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that the application is not limited by the order of the described actions, because according to the application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily required by the application.

[0140] In the above embodiments, the basic principles and main features of the application and the advantages of the application are described. Those skilled in the art should understand that the application is not limited by the above embodiments, and the above embodiments and the description in the specification are only to illustrate the principles of the application. Any modification and change made by those skilled in the art without departing from the spirit and scope of the application should be within the protection scope of the claims of the application.

Claims

1. A method for enhancing table question answering based on a large language model based on thought chain reasoning, characterized in that, The method comprises the following steps: Step S1: constructing a table question and answer CoTSQL model based on a thought chain and a framework extraction model, the table question and answer CoTSQL model based on the thought chain comprising an example screening module, a schema processing module, a prompt construction module and a structured query language (SQL) correction module; Step S2: the example screening module generates an SQL framework architecture of the natural language question through the framework extraction model, and selects examples similar to the target question according to the category of the framework using a screening algorithm; Step S3: mode linking is performed in the schema processing module, key table information and mode relationships are extracted, a context environment is provided, and processed schema information is obtained; Step S4: the prompt construction module integrates the screened examples and the processed schema information into a CoTSQL style prompt, inputs the prompt into a large language model, and generates a preliminary SQL query; Step S5: the SQL correction module detects and corrects hallucination errors generated by the large language model through SQL repair and consistency strategies, executes the corrected SQL query in a database, and finally obtains an accurate table question and answer result; In step S3, the schema processing module comprises mode linking and relationship extraction, the mode linking obtains a question-schema relevance score based on token matching and mask mechanism, and for each column appearing in the SQL query, the mode linking finds the question sentence slice with the highest relevance score to constitute a prompt based on the thought chain; the relationship extraction extracts and identifies key mode features from the database, including primary keys, foreign keys and one-to-many relationships; The calculation of the question-schema relevance score comprises: Step A1: Connect questions and patterns into a long sequence , mark using a pre-set mask Replace the pattern words in the database with masked sequences , and put all the sequences into a pre-trained language model to obtain deep context representation; Long sequence is represented as: ； masked sequence is represented as: ； wherein, and denote classification and sentence separation, respectively; , and denote the number of tokens for question, table, and column, respectively; denotes a question sequence, denotes a schema table sequence, denotes a schema column sequence; denotes a schema item name; Step A2: projection representation is performed using a hyperbolic space model to obtain hyperbolic representation: ； wherein, represents a sequence to be subjected to hyperbolic processing, represents a sequence after hyperbolic processing; Step A3: Compute problem tags in hyperbolic space and pattern items Compute the correlation between each pair of tags on the correlation matrix , resulting in a relation matrix is represented as: ； wherein represents the long sequence before masking after hyperbolic processing, represents the sequence after masking after hyperbolic processing, represents the real number field.

2. The large language model augmented table question answering method based on thought chain reasoning of claim 1, wherein, The construction of the framework extraction model in step S1 comprises: The SQL in the data set is normalized to obtain normalized SQL according to the natural language question and the original SQL corresponding to the natural language question ; wherein the original SQL corresponding to the natural language question is expressed as , and the normalized SQL is expressed as . Based on the normalized SQL, a framework containing only the keywords, operators and slots of the SQL is extracted, denoted as . 3.The method of claim 2, wherein the method is based on a chain-of-thought reasoning large language model augmented table question answering method, and the method comprises: The example selection of the example screening module in step S2 specifically comprises: frames obtained based on the framework extraction model converted into a discrete feature vector representing the test example and performing example retrieval, where each entry represents the presence of a certain syntactic element in the query, and if a certain syntactic element is present in the SQL query, the corresponding position is assigned a value of 1; A classification method based on difficulty levels is adopted, the test example set is divided into mutually exclusive partitions representing different categories, the category to which the test example set belongs is determined, and candidate examples in the relevant partition are retrieved according to the category; The k-means clustering algorithm is applied to the discrete vector of the candidate examples, and k diversified examples are selected from each cluster center.

4. The large language model augmented table question answering method based on thought chain reasoning of claim 1, wherein, The prompt construction module in step S4 adopts a thought chain framework to integrate the screened examples, the database schema information processed by the schema processing module and the original user query, to construct basic context information of the CoTSQL style prompt, the prompt content comprising a question, a database schema and a thought chain CoT, and the synthesis of the prompt comprising: The question is used as a guide to give a natural language question that needs to be converted; The table structure in the database schema is listed, including table name, column name, primary key, foreign key and one-to-many relationship; In the thought chain CoT, the beginning is preset, the mode linking process is gradually displayed through a template, the natural language question is associated with the database column name, and a preliminary SQL framework containing only keywords is given. The preliminary SQL framework is combined with the results of the mode linking to generate a complete SQL query statement. 5.The method of claim 4, wherein the method is based on a chain-of-thought reasoning large language model augmented table question answering method, and the method comprises: The generating of the preliminary SQL query specifically involves: constructing each example obtained from the example filtering module into a CoT SQL style prompt, and aggregating to form an example prompt set , database schema of the test instance Organized in code format, and test questions Together constitute a complete prompt input for large language model processing, generating a preliminary SQL query.

6. The large language model augmented table question answering method based on thought chain reasoning of claim 5, wherein, In step S5, the SQL repair comprises table name or column name mismatch, column name ambiguity, table absence, mode hallucination and aggregation function hallucination.

7. The large language model augmented table question answering method based on chain-of-thought reasoning of claim 6, wherein, The consistency policy in step S5 is specifically: In the SQL generation module, each time the large language model is called, it is prompted to generate multiple SQL queries, and SQL repair is performed on the generated invalid SQL; Execute each executable SQL query in the database and adopt a voting mechanism based on the execution results; Select the first SQL query that produces a consensus execution result as the output.

Images