Large-scale language model enhanced table question and answer method based on thinking chain reasoning

Abstract

The invention discloses a large language model enhanced table question answering method based on thinking chain reasoning, and belongs to the technical field of natural language processing. The method comprises the steps that a form question and answer CoTSQL model based on a thinking chain and a framework extraction model are constructed, an example screening module generates an SQL framework architecture of a natural language question through the framework extraction model, and an example similar to a target question is selected by using a screening algorithm according to a framework category; mode linking is carried out in the mode processing module, key table information and mode relations are extracted, and a context environment is provided; the prompt construction module integrates the screened examples and the processed mode information into a CoTSQL style prompt, and inputs a large language model to generate a preliminary SQL query; and the SQL correction module detects and repairs illusion errors generated by the large language model through SQL repairs and a consistency strategy, and executes revised SQL query in a database to finally obtain an accurate table question and answer result. According to the method, the accuracy and robustness of SQL query are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a large-scale language model-enhanced table question answering method based on thought chain reasoning. Background Art

[0002] In recent years, large language models (LLMs) have achieved breakthroughs in natural language processing (NLP) thanks to their powerful semantic understanding and generation capabilities. In particular, in data-intensive tasks, LLMs (Large Language Models) demonstrate unique advantages in converting natural language into structured query languages ​​(such as SQL), providing a new technical path for building intelligent database interaction systems. In the field of tabular question answering, text-to-SQL semantic parsing technology serves as a bridge between natural language and database operations, and its performance directly impacts the efficiency and accuracy of human-computer collaboration. Existing research indicates that LLM-based text-to-SQL methods have gradually become a hot research topic, but in practical applications, they still face multiple challenges such as semantic alignment, logical reasoning, and knowledge illusions.

[0003] The current mainstream semantic parsing modules for table-based question answering (LAMs) based on large language models fall into two main categories: one is fine-tuning methods based on parameter updates, which enable LLMs to generate SQL by adjusting model parameters on a domain corpus; the other is prompt engineering methods based on contextual learning, which activate the model's inherent reasoning capabilities by constructing prompt templates containing task descriptions, database schemas, and examples. Compared to fine-tuning or training models from scratch, contextual learning offers significant advantages, enabling emergent capabilities to accomplish tasks without requiring large amounts of task-specific training data. Furthermore, fine-tuning or training models often consumes significant computational resources, while contextual learning has demonstrated on multiple benchmark datasets that, even with limited training samples, methods based on small-sample prompts can surpass previous state-of-the-art techniques and achieve superior performance.

[0004] Currently, LLM-based Text-to-SQL methods typically enhance their understanding of user intent by inputting similar examples. However, the choice of examples and the construction of prompts have a direct impact on the accuracy of generated SQL. Furthermore, with the widespread adoption of LLMs, how to provide them with the necessary logical composition knowledge without exceeding the input length limit of the model, and how to correct for hallucinations that may arise during the generation process, have become urgent challenges.

[0005] To enhance the logical reasoning capabilities of LLMs, researchers have proposed a Chain of Thought (CoT) prompting method. This method guides the model through a series of intermediate reasoning steps, typically starting with the instruction "Let's think step by step." This prompting method has demonstrated superior performance in complex logical tasks such as Text-to-SQL. However, how to effectively apply CoT (Chain of Thought) prompts to text-to-SQL parsing remains an unresolved problem. Summary of the Invention

[0006] In response to the above technical problems, the present invention provides a large-scale language model-enhanced table question answering method based on thought chain reasoning, which combines framework extraction, pattern linking and semantic correction to generate target SQL.

[0007] The present invention is achieved by adopting the following technical solutions:

[0008] The large-scale language model-enhanced table question answering method based on thought chain reasoning includes the following steps:

[0009] Step S1: Constructing a table question answering CoTSQL model based on a thinking chain and a framework extraction model. The table question answering CoTSQL model based on a thinking chain includes an example screening module, a pattern processing module, a prompt construction module, and a structured query language (SQL) correction module.

[0010] Step S2: The example screening module generates the SQL framework architecture of the natural language question through the framework extraction model, and uses the screening algorithm to select examples similar to the target question based on the category of the framework;

[0011] Step S3: Perform pattern linking in the pattern processing module, extract key table information and pattern relationships, provide context, and obtain processed pattern information;

[0012] Step S4: The prompt construction module integrates the filtered examples and processed pattern information into CoTSQL-style prompts, inputs them into the LLM large language model, and generates preliminary SQL queries;

[0013] Step S5: The SQL correction module detects and repairs the hallucination errors generated by the large language model through SQL repair and consistency strategies, executes the corrected SQL query in the database, and finally obtains accurate tabular question answering results.

[0014] Specifically, the construction of the framework extraction model includes:

[0015] Normalize the SQL in the data set and solve the problem according to the natural language The original SQL corresponding to the natural language question is normalized to obtain the SQL; the original SQL corresponding to the natural language question is expressed as , the normalized SQL expression is ;

[0016] Based on the normalized SQL, a framework containing only SQL keywords, operators, and slots is extracted, which is represented as .

[0017] Specifically, the example selections of the example screening module specifically include:

[0018] The framework obtained based on the framework extraction model Convert it into a discrete feature vector to represent the test example and perform example retrieval, where each entry represents the presence of a grammatical element in the query. If an element appears in the SQL query, the corresponding position is assigned a value of 1;

[0019] A classification method based on difficulty level is adopted, which divides the test example set into mutually exclusive partitions representing different categories, determines the category to which it belongs, and retrieves candidate examples in the relevant partition according to the category;

[0020] Apply the k-means clustering algorithm to the discrete vector of candidate examples and select k diverse examples from each cluster center.

[0021] Specifically, the pattern processing module includes pattern linking and relationship extraction. The pattern linking obtains the question-pattern relevance score based on tag matching and masking mechanism, and for each column appearing in the SQL query, finds the question sentence slice with the highest relevance score to form a pattern link, which is used to construct prompts based on the thought chain; the relationship extraction extracts and identifies key pattern features from the database through data-driven, including primary keys, foreign keys and one-to-many relationships.

[0022] Specifically, the calculation of the question-pattern relevance score includes:

[0023] Step A1: Connect the question and pattern into one long sequence , using a special mask marker Replace the database pattern word to get the masked sequence , put all sequences into a pre-trained language model to obtain deep contextual representations; long sequences Expressed as:

[0024] ;

[0025] Masked sequence Expressed as:

[0026] ;

[0027] in, and Respectively represent classification and sentence separation; 、 and The number of tokens representing questions, tables, and columns, respectively; represents the problem sequence, represents a sequence of pattern tables, Represents a sequence of pattern columns; Indicates the name of the mode item;

[0028] Step A2: Use the hyperbolic space model for projection representation. The hyperbolic representation is:

[0029] ;

[0030] in, represents the hyperbolic space model, Represents the sequence to be projected, that is, the sequence to be hyperbolic processed, represents the sequence after hyperbolic processing;

[0031] Step A3: Compute the problem token in hyperbolic space and mode items The correlation between each pair of markers is calculated Correlation on , and get the relationship matrix Expressed as:

[0032] ;

[0033] in, represents the long sequence before the mask after hyperbolic processing, represents the masked sequence after hyperbolic processing, Represents the field of real numbers.

[0034] Specifically, the prompt construction module uses the thought chain framework to integrate the filtered examples, the database schema information after the schema processing module, and the original user query to construct the basic context information of the CoTSQL-style prompt. The prompt content includes the question, the database schema, and the thought chain CoT. The prompt synthesis includes:

[0035] Guided by questions, clearly give the natural language questions that need to be converted;

[0036] List the table structure in the database in detail in the database schema, including table name, column name, primary key, foreign key and one-to-many relationship;

[0037] In the Thinking Chain CoT, the beginning is set in advance, and the process of pattern linking is gradually demonstrated through the template to establish a connection between natural language questions and database column names;

[0038] A preliminary SQL framework containing only keywords is given, and a complete SQL query statement is generated by combining the results of pattern linking with the preliminary SQL framework.

[0039] Specifically, the preliminary SQL query is as follows: construct each example obtained from the example screening module into a CoTSQL-style prompt, and summarize it to form an example prompt set , the database schema of the test instance Organize test questions in a compact code-like format Together they form a complete prompt input for processing by the LLM large language model to generate preliminary SQL queries.

[0040] Specifically, the SQL repair in step S5 includes table name or column name mismatch, column name ambiguity, table missing, schema hallucination, and aggregate function hallucination.

[0041] Specifically, the consistency strategy in step S5 is:

[0042] In the SQL generation module, each time the large language model is called, it is prompted to generate several SQL queries and SQL repair is performed on invalid SQL generated;

[0043] Execute each executable SQL query in the database and use a voting mechanism based on the execution results;

[0044] The first SQL query that produces a consensus execution result is selected as the output.

[0045] The beneficial effects of this invention lie in: It employs a few-shot learning strategy, combined with an SQL framework and classification similarity screening methods, to screen the most representative similar samples from a large sample set. By combining database schema information in a compact code style with thought chain-style prompts, the model fully leverages the contextual learning capabilities of a large language model to generate accurate SQL queries. Furthermore, the model incorporates an SQL correction module to automatically correct generated queries, further improving the accuracy and robustness of SQL queries. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0047] Figure 1 Schematic diagram of the flow of a large-scale language model-enhanced table question-answering method based on thought chain reasoning in an embodiment of the present invention;

[0048] Figure 2 This is a schematic diagram of SQL framework extraction in an embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of an example screening algorithm in an embodiment of the present invention;

[0050] Figure 4 Schematic diagram of pattern correlation calculation based on a mask mechanism in an embodiment of the present invention;

[0051] Figure 5 This is a schematic diagram of an example of a thinking chain in an embodiment of the present invention; DETAILED DESCRIPTION

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0053] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0054] The following is combined with Figure 1-5 , some embodiments of the present invention are described in detail. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0055] The present invention proposes a large-scale language model-enhanced table question answering method based on thought chain reasoning, which combines framework extraction, pattern linking and semantic correction to generate target SQL. Specifically, the pre-trained language model is first used to generate the SQL framework, and the alignment of the question and the database schema is achieved through pattern linking. Then, based on the generated SQL framework, the most similar examples are retrieved from the database. Due to the structured and discrete nature of the SQL language, framework-based example retrieval can better reflect semantic similarity than retrieval based on natural language questions. On this basis, through the CoT-style prompt construction, the "Let's think step by step" instruction is set, and the SQL framework and pattern linking are combined to gradually guide the LLM to generate the target SQL. Finally, the generated SQL is optimized through semantic correction to solve the possible hallucination problem, thereby improving the accuracy and reliability of the generation.

[0056] In a preferred embodiment, the large language model enhanced table question answering method based on thought chain reasoning proposed by the present invention is as follows: Figure 1 As shown, the following steps are included:

[0057] Step S1: Constructing a table question answering CoTSQL model based on a thinking chain and a framework extraction model. The table question answering CoTSQL model based on a thinking chain includes an example screening module, a pattern processing module, a prompt construction module, and an SQL correction module;

[0058] Step S2: The example screening module generates the SQL framework architecture of the natural language question through the framework extraction model, and uses the screening algorithm to select examples similar to the target question based on the category of the framework;

[0059] Step S3: Perform schema linking in the schema processing module, extract key table information and schema relationships, and provide context;

[0060] Step S4: The prompt construction module integrates the filtered examples and processed pattern information into CoTSQL-style prompts, inputs them into the LLM large language model, and generates preliminary SQL queries;

[0061] Step S5: The SQL correction module detects and repairs the hallucination errors generated by the large language model through SQL repair and consistency strategies, executes the corrected SQL query in the database, and finally obtains accurate tabular question answering results.

[0062] The functional implementation of each module of the model is described in detail below.

[0063] 1. Example Screening Module and Framework Extraction Model

[0064] (1) Frame extraction of frame extraction model

[0065] Since Text-to-SQL datasets are usually manually annotated by multiple annotators, their annotation habits vary. This results in slightly different SQL query styles in the final annotations, such as uppercase and lowercase keywords. Although different styles have no effect on the execution results, the model requires additional effort to learn and adapt to them. To reduce the learning difficulty, the SQL in the dataset must first be normalized. Table 1 gives a specific example, where Representing natural language questions, Indicates the original SQL corresponding to the natural language question, Indicates the SQL after SQL normalization processing. Indicates the final extracted SQL framework.

[0066] The SQL normalization operations are as follows:

[0067] 1. Unify SQL statement keywords to uppercase and pattern items to lowercase;

[0068] 2. Add the ASC (Ascending) keyword to the default order of the ORDER BY clause (a keyword in SQL);

[0069] Based on the normalized SQL query, we further extract its SQL skeleton. The skeleton consists only of SQL keywords and slots (used to populate tables, columns, and values). For example, for a normalized SQL query, only keywords and operators are retained, and the rest is replaced with slots. Note that the ASC keyword is not retained during this operation.

[0070] The examples shown in Table 1 show that although the original SQL query appears complex, its framework is relatively simple, and each keyword can be found in the question. For example, the "ORDER BY" in the framework can be inferred from the "sort by joining date in descending order" in the question.

[0071] Table 1 SQL framework extraction

[0072]

[0073] Through this process, the framework extraction model of the present invention offers a richer variety of SQL operator combinations than directly predicted SQL queries. This approach ensures high recall of the required SQL operators because the prediction framework, as an intermediate output, provides broader SQL operator coverage than the final output of methods based on pre-trained language models (PLMs), thereby enhancing the comprehensiveness and accuracy of detection.

[0074] The proprietary framework prediction model has been fine-tuned and trained to have the ability to identify and generate necessary combined knowledge. Compared with the natural language to SQL method based on pre-trained language models, this model has a dual advantage. Figure 2As shown in the figure, first, the framework generation process abstracts the SQL implementation details (such as specific column names, table aliases, etc.) and decomposes the complex SQL generation task into two stages: structure prediction and entity filling. This significantly reduces the complexity of model learning and enables the model to focus on higher-level structural understanding rather than being limited to specific database details. Secondly, because the model ignores instances that have the same structural framework but contain unique tags of different databases during training (for example, queries with different column names but the same WHERE (SQL instruction used to set filter conditions in queries to limit the range of returned data records) clause structure), the generated SQL framework shows richer semantic diversity. This feature effectively avoids the problem of incomplete combination pattern coverage caused by overfitting to specific database patterns in traditional end-to-end methods, thereby improving the generalization and adaptability of the model and ensuring that effective SQL query generation can be provided in a wider range of database environments. Figure 2 The diagram below shows an SQL framework extraction diagram for the natural language query "Find countries that do not broadcast cartoons produced by screenwriter M". Figure 2 In the example, the TV channel table has fields including channel ID, TV series title, country, language, and content; the TV series table has fields including episode ID, number of episodes, ratings, market share, and broadcast channel; and the cartoon table has fields including cartoon ID, title, director, screenwriter, and broadcast channel. The final extracted SQL schema is "SELECT _FROM _ EXCEPT SELECT _ FROM _ JOIN _ ON _= _ WHERE _= _ ".

[0075] (2) Example filtering of the example filtering module

[0076] In this embodiment, the goal of example screening is to select a subset from a set of examples that provides the best context for solving the test problem. The simplest approach is to randomly select examples from the dataset, but randomly selected examples may not provide sufficiently effective learning samples for large models. To address this issue, some researchers have proposed the k-NN Enhanced Example Selection method (KATE), which retrieves the k nearest neighbors from the dataset using input comparison examples. Specifically, all instances in the dataset are first converted into continuous vectors using a sentence encoder. During inference, the input of the test example is also mapped into the latent space using the same encoder and compared with the vectors in the dataset using a similarity metric (such as negative Euclidean distance or cosine similarity). Finally, the k annotated examples that are most similar to the test example are selected from the example set.

[0077] However, unlike many tasks where the output is a categorical label with little correlation with the question structure, the Text-to-SQL task requires structured predictions that contain more explicit information about the question structure than the input question provides. In addition, unlike natural language questions that can only be converted into continuous semantic vectors, SQL frameworks can be easily converted into discrete feature vectors based on their syntax, making their comparison more efficient and transparent.

[0078] Therefore, this paper proposes a new example selection strategy. Unlike the k-NN-enhanced example selection method that uses continuous-valued metrics to retrieve the most similar examples for each test instance, the example selection strategy of this paper achieves both example similarity and diversity by converting the representation of a given example into a discrete-valued vector that captures the SQL framework. It then classifies the example set by difficulty and finally retrieves similar examples within the category using the discrete vector of the SQL framework.

[0079] Specifically, the SQL frames of all examples are first converted into discrete grammatical vectors. This process directly uses the invented frame extraction model to remove the SQL frame dataset after removing tables, columns, and values. Each SQL frame is mapped into a "bag of grammatical elements" feature vector, where each entry represents the presence of a grammatical element in the query, and if an element appears in the SQL query, the corresponding position is assigned a value of 1. During the reasoning process, the frame extraction module is first used to generate the SQL frame of the question. Then, the same method is used to convert this frame into a discrete feature vector for representing the test instance and performing example retrieval.

[0080] Next, in order to obtain examples similar to the given example, the example set is divided into mutually exclusive partitions representing different categories. Specifically, a difficulty level-based classification method is adopted, which is derived from the Spider dataset (a multi-database, multi-table, single-round query Text-to-SQL dataset). Its classification is strictly based on the grammatical coverage and structure of SQL queries, ensuring that queries that meet the same conditions are classified into the same category. For example, there are four levels in total: Easy, Medium, Hard, and Extra Hard, and the grammatical coverage increases with difficulty. The SQL framework of the test example previously obtained is used to determine its category, and candidate examples in the relevant partition are retrieved based on the category.

[0081] Finally, to select diverse examples from the candidate partitions, we use the classification similarity screening method to apply the k-means clustering algorithm to the discrete vectors of examples, and select k diverse examples from each cluster center. The examples finally selected are not only similar to the test examples in terms of category, but also maintain diversity in the problem structure. These selected examples will then be used to construct prompts. The specific code implementation process of the example selection strategy (i.e., the example screening algorithm) is as follows Figure 3 As shown, the input of the example selection strategy includes: a sample data SQL frame set , SQL framework collection of test sets , the number of examples required ,Classification ; Output includes: prompt set ,in, For test examples hint. Represents the set of objects to be processed. The getCategory() function is used to retrieve categorical data. The append() function is used to add a list, and when adding a list, it is the reference address of the added list. The getSyntaxvectors() function is used to generate discrete vectors under the SQL framework, that is, to convert the SQL framework of all examples into discrete syntax vectors. The K-MeansClustering() function represents the K-Means clustering algorithm. The getNearest() function is used to obtain the nearest neighbor, and the frameExtraction() function is used to extract test cases. The buildPrompt() function is used to build multimodal input.

[0082] 2. Mode Processing Module

[0083] In this embodiment, the pattern processing module is divided into two parts: pattern linking and relationship extraction. Pattern linking uses the question-pattern relevance score obtained through token matching and a masking mechanism. For each column appearing in the SQL query, the question sentence slice with the highest relevance score is found to form a pattern link, which is then used to construct a prompt based on the thought chain.

[0084] In this embodiment, pattern linking uses masking techniques to identify semantic relevance between question words and their corresponding database patterns, e.g. Figure 4 As shown in Figure 2, specifically, there are three steps to calculate the similarity between question tokens and pattern terms:

[0085] 1. Mask: and schema (i.e., table ,List ) are connected into a long sequence, as shown below:

[0086] ;

[0087] in, and Represent classification and sentence separation respectively, 、 and The number of tokens representing the question, table, and column respectively. Then a special mask is used to mark To replace the database mode word To generate a series of sequences, as shown below:

[0088] ;

[0089] Next, all sequences are fed into a pre-trained language model to obtain deep contextual representations. Represents the representation before masking (i.e. the original length sequence), Represented as a pattern item The masked representation (i.e., the masked sequence).

[0090] 2. Representation transformation: Using standard hyperbolic space (Poincare ( ) sphere) to project the representation.

[0091] Here we use the following method to get the hyperbolic representation, denoted as :

[0092] ;

[0093] in, represents the selected hyperbolic space model, represents the sequence to be hyperbolic processed, represents the hyperbolic tangent function.

[0094] Hyperbolic space was chosen for the projected representation because it provides a suitable geometry for modeling hierarchical structures. Hyperbolic space has a property called negative curvature that allows for more efficient representation of hierarchical structures, enabling our method to capture long-term dependencies and overall sentence structure. Furthermore, hyperbolic space has greater representation power than the lower-dimensionality of Euclidean space, making it more effective for representing sentences with a semantic hierarchy.

[0095] 3. Relevance measurement: Calculating question tags in hyperbolic space and mode items Specifically, the correlation between question tokens and pattern terms is derived from the pre-trained language model based on the Poincare distance matrix. By calculating the correlation between each pair of tokens on , and get the relationship matrix as follows:

[0096] ;

[0097] in It is Mobius ( )addition, represents the representation before the mask after the hyperbolic representation, Database schema notation after hyperbolic representation Indicates when blocked. and are all hyperbolic representations, Represents the field of real numbers.

[0098] The sequence formed by connecting all tables and columns can be regarded as a representation of domain knowledge, which is mainly composed of vocabulary related to the domain or scenario. A method based on masking specific tags in the database pattern sequence is used to detect the relevance of these words to domain knowledge. The core concept is to evaluate the importance of a tag by blocking it and observing the impact of this operation on the vector representation of the entire sequence. Specifically, if the vector representation of the sequence changes significantly after blocking a tag, and the magnitude of the change exceeds a preset threshold, it can be considered that the tag plays an important role in the semantic expression of the sequence. This indicates that the tag is closely related to the domain knowledge conveyed by the sequence.

[0099] The matrix based on the tag matching And the matrix obtained based on the mask mechanism Combined, the final question-pattern relevance score is calculated, which measures the probability that the pattern item will be used to form an SQL query. The final relevance score matrix as follows:

[0100] ;

[0101] in, represents the field of real numbers, The relative influence of the two strategies is determined by the pre-configured coefficient. Finally, the threshold , filtered to a series of tables ;matrix Indicates the problem pattern matching similarity:

[0102] ;

[0103] in, Indicates the problem sequence from to A subsequence of labels, Indicates the name of the mode item, Represents a collection of column values.

[0104] In this embodiment, the design goal of relationship extraction is to extract and identify key pattern features from the database in a data-driven manner. These features include table structure information, primary keys, foreign keys, and one-to-many relationships, which provide rich contextual support for subsequent processing modules. These extracted key information not only helps to understand the structure and data distribution of the database, but also provides a solid foundation for the SQL generation module. By accurately identifying the required pattern components, the SQL generation module can improve accuracy when processing complex queries, especially when involving aggregation operations and complex connections. The relationship extraction module integrates existing knowledge and advanced technical methods in the database management system, and through in-depth analysis of the database structure, it lays a solid foundation for generating high-quality, clearly structured SQL query statements. In this way, the module can achieve more accurate and efficient query generation in the process of converting text to SQL, thereby improving the quality and performance of SQL queries.

[0105] Primary Key: A primary key uniquely identifies each record in a database table and is typically composed of a single column or a combination of multiple columns to ensure record uniqueness. Primary keys play a crucial role in ensuring data integrity and consistency, especially when performing aggregation operations. During the text-to-SQL conversion process, accurately identifying and effectively utilizing primary key information is crucial for generating high-quality SQL queries. Primary keys are often a core element when constructing aggregate queries (such as SUM, AVG, COUNT, MIN, and MAX functions), especially when used in conjunction with the GROUP BY clause. In other words, providing primary key information in advance during SQL generation significantly improves the accuracy of text-to-SQL conversion, especially when processing queries involving aggregation operations, ensuring the correctness and efficiency of query results.

[0106] Foreign Key: A foreign key in a database is a field or combination of fields that references the primary key of another table, thereby establishing an important association between the two tables. This association is crucial for maintaining referential integrity and achieving consistency in cross-table queries. Understanding the role of foreign keys is particularly critical for writing SQL queries involving multi-table joins. For example, if you need to associate with through the [Student ID] field, where [Student ID] in the student table exists as the primary key and in the course selection table it is used as a foreign key. This primary-foreign key relationship clearly defines the logical connection between the tables, making the foreign key a key hub when performing JOIN operations. During the SQL generation process, providing foreign key information can significantly improve the generation tool's ability to handle complex multi-table relationships and ensure the accuracy of query conversion, especially when dealing with complex queries that require multi-table joins.

[0107] One-to-many relationship: A one-to-many (1:N) relationship in a database means that a single record in one table is associated with multiple records in another table. For example, in one example, each student (the "one" side) may take multiple courses, and this course information is stored in (the "many" side). While primary keys and foreign keys explicitly define the connection between tables, 1:N relationships are often implicit, yet they are crucial to understanding database structure and constructing effective SQL queries. In text-to-SQL translation, accurately identifying these relationships is particularly important for executing aggregation queries involving sums, averages, or counts. Identifying 1:N relationships can help the SQL generation module make reasonable column selections in aggregation operations and GROUP BY clauses. Methods for identifying these relationships include:

[0108] 1. Data value analysis: By carefully examining the actual data, it can be found that there may be a 1:N relationship between columns with unique values ​​and those with duplicate corresponding entries.

[0109] 2. Machine Learning Methods: Machine learning algorithms can classify or cluster records in a table, revealing hidden data relationships and helping identify 1:N associations. By combining these methods, the pattern processing module can obtain a filtered set of database patterns and effectively identify complex relationship networks within the database through relationship extraction, significantly improving the accuracy and logic of SQL query translation.

[0110] 3. Prompt Construction Module

[0111] The Thought Chain Prompt Construction Module is the core component of the model. It constructs the basic context for prompts by integrating filtered examples, database schema information from the Schema Processing Module, and the original user query. This module uniquely encapsulates these elements using the Thought Chain framework and passes the combined prompts to the larger language model to generate corresponding SQL predictions.

[0112] Common methods for Table Question and Answer to provide database schemas usually adopt three prompt styles: one is code-centric prompting, which strictly follows the database schema elements and constructs database prompts through SQL CREATE statements; one is natural language style prompting, which uses natural language to describe the columns in the table and provides primary key and foreign key relationships, tending to be more flexible but less code-oriented format; and the other is to express the SQL schema in a compact code-like format.

[0113] Through comparative experiments with different prompt structures, the present invention proposes a prompt style of CoTSQL, such as Figure 5 This style allows you to express SQL patterns in a compact, code-like format while still showing the chain of thoughts that generate the SQL step-by-step in an easy-to-understand, conversational way.

[0114] Specifically, the prompt content is mainly divided into three parts: Question, Database Schema, and Chain of Thought (CoT). First, guided by "Question", clearly give the natural language question that needs to be converted. Then, in the "Database Schema" section, list the table structure in the database in detail, including table name, column name, primary key (PrimaryKey), foreign key (ForeignKey) and one-to-many relationship (One-to-Many) identified by the schema processing module. Finally, in the "CoT" section, it starts with "Let's think step by step" and uses the template "According to [NLQ], [table.column] is allowed" (NLQ represents natural language query) to gradually demonstrate the process of schema linking and establish a connection between natural language questions and database column names. Subsequently, a preliminary SQL framework containing only keywords is given. Finally, combining the results of schema linking and the preliminary SQL framework, a complete SQL query statement is generated, such as Figure 5 As shown, Figure 5 In the query, the question is "Count the number of singers participating in each concert and list the concert names." The database schema includes: 1) a venue table, whose fields (i.e., database column names) include: venue ID (primary key), location, venue name, capacity, maximum attendance, minimum attendance, and average attendance; 2) an artist table, whose fields include: artist ID (primary key), artist name, nationality, representative songs, song release year, age, and gender; 3) a concert table, whose fields include: concert ID (primary key), concert name, theme, venue ID (foreign key), and year of concert; and 4) an artist participation table, whose fields include: concert ID (primary key / foreign key), artist ID (foreign key). The concert ID in the artist participation table forms a one-to-many relationship with the artist ID in the artist participation table. This means that one concert (concert ID) can correspond to multiple artists (artist IDs), but an artist can typically only participate in one concert at a time.

[0115] During the chain of thought (CoT) analysis, the step-by-step thinking process (let's think about it step by step) is: first determine the concert noun field, that is, based on [concert table.concert name]; then determine the singer statistics field, based on [singer participation table.singer ID]; finally, generate the corresponding SQL framework "SELECT _ COUNT(_)AS _ FROM _ JOIN _ ON _=_ GROUP BY_".

[0116] Next, the SQL framework is combined with the database schema. The resulting SQL statement is: SELECT T1.Concert Name, COUNT(T2.Singer ID) AS Singer Count FROM concert table T1, JOIN singer participation table T2 ON T1.Concert ID = T2.Concert ID GROUP BY T1.Concert ID; that is, select T1.Concert Name from the concert table (T1), COUNT(T2.Singer ID) as the singer count, to achieve the query purpose of querying the name of each concert and the corresponding number of participating singers.

[0117] Each example obtained from the example filtering module will be constructed into a CoTSQL-style prompt and aggregated to form an example prompt set . Also, test the database schema of the instance It will also be organized in a compact code-like format, culminating in a test question Together they form a complete prompt input for processing by the large language model to generate preliminary SQL queries.

[0118] 4. SQL Correction Module

[0119] Existing large language models commonly suffer from hallucinations, which can easily lead to invalid or erroneous SQL queries during the text-to-SQL conversion process. To address this issue, this paper conducts an in-depth analysis of the output of large language models, systematically classifies common errors, and proposes corresponding correction algorithms for each type of error to effectively correct deviations in the generation process. Furthermore, this paper introduces a consistent execution strategy to ensure that the resulting SQL query is a valid target query.

[0120] (1) SQL correction

[0121] In this embodiment, errors can be mainly divided into five categories: table name / column name mismatch, column name ambiguity, table missing, pattern illusion, and aggregate function illusion. For each type of error, the present invention designs a heuristic algorithm to correct SQL queries that will fail during execution, specifically: Table name / column name mismatch: The large language model performs reasoning based on statistical patterns, resulting in incorrect alignment of columns and tables. For example, the column name "name" belongs to the table "movie table", so "T2.name" is an error (here T2 is an alias for the TV channel information table). This type of error can be corrected by mapping the column to the correct table and adjusting the table identifier accordingly.

[0122] Column name ambiguity: If multiple tables contain columns with the same name, this may cause invalid SQL. Randomly assign the column to one of the possible tables to ensure its uniqueness.

[0123] Table missing: For example, the "Join Date" column belongs to the "Employees" table, but is missing in the SQL. This can be corrected by adding the table to the FROM clause based on the primary and foreign key relationships.

[0124] Schema hallucination: Large language models may generate SQL that references non-existent tables or columns. For example, the column "Course ID" does not exist in any table. This can be addressed by identifying the column and replacing it with a column that has the smallest string edit distance.

[0125] Aggregate function illusion: Aggregate functions in standard SQL can only accept a single column name or expression as an argument. To solve this error, you can split the COUNT function into two independent counts and retain the DISTINCT keyword for both columns.

[0126] In our implementation, we attempt to correct unexecutable SQL statements up to five times to avoid an infinite loop. Hallucination errors are an unavoidable challenge for large language models. Categorizing hallucination errors can help fix these errors.

[0127] (2) Consistency strategy

[0128] In this embodiment, an execution consistency strategy is integrated into the model, achieved by increasing the cost of output tokens. Specifically, in the SQL generation module, each time the large language model is called, it is prompted to generate several SQL queries. Invalid SQL generated is corrected using a SQL correction algorithm. Each executable SQL query is then executed in the database, and a voting mechanism is used based on the execution results. Ultimately, the first SQL query that produces a consensus execution result is selected as the output.

[0129] For example, SQL queries are first categorized based on their execution results. The results across categories are then compared, and the category containing the most valid SQL queries is selected. The first SQL query from that category is then chosen as the final output. This approach leverages the collective knowledge generated by multiple paths, improving the reliability and flexibility of the model's SQL query generation, resulting in more robust results.

[0130] The present invention also provides a comparative analysis of the proposed table question answering CoTSQL model based on thought chain and the existing model. The experimental results are shown in Table 2:

[0131] Table 2 Experimental results of the model on the Spider dataset

[0132]

[0133] Table 2 shows that on the Spider dataset, the proposed Tabular Question Answering (CoTSQL) model based on thought chaining shows significant performance improvements over the task-related large language model versions GPT-3.5-Turbo and CodexDaVinci. Compared to the ChatGPT-SQL benchmark, which also uses GPT-3.5-Turbo but does not employ the CoT prompting style, execution accuracy increased by 12.1 percentage points. Compared to the ACT-SQL model, which also uses CoT prompting, execution accuracy increased by 1.8 percentage points, demonstrating the effectiveness of the model. Furthermore, compared to the DIN-SQL model, which requires four API (application) calls to generate each SQL statement, CoTSQL is more cost-effective, requiring only a single LLM API call.

[0134] In the large language model-based table question answering task, although contextual learning methods can directly use LLM to generate SQL code, the generated results are highly dependent on the quality of prompts and are easily affected by hallucinations. To this end, this paper proposes an LLM-enhanced table question answering model based on thought chain reasoning, aiming to improve the reliability of SQL generation through example screening, step-by-step reasoning, and SQL correction strategies. Specifically:

[0135] (1) Propose a classification similarity example screening algorithm: In the example pool, high-quality examples are screened based on SQL framework structure and semantic similarity, ensuring that LLM can draw on more relevant SQL query patterns when prompting learning;

[0136] (2) Constructing reasoning prompts based on CoTSQL: Using the thought chain reasoning method and the framework-statement two-stage SQL generation idea, guide LLM to gradually derive SQL queries and improve reasoning capabilities in complex query scenarios;

[0137] (3) Design SQL correction strategy: Combine SQL syntax correction and consistency detection to automatically correct the SQL generated by LLM to reduce hallucination problems and improve query accuracy.

[0138] This paper proposes a table question answering model CoTSQL based on a large language model for table question answering tasks. The model adopts a few-sample learning strategy, combined with an SQL framework and a classification similarity screening method, to screen out the most representative similar samples from a large number of samples. By combining database schema information in a compact code style with prompts in a thought chain style, the model can fully utilize the contextual learning capabilities of the large language model to generate accurate SQL queries. At the same time, the model also introduces an SQL correction module to automatically correct the generated queries, thereby further improving the accuracy and robustness of SQL queries. Experimental results show that the CoTSQL model exhibits excellent performance, verifying the effectiveness of the model in table question answering tasks.

[0139] For the sake of simplicity, the aforementioned embodiments are described as a series of actions. However, those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are preferred embodiments, and the actions involved are not necessarily required by this application.

[0140] The above embodiments describe the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Without departing from the spirit and scope of the present invention, modifications and variations made by those skilled in the art without departing from the spirit and scope of the present invention should be within the scope of protection of the appended claims.

Claims

1. A large-scale language model-enhanced table question answering method based on thought chain reasoning, characterized by: The following steps are involved: Step S1: Constructing a table question answering CoTSQL model based on a thinking chain and a framework extraction model. The table question answering CoTSQL model based on a thinking chain includes an example screening module, a pattern processing module, a prompt construction module, and a structured query language (SQL) correction module. Step S2: The example screening module generates the SQL framework architecture of the natural language question through the framework extraction model, and uses the screening algorithm to select examples similar to the target question based on the category of the framework; Step S3: Perform pattern linking in the pattern processing module, extract key table information and pattern relationships, provide context, and obtain processed pattern information; Step S4: The prompt construction module integrates the filtered examples and processed pattern information into CoTSQL-style prompts, inputs the large language model, and generates preliminary SQL queries; Step S5: The SQL correction module detects and repairs the hallucination errors generated by the large language model through SQL repair and consistency strategies, and then executes the corrected SQL query in the database to finally obtain accurate tabular question and answer results.

2. The large-scale language model-enhanced table question answering method based on thought chain reasoning according to claim 1 is characterized in that: Step S1: The construction of the framework extraction model includes: Normalize the SQL in the data set and solve the problem according to the natural language The original SQL corresponding to the natural language question is normalized to obtain the SQL; the original SQL corresponding to the natural language question is expressed as , the normalized SQL expression is ; Based on the normalized SQL, a framework containing only SQL keywords, operators, and slots is extracted, which is represented as .

3. The large-scale language model-enhanced table question answering method based on thought chain reasoning according to claim 2 is characterized in that: The example selection of the example screening module in step S2 specifically includes: The framework obtained based on the framework extraction model Convert it into a discrete feature vector to represent the test example and perform example retrieval, where each entry indicates the presence of a syntax element in the query. If a syntax element appears in the SQL query, the corresponding position is assigned a value of 1; A classification method based on difficulty level is adopted, which divides the test example set into mutually exclusive partitions representing different categories, determines the category to which it belongs, and retrieves candidate examples in the relevant partition according to the category; Apply the k-means clustering algorithm to the discrete vector of candidate examples and select k diverse examples from each cluster center.

4. The large-scale language model-enhanced table question answering method based on thought chain reasoning according to claim 3 is characterized in that: Step S3: The pattern processing module includes pattern linking and relationship extraction. Pattern linking obtains question-pattern relevance scores based on token matching and masking mechanisms. For each column appearing in the SQL query, the question sentence slice with the highest relevance score is found to form a pattern link, which is used to construct a prompt based on the thought chain. Relationship extraction extracts and identifies key schema features from databases through data-driven analysis, including primary keys, foreign keys, and one-to-many relationships.

5. The large-scale language model-enhanced table question answering method based on thought chain reasoning according to claim 4 is characterized in that: The calculation of the question-pattern relevance score includes: Step A1: Connect the question and pattern into one long sequence , using a preset mask mark Replace the database pattern word to get the masked sequence , and then put all the sequences into a pre-trained language model to obtain deep contextual representations; Long sequence Expressed as: ； Masked sequence Expressed as: ； in, and Respectively represent classification and sentence separation; 、 and The number of tokens representing questions, tables, and columns, respectively; represents the problem sequence, represents a sequence of pattern tables, Represents a sequence of pattern columns; Indicates the name of the mode item; Step A2: Use the hyperbolic space model to perform projection representation and obtain the hyperbolic representation: ； in, represents the sequence to be hyperbolic processed, represents the sequence after hyperbolic processing; Step A3: Compute the problem token in hyperbolic space and mode items The correlation between each pair of markers is calculated Correlation on , and get the relationship matrix Expressed as: ； in, represents the long sequence before the mask after hyperbolic processing, represents the masked sequence after hyperbolic processing, represents the field of real numbers.

6. The large-scale language model-enhanced table question answering method based on thought chain reasoning according to claim 5 is characterized in that: In step S4, the prompt construction module uses the thought chain framework to integrate the filtered examples, the database schema information after the schema processing module, and the original user query to construct the basic context information of the CoTSQL style prompt. The prompt content includes the question, the database schema, and the thought chain CoT. The prompt synthesis includes: Guided by questions, give natural language questions that need to be converted; List the table structure in the database in the database schema, including table name, column name, primary key, foreign key and one-to-many relationship; In the Thinking Chain CoT, the beginning is set in advance, and the process of pattern linking is gradually demonstrated through the template to establish a connection between natural language questions and database column names; A preliminary SQL framework containing only keywords is given, and a complete SQL query statement is generated by combining the results of pattern linking with the preliminary SQL framework.

7. The large-scale language model-enhanced table question answering method based on thought chain reasoning according to claim 6 is characterized in that: Generate preliminary SQL queries by constructing each example obtained from the example screening module into a CoTSQL-style prompt and aggregating them into a set of example prompts , the database schema of the test instance Organize in code-like format and test questions Together they form a complete prompt input for the large language model to process and generate preliminary SQL queries.

8. The large-scale language model-enhanced table question answering method based on thought chain reasoning according to claim 7 is characterized in that: The SQL repair in step S5 includes table name or column name mismatch, column name ambiguity, table missing, schema hallucination, and aggregate function hallucination.

9. The large-scale language model-enhanced table question answering method based on thought chain reasoning according to claim 8, characterized in that: The consistency strategy in step S5 is specifically: In the SQL generation module, each time the large language model is called, it is prompted to generate multiple SQL queries and SQL repair is performed on invalid SQL generated; Execute each executable SQL query in the database and use a voting mechanism based on the execution results; The first SQL query that produces a consensus execution result is selected as the output.

Images