A Domain-Adaptive Zero-Sample Table Retrieval Method
By training a query converter and designing a table retrieval tool, and using SQL templates to generate natural language text query statements, the semantic matching problem in zero-shot table retrieval is solved, achieving efficient table retrieval and making it suitable for zero-shot table retrieval tasks in complex domains.
Patent Information
- Application Number
- CN202410029914.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-09
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2044-01-09
AI Technical Summary
Existing zero-shot table retrieval methods are ineffective in handling synonyms, ambiguous words, and contextual relevance issues, and directly using dense vector retrieval methods is not ideal for tabular data, thus failing to effectively complete zero-shot table retrieval tasks.
By acquiring the WikiSQL dataset, SQUALL template data, and NQTables dataset, a query converter was trained to generate natural language text query statements. Training data was constructed by combining SQL templates and a table corpus. A table retrieval tool and its loss function were designed. The BERT model and FAISS tool were used to calculate relevance scores and perform retrieval.
It can effectively solve zero-sample table retrieval tasks without relying on labeled data, understand table structure information, improve retrieval efficiency, save time and money, and is suitable for professional and complex fields such as medicine and aviation.
Smart Images

Figure CN117851567B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of information retrieval technology and relates to a domain-adaptive zero-sample table retrieval method. Background Technology
[0002] The rapid development of the Internet age has led to an explosive growth in information and data. It has become crucial to quickly and effectively find valuable information from such a massive amount of data. To meet this need, information retrieval technology has emerged and been widely used.
[0003] Traditional information retrieval methods based on lexical retrieval, such as BM25, only consider the degree of word matching, neglecting semantic relevance and struggling to handle synonyms, ambiguous words, and contextual relevance. Dense vector retrieval methods, however, overcome these limitations by leveraging the semantic understanding capabilities of pre-trained deep learning language models, significantly improving information retrieval performance. Specifically, these methods use pre-trained language models to encode the input user query and target article into dense vectors, respectively. The relevance between the user query and the target article is determined by calculating the similarity between the two vectors. Currently, model training primarily relies on contrastive learning techniques, with the training objective being to maximize the relevance scores between positive examples (in fact, relevant queries and articles) and minimize the relevance scores between negative examples (in fact, irrelevant queries and articles).
[0004] Dense vector retrieval methods typically require large-scale labeled data for model training; however, constructing such labeled datasets is time-consuming and expensive, especially in domain-specific areas requiring extensive expertise. Therefore, zero-shot retrieval has become a crucial topic in information retrieval. Zero-shot retrieval refers to training a model without a labeled dataset (i.e., relevant query-article pairs), expecting the model to retrieve relevant articles from the target corpus based on text queries in the test data. Research shows that retrieval models trained directly on the source dataset (not the target dataset) suffer significant performance loss on the target dataset in zero-shot retrieval, even performing worse than the traditional vocabulary retrieval method BM25. In work on zero-shot text retrieval tasks, the most commonly used and effective method is domain adaptation, which uses generative models to generate document-related queries on the target dataset to build a training dataset for domain adaptation training.
[0005] Tables, as a widely used method for organizing and displaying information, also possess significant informational value. Therefore, table retrieval is a crucial topic in the field of information retrieval. Similar to text retrieval, table retrieval also faces the challenge of data annotation, and the complex structure of tables greatly increases this difficulty. Thus, zero-shot table retrieval tasks also hold significant research value. However, to date, no representative research has emerged on zero-shot table retrieval tasks. Furthermore, some methods for simpler table retrieval tasks have the following limitations:
[0006] 1) The BM25 method is a traditional sparse retrieval method that relies primarily on matching words that exist between the query and the retrieved objects. Therefore, the BM25 method suffers from word mismatch issues; for example, it cannot match the words "bike" and "bicycle," even though they have the same meaning. Consequently, the BM25 method is not very effective in either text or table retrieval.
[0007] 2) Zero-shot text retrieval models, through special design, such as introducing relevant pre-training tasks, can effectively complete zero-shot text retrieval tasks. However, due to the differences between tabular data and text data, especially the unique structural information and high information density of tabular data, directly using zero-shot text retrieval models results in less than ideal performance.
[0008] 3) Table Retrieval Models: Table retrieval models are often modified specifically for table retrieval tasks. For example, the TAPAS model incorporates embeddings related to table row and column positions, superimposed with the original word embeddings as model input. Table retrieval models can more effectively capture the structural information of tables, improving retrieval performance to some extent. However, these models are often more complex, reducing their feasibility for integration with text retrieval tasks. Furthermore, since they haven't been improved for zero-shot retrieval and require training with labeled data, they cannot directly handle zero-shot table retrieval tasks and require further improvement. Summary of the Invention
[0009] To address the aforementioned technical problems, the purpose of this invention is to provide a domain-adaptive zero-sample table retrieval method, enabling the table retrieval tool to effectively complete zero-sample table retrieval tasks, i.e., to effectively retrieve tables without relying on labeled data.
[0010] This invention provides a domain-adaptive zero-sample table retrieval method, comprising:
[0011] Step 1: Obtain the WikiSQL dataset, SQUALL template data, and NQTables dataset;
[0012] Step 2: Train the query transformer using the WikiSQL dataset;
[0013] Step 3: Based on the SQUALL template data and the NQTables dataset, construct the training data for the table retrieval machine using the query transformer;
[0014] Step 4: Design the table retrieval tool and its loss function, and train it using training data;
[0015] Step 5: Use the trained table search engine to perform table retrieval.
[0016] Furthermore, step 1 includes:
[0017] Step 1.1: Obtain the WikiSQL dataset as an SQL-to-Text dataset. Each piece of data in the WikiSQL dataset contains a natural language text query and its corresponding SQL query.
[0018] Step 1.2: Obtain SQUALL template data as an SQL template. Each piece of data in the SQUALL template data contains the source table and the corresponding SQL query statement.
[0019] Step 1.3: Obtain the NQTables dataset as a table retrieval dataset. Each data entry in the NQTables dataset contains a table corpus, a text query statement, and related tables.
[0020] Furthermore, step 2 includes:
[0021] Step 2.1: Use the BART model as the base model for the query converter;
[0022] Step 2.2: Use the text generation loss function to train the base model, specifically:
[0023]
[0024] Where θ represents the model parameters, x represents the input text, and y represents the output text; i For the predicted i-th label, P θ Predict the next label y for the model i The probability of predicting the next tag based on the input text and the already predicted tags of the output text; y 1:i-1 These are the first to i-1 predicted labels;
[0025] Step 2.3: Use the WikiSQL dataset obtained in Step 1.1 as training data, and train using the text generation loss function. Use the SQL query statement of each data in the WikiSQL dataset as input text x, and the natural language text query statement as output text y.
[0026] Furthermore, step 3 includes:
[0027] Step 3.1: Use the SQL template obtained in Step 1.2 to guide the SQL query statement for generating the target table;
[0028] Step 3.2: Use the table corpus of the NQTables dataset obtained in Step 1.3 as the target tables to be retrieved;
[0029] Step 3.3: Randomly select m target tables from the target tables to be searched, traverse the m target tables, randomly select an SQL template for each target table to generate the corresponding SQL query statement, skip the SQL template if it is not suitable, until n related SQL query statements are generated for each target table;
[0030] Step 3.4: Collect the SQL query statements generated in Step 3.3 and the relevant target tables to form a dataset, that is, each data entry contains the target table and its corresponding SQL query statement;
[0031] Step 3.5: For each data point in the dataset from Step 3.4, use the query converter trained in Step 2 to convert the SQL query statement into a natural language text query statement. The final training data for the table retrieval machine contains a natural language text query statement and its associated target table for each data point.
[0032] Furthermore, the SQL query statement generation steps in step 3.3 are as follows:
[0033] Step 3.3.1: Divide the SQL query statement in the SQL template corresponding to the source table into word fragments, specifically into keywords, column names, strings, or numbers;
[0034] Step 3.3.2: Traverse the word fragments of the SQL query statement corresponding to the source table, obtain the current word fragment, and if there is no fragment, jump to step 3.3.7;
[0035] Step 3.3.3: Determine if the current word fragment is a keyword. If it is, jump back to step 3.3.2 to get the next word fragment; otherwise, proceed to step 3.3.4.
[0036] Step 3.3.4: Determine if the current word fragment is a column name. If it is, randomly select a column name in the target table that has the same data type as the column in the source table to replace the column name in the source table, and record the corresponding column name. Then jump to step 3.3.2 to try to get the next word fragment. If there is no column with the same data type in the target table, abandon the SQL template and exit the process.
[0037] Step 3.3.5: If the current word fragment is data, i.e. a string or a number, determine whether the current data is the content of a cell in the source table. If so, find the corresponding column name of the column to which the data belongs in the target table in the records of step 3.3.4, then randomly select the data of the cell under that column to replace it, and then jump back to step 3.3.2 to try to get the next word fragment;
[0038] Step 3.3.6: If the current data is not cell content, and if the current data is a number, then randomly generate a number to replace it; otherwise, randomly select a cell text content from the target table to replace it, and then jump back to step 3.3.2 to try to get the next word fragment.
[0039] Step 3.3.7: Reassemble the word fragments into a complete SQL query statement and return it. The word fragments in the reassembled SQL query statement, i.e., the column names and data in the source table, should have been replaced in the previous steps.
[0040] Furthermore, step 4 includes:
[0041] Step 4.1: Design a table retrieval engine, which includes a query encoder, a table encoder, and a FAISS tool. The BERT model is used as the initialization model for the query encoder and the table encoder.
[0042] Step 4.2: Training Loss Function Design: The goal of training is to ensure that positive examples, i.e., relevant text queries and target tables, have higher relevance scores, while negative examples, i.e., irrelevant text queries and target tables, have lower relevance scores. The specific loss function is as follows:
[0043]
[0044] Among them, for a batch of data i represents the sequence number of the data in the batch, and q represents the text query and target table in each data entry. i ,t i ) is a positive example, while text queries and target tables in different datasets (q) i ,t j ) is a negative example, q i For each data point, t is the text query. i Let S be the target table, and S be the relevant score calculation function;
[0045] Step 4.3: Use the training data of the table retrieval tool built in Step 3 as training data, and train using the loss function in Step 4.2.
[0046] Furthermore, the training and retrieval of the table retrieval device designed in step 4.1 both rely on calculating the relevance scores between the text query and the target table. The specific tool flow of the table retrieval device is as follows:
[0047] Step 4.1.1: Input Preprocessing: Preprocess the input text, which includes text queries and target tables; for text queries, add the special marker "[CLS]" directly before the text; linearize the target table to obtain a linearized table, add the special marker "[CLS]" before the linearized table, and add the special markers "|" and "[ROW]" to the linearized table to separate cells and table rows, respectively;
[0048] Step 4.1.2: Tokenization: The tokenizer divides the input text into tokens in the vocabulary, then maps them to the corresponding token IDs according to the vocabulary and passes them to the corresponding encoder;
[0049] Step 4.1.3: Encoding: The embedding layers of the query encoder and table encoder map the tag IDs to a 768-dimensional tensor, and then interact with each other in subsequent layers using a self-attention mechanism, finally outputting the hidden state corresponding to each tag, which is also a 768-dimensional tensor;
[0050] Step 4.1.4: Relevance Score Calculation: Take the hidden state marked "[CLS]" as the representation vector of the input text, and then calculate the inner product of the representation vectors of the text query and the target table as the relevance score; the higher the relevance score, the more relevant the text query and the target table are considered; the formula for calculating the relevance score is as follows:
[0051] S(q,t)=E Q (q) T E T (t)
[0052] Where q represents a text query, t represents the target table, and E Q Indicates the query encoder, E T E represents the table encoder. Q (q) is the representation vector of the text query, E T (t) is the representation vector of the target table.
[0053] Furthermore, step 5 includes:
[0054] Step 5.1: Table Corpus Selection: Select the table corpus of the NQTables dataset obtained in Step 1.3 as the retrieval target; for a given text query, the table retrieval tool will retrieve the target table from the table corpus and return it to the user;
[0055] Step 5.2: Table Corpus Encoding: Use the table encoder in the table retrieval tool to encode all tables in the table corpus to obtain the representation vector of each table;
[0056] Step 5.3: Index Building: Use the FAISS tool to build a retrieval index for all the representation vectors obtained in Step 5.2;
[0057] Step 5.4: User Query Encoding: Encode the given text query using the query encoder in the table retrieval tool to obtain a representation vector;
[0058] Step 5.5: Relevant Target Table Retrieval: The representation vector of the text query obtained in Step 5.4 is passed to the FAISS tool. The FAISS tool will then use the index built on the representation vector of the table corpus to efficiently retrieve the table with the largest inner product result, that is, the target table with the highest relevance score to the text query.
[0059] The domain-adaptive zero-sample table retrieval method of the present invention has at least the following beneficial effects:
[0060] 1) The table retrieval method of the present invention does not rely on labeled data to train the model. It can automatically generate high-quality training data by providing only a table corpus, which can effectively solve the zero-sample table retrieval task.
[0061] 2) The table retrieval method of this invention linearizes the table and inputs special tags to identify the table's structural information, enabling the model to better understand the table. It does not require modification of the model structure, making it simple and efficient.
[0062] 3) The table retrieval method of this invention uses SQL queries converted into text queries to construct training data, instead of directly using SQL queries or directly generating text queries. This is because SQL statements inherently focus on the structural information of tables, and text queries converted from SQL statements inherit this advantage, enabling the model to understand the structural information of tables. Directly using automatically generated text queries fails to capture the structural information of tables, and SQL statements differ from natural language (different from the usage scenario), making them unsuitable as training data. Attached Figure Description
[0063] Figure 1 This is a flowchart of a domain-adaptive zero-sample table retrieval method according to the present invention;
[0064] Figure 2 This is a flowchart of the SQL query statement generation process;
[0065] Figure 3 The calculation process for the relevant scores
[0066] Figure 4 This is a flowchart of the table search function. Detailed Implementation
[0067] This invention provides a domain-adaptive zero-shot table retrieval method. First, a query transformer is trained using an SQL-to-Text dataset to convert SQL queries into natural language text queries. Then, an SQL template guides the generation of SQL queries for the target table, which are then converted back into natural language text queries using the previously trained query transformer. Next, the converted natural language text queries are combined with target tables from a table corpus to construct a training dataset for training a table retrieval tool. Finally, the table retrieval tool encodes both the user's text query and the target table from the table corpus, and then uses the FAISS tool to efficiently retrieve the relevant target tables.
[0068] like Figure 1 As shown, the present invention provides a domain-adaptive zero-sample table retrieval method, which specifically includes the following steps:
[0069] Step 1: Obtain the WikiSQL dataset, SQUALL template data, and NQTables dataset. Step 1 includes:
[0070] Step 1.1: Obtain the WikiSQL dataset as an SQL-to-Text dataset. Each record in the WikiSQL dataset contains a natural language text query and its corresponding SQL query. The WikiSQL dataset is used in subsequent steps to train the query transformer.
[0071] Step 1.2: Obtain SQUALL template data as an SQL template. Each piece of data in the SQUALL template data contains the source table and the corresponding SQL query statement.
[0072] Step 1.3: Obtain the NQTables dataset as a table retrieval dataset. Each data entry in the NQTables dataset contains a table corpus, a text query statement, and related tables.
[0073] Step 2: Train the query transformer using the WikiSQL dataset. Step 2 includes:
[0074] Step 2.1: Use the BART model as the base model for the query converter.
[0075] The BART model is a pre-trained language model with a complete Transformer architecture, a type of neural network model that can understand natural language through pre-training. The Transformer architecture includes an Encoder and a Decoder, primarily used for text generation tasks. The Transformer's workflow is as follows: Given input text, a tokenizer segments the input text into tokens, and then uses a vocabulary to obtain the corresponding IDs. The embedding layer in the encoder embeds the serialized token IDs to obtain a 768-dimensional tensor (input length), which is then encoded (including its unique self-attention mechanism), resulting in a 768-dimensional tensor as the hidden state. The decoder predicts one token ID at a time based on the encoder's result and its own predictions; this prediction is autoregressive, meaning the decoder stops when it predicts the last token ID. Finally, all token IDs are decoded by the tokenizer and concatenated to obtain the output text.
[0076] Step 2.2: Use the text generation loss function to train the base model, specifically:
[0077]
[0078] Where θ represents the model parameters, x represents the input text, and y represents the output text; i For the predicted i-th label, P θ Predict the next label y for the model i The probability of predicting the next tag based on the input text and the already predicted tags of the output text; y 1:i-1 These are the first to i-1 predicted labels;
[0079] Step 2.3: Use the WikiSQL dataset obtained in Step 1.1 as training data, and train using the text generation loss function. Use the SQL query statement of each data in the WikiSQL dataset as input text x, and the natural language text query statement as output text y.
[0080] Step 3: Based on the SQUALL template data and the NQTables dataset, construct training data for the table retrieval tool using a query transformer. Step 3 includes:
[0081] Step 3.1: Use the SQL template obtained in Step 1.2 to guide the SQL query statement for generating the target table.
[0082] Step 3.2: Use the table corpus of the NQTables dataset obtained in Step 1.3 as the target tables to be retrieved.
[0083] Step 3.3: Randomly select m target tables from the target tables to be searched. Iterate through the m target tables, and randomly select an SQL template for each target table to generate the corresponding SQL query statement. If the template is unsuitable, skip it. Continue until n relevant SQL query statements are generated for each target table, such as... Figure 2 As shown, the steps for generating the SQL query statement in step 3.3 are as follows:
[0084] Step 3.3.1: Divide the SQL query statement in the SQL template corresponding to the source table into word fragments, specifically into keywords, column names, strings, or numbers;
[0085] Step 3.3.2: Traverse the word fragments of the SQL query statement corresponding to the source table, obtain the current word fragment, and if there is no fragment, jump to step 3.3.7;
[0086] Step 3.3.3: Determine if the current word fragment is a keyword. If it is, jump back to step 3.3.2 to get the next word fragment; otherwise, proceed to step 3.3.4.
[0087] Step 3.3.4: Determine if the current word fragment is a column name. If it is, randomly select a column name in the target table that has the same data type as the column in the source table to replace the column name in the source table, and record the corresponding column name. Then jump to step 3.3.2 to try to get the next word fragment. If there is no column with the same data type in the target table, abandon the SQL template and exit the process.
[0088] Step 3.3.5: If the current word fragment is data, i.e. a string or a number, determine whether the current data is the content of a cell in the source table. If so, find the corresponding column name of the column to which the data belongs in the target table in the records of step 3.3.4, then randomly select the data of the cell under that column to replace it, and then jump back to step 3.3.2 to try to get the next word fragment;
[0089] Step 3.3.6: If the current data is not cell content, and if the current data is a number, then randomly generate a number to replace it; otherwise, randomly select a cell text content from the target table to replace it, and then jump back to step 3.3.2 to try to get the next word fragment.
[0090] Step 3.3.7: Reassemble the word fragments into a complete SQL query statement and return it. The word fragments in the reassembled SQL query statement, i.e., the column names and data in the source table, should have been replaced in the previous steps.
[0091] Step 3.4: Collect the SQL query statements generated in Step 3.3 and the relevant target tables to form a dataset, that is, each data entry contains the target table and its corresponding SQL query statement.
[0092] Step 3.5: For each data point in the dataset from Step 3.4, use the query converter trained in Step 2 to convert the SQL query statement into a natural language text query statement. The final training data for the table retrieval machine contains a natural language text query statement and its associated target table for each data point.
[0093] Step 4: Design the table retrieval tool and its loss function, and train it using training data. Step 4 includes:
[0094] Step 4.1: Design the table retrieval engine, which includes a query encoder and a table encoder. These are both encoder structures within the transformer architecture, using the BERT model as the initialization model for both the query encoder and the table encoder. Additionally, the table retrieval engine includes a FAISS tool, but it does not require training and is only used during the retrieval phase.
[0095] The training and retrieval of the table retrieval device designed in step 4.1 both rely on calculating the relevance scores between the text query and the target table, such as... Figure 3 As shown, the specific calculation process for the relevant scores is as follows:
[0096] Step 4.1.1: Input Preprocessing: The input text is preprocessed, including text queries and target tables. For text queries, a special marker "[CLS]" is added before the text to indicate that the encoder aggregates the semantic information of the entire input text during the encoding process, and uses this to determine the relevance of different input texts. Unlike text queries, tables are structured data (two-dimensional structure) and cannot be directly used as input. The target table needs to be linearized to obtain a linearized table. A special marker "[CLS]" is added before the linearized table, and special markers "|" and "[ROW]" are added to the linearized table to separate cells and table rows, respectively.
[0097] Step 4.1.2: Tokenization: The tokenizer divides the input text into tokens in the vocabulary, and then maps them to the corresponding token IDs according to the vocabulary before passing them to the corresponding encoder.
[0098] Step 4.1.3: Encoding: The embedding layers of the query encoder and table encoder map the tag ID to a 768-dimensional tensor, and then interact with it in subsequent layers using a self-attention mechanism, finally outputting the hidden state corresponding to each tag, which is also a 768-dimensional tensor.
[0099] Step 4.1.4: Relevance Score Calculation: Take the hidden state marked "[CLS]" as the representation vector of the input text, and then calculate the inner product of the representation vectors of the text query and the target table as the relevance score; the higher the relevance score, the more relevant the text query and the target table are considered; the formula for calculating the relevance score is as follows:
[0100] S(q,t)=E Q (q) T E T (t)
[0101] Where q represents a text query, t represents the target table, and E Q Indicates the query encoder, E T E represents the table encoder. Q (q) is the representation vector of the text query, E T (t) is the representation vector of the target table.
[0102] Step 4.2: Training Loss Function Design: The goal of training is to ensure that positive examples, i.e., relevant text queries and target tables, have higher relevance scores, while negative examples, i.e., irrelevant text queries and target tables, have lower relevance scores. The specific loss function is as follows:
[0103]
[0104] Among them, for a batch of data i represents the sequence number of the data in the batch, and q represents the text query and target table in each data entry. i ,t i ) is a positive example, while text queries and target tables in different datasets (q) i ,t j ) is a negative example, q i For each data point, t is the text query. i Let S be the target table, and S be the relevant score calculation function.
[0105] Step 4.3: Use the training data of the table retrieval tool built in Step 3 as training data, and train using the loss function in Step 4.2.
[0106] Step 5: Perform table retrieval using the trained table retrieval tool, such as... Figure 4 As shown, step 5 includes:
[0107] Step 5.1: Table Corpus Selection: Select the table corpus of the NQTables dataset obtained in Step 1.3 as the retrieval target; for a given text query, the table retrieval tool will retrieve the target table from the table corpus and return it to the user.
[0108] Step 5.2: Table Corpus Encoding: Use the table encoder in the table retrieval tool to encode all tables in the table corpus (the specific process is as described in Step 4.1, but text queries are not encoded and relevance scores are not calculated), to obtain the representation vector of each table.
[0109] Step 5.3: Index Building: Use the FAISS tool to build a retrieval index for all the representation vectors obtained in Step 5.2. FAISS is a framework that provides efficient similarity search for dense vectors (such as encoded 768-dimensional tensors), offering multiple retrieval methods, high speed, and the ability to be stored in memory and on disk.
[0110] Step 5.4: User Query Encoding: Encode the given text query using the query encoder in the table retrieval tool to obtain a representation vector.
[0111] Step 5.5: Relevant Target Table Retrieval: The representation vector of the text query obtained in Step 5.4 is passed to the FAISS tool. The FAISS tool then uses an index built from the representation vectors of the table corpus to efficiently retrieve the table with the largest inner product result, i.e., the target table with the highest relevance score to the text query. In practical applications, the k tables with the highest relevance scores will be selected and returned to the user to improve the actual effect.
[0112] This invention proposes a domain-adaptive zero-shot table retrieval method, which effectively solves the zero-shot table retrieval task. With only a table corpus provided, it can automatically generate high-quality training data for training the table retrieval machine. When a user query arrives, it can effectively and efficiently retrieve relevant tables from the table corpus and provide them to the user.
[0113] Labeling data is often extremely expensive, while the method of this invention can effectively complete retrieval tasks without relying on labeled data for training, thus saving a significant amount of time and money. Furthermore, for some specialized and complex fields, such as medicine and aviation, there is often a difficulty in labeling data, and the method of this invention can also effectively solve this problem.
[0114] The above description is only a preferred embodiment of the present invention and is not intended to limit the ideas of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A domain-adaptive zero-sample table retrieval method, characterized in that, include: Step 1: Obtain the WikiSQL dataset, SQUALL template data, and NQTables dataset; Step 2: Train the query transformer using the WikiSQL dataset; Step 3: Based on the SQUALL template data and the NQTables dataset, construct the training data for the table retrieval machine using the query transformer; Step 4: Design the table retrieval tool and its loss function, and train it using training data; Step 5: Perform table retrieval using the trained table retrieval tool; Step 3 includes: Step 3.1: Use the SQL template obtained in Step 1.2 to guide the SQL query statement for generating the target table; Step 3.2: Use the table corpus of the NQTables dataset obtained in Step 1.3 as the target tables to be retrieved; Step 3.3: Randomly select m target tables from the target tables to be searched, traverse the m target tables, randomly select an SQL template for each target table to generate the corresponding SQL query statement, skip the SQL template if it is not suitable, until n related SQL query statements are generated for each target table; Step 3.4: Collect the SQL query statements generated in Step 3.3 and the relevant target tables to form a dataset, that is, each data entry contains the target table and its corresponding SQL query statement; Step 3.5: For each data point in the dataset from Step 3.4, use the query converter trained in Step 2 to convert the SQL query statement into a natural language text query statement. The final training data for the table retrieval machine contains a natural language text query statement and its associated target table for each data point. Step 4 includes: Step 4.1: Design a table retrieval engine, which includes a query encoder, a table encoder, and a FAISS tool. The BERT model is used as the initialization model for the query encoder and the table encoder. Step 4.2: Training Loss Function Design: The goal of training is to ensure that positive examples, i.e., relevant text queries and target tables, have higher relevance scores, while negative examples, i.e., irrelevant text queries and target tables, have lower relevance scores. The specific loss function is as follows: Among them, for a batch of data 'i' represents the sequence number of the data in the batch, and the text query and target table for each data entry. This is a positive example, while the text queries and target tables in different data are different. For negative examples, For each data entry, perform a text query. Let S be the target table, and S be the relevant score calculation function; Step 4.3: Use the training data of the table retrieval tool built in Step 3 as training data, and train using the loss function in Step 4.
2.
2. The domain-adaptive zero-sample table retrieval method as described in claim 1, characterized in that, Step 1 includes: Step 1.1: Obtain the WikiSQL dataset as an SQL-to-Text dataset. Each piece of data in the WikiSQL dataset contains a natural language text query and its corresponding SQL query. Step 1.2: Obtain SQUALL template data as an SQL template. Each piece of data in the SQUALL template data contains the source table and the corresponding SQL query statement. Step 1.3: Obtain the NQTables dataset as a table retrieval dataset. Each data entry in the NQTables dataset contains a table corpus, a text query statement, and related tables.
3. The domain-adaptive zero-sample table retrieval method as described in claim 2, characterized in that, Step 2 includes: Step 2.1: Use the BART model as the base model for the query converter; Step 2.2: Use the text generation loss function to train the base model, specifically: in, Here are the parameters of the model, x is the input text, and y is the output text; For the predicted i-th label, Predict the next label for the model The probability of predicting the next tag based on the input text and the already predicted tags of the output text; These are the first to i-1 predicted labels; Step 2.3: Use the WikiSQL dataset obtained in Step 1.1 as training data, and train using the text generation loss function. Use the SQL query statement of each data in the WikiSQL dataset as input text x, and the natural language text query statement as output text y.
4. The domain-adaptive zero-sample table retrieval method as described in claim 1, characterized in that, The SQL query statement generation steps in step 3.3 are as follows: Step 3.3.1: Divide the SQL query statement in the SQL template corresponding to the source table into word fragments, specifically into keywords, column names, strings, or numbers; Step 3.3.2: Traverse the word fragments of the SQL query statement corresponding to the source table, obtain the current word fragment, and if there is no fragment, jump to step 3.3.7; Step 3.3.3: Determine if the current word fragment is a keyword. If it is, jump back to step 3.3.2 to get the next word fragment; otherwise, proceed to step 3.3.
4. Step 3.3.4: Determine whether the current word fragment is a column name. If it is, randomly select a column name in the target table that has the same data type as the column in the source table to replace the column name in the source table, and record the corresponding column name. Then jump to step 3.3.2 to try to obtain the next word fragment. If the target table does not have a column with the same data type, the SQL template will be abandoned and the process will be terminated. Step 3.3.5: If the current word fragment is data, i.e. a string or a number, determine whether the current data is the content of a cell in the source table. If so, find the corresponding column name of the column to which the data belongs in the target table in the records of step 3.3.4, then randomly select the data of the cell under that column to replace it, and then jump back to step 3.3.2 to try to get the next word fragment; Step 3.3.6: If the current data is not cell content, and if the current data is a number, then randomly generate a number to replace it; otherwise, randomly select a cell text content from the target table to replace it, and then jump back to step 3.3.2 to try to get the next word fragment. Step 3.3.7: Reassemble the word fragments into a complete SQL query statement and return it. The word fragments in the reassembled SQL query statement, i.e., the column names and data in the source table, should have been replaced in the previous steps.
5. The domain-adaptive zero-sample table retrieval method as described in claim 1, characterized in that, The training and retrieval of the table retrieval device designed in step 4.1 both rely on calculating the relevance scores between the text query and the target table. The specific tool flow of the table retrieval device is as follows: Step 4.1.1: Input Preprocessing: Preprocess the input text, which includes text queries and target tables; for text queries, add the special marker "[CLS]" directly before the text; linearize the target table to obtain a linearized table, add the special marker "[CLS]" before the linearized table, and add the special markers "|" and "[ROW]" to the linearized table to separate cells and table rows, respectively; Step 4.1.2: Tokenization: The tokenizer divides the input text into tokens in the vocabulary, then maps them to the corresponding token IDs according to the vocabulary and passes them to the corresponding encoder; Step 4.1.3: Encoding: The embedding layers of the query encoder and table encoder map the tag IDs to a 768-dimensional tensor, and then interact with each other in subsequent layers using a self-attention mechanism, finally outputting the hidden state corresponding to each tag, which is also a 768-dimensional tensor; Step 4.1.4: Relevance Score Calculation: Take the hidden state marked "[CLS]" as the representation vector of the input text, and then calculate the inner product of the representation vectors of the text query and the target table as the relevance score; the higher the relevance score, the more relevant the text query and the target table are considered; the formula for calculating the relevance score is as follows: Where q represents a text query, t represents the target table, and E Q Indicates the query encoder, E T Indicates a table encoder. This is the representation vector for the text query. is the representation vector of the target table.
6. The domain-adaptive zero-sample table retrieval method as described in claim 1, characterized in that, Step 5 includes: Step 5.1: Table Corpus Selection: Select the table corpus of the NQTables dataset obtained in Step 1.3 as the retrieval target; for a given text query, the table retrieval tool will retrieve the target table from the table corpus and return it to the user; Step 5.2: Table Corpus Encoding: Use the table encoder in the table retrieval tool to encode all tables in the table corpus to obtain the representation vector of each table; Step 5.3: Index Building: Use the FAISS tool to build a retrieval index for all the representation vectors obtained in Step 5.2; Step 5.4: User Query Encoding: Encode the given text query using the query encoder in the table retrieval tool to obtain a representation vector; Step 5.5: Relevant Target Table Retrieval: The representation vector of the text query obtained in Step 5.4 is passed to the FAISS tool. The FAISS tool will then use the index built on the representation vector of the table corpus to efficiently retrieve the table with the largest inner product result, that is, the target table with the highest relevance score to the text query.
Citation Information
Patent Citations
SQL (Structured Query Language) conversion method and system based on language model coding and multi-task decoding
CN112988785A
Method and device for generating structured query language statement
CN114528312A
