Secure Table Question Answering via Schema-Only Code Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing large language models face challenges in contextually analyzing table question-answering tasks due to finite context windows, tokenization discrepancies, and data confidentiality issues, leading to impractical exposure of underlying data and potential privacy breaches.
Innovation Solution
A method involving a cooperative game between an 'Oracle' and a 'Solver' agent, where the Solver generates executable code based on schema and instructions provided by the Oracle, ensuring data security by not exposing the underlying data, using a framework like HiddenTables and a dataset like PyQTax.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If encoder-based approaches expose tabular content in its entirety to achieve accuracy, then question-answering accuracy is improved, but data privacy and security deteriorate
Solution Approach 1:
The system segments the tabular data processing into two distinct parts: (1) schema information and column headers are exposed to the language model for understanding table structure, while (2) actual underlying data values are hidden and processed separately through code execution. This segmentation allows the model to learn from structural patterns without accessing sensitive data content.
Solution Approach 2:
The patent introduces an intermediary code-generation layer that acts as a mediator between the language model and the actual data. The model generates executable code (SQL queries, Python code) based on schema information, and this code is then executed against the hidden data to retrieve answers. This intermediary mechanism enables accurate querying without direct model access to sensitive data.
2Loss of information
If the model ingests tokens liberally to understand tabular content, then semantic understanding is improved, but computational cost and context window limitations worsen
Solution Approach 1:
The patent extracts and removes the actual data values from the token input stream, retaining only the schema information and column headers. This extraction eliminates the need to process thousands of data tokens while preserving the structural information necessary for generating accurate queries. The model processes minimal tokens (schema only) rather than liberal ingestion of all tabular content.
Solution Approach 2:
The system applies partial action by providing only the necessary portion of tabular information (schema and headers) to the language model, rather than the complete table content. This partial input is sufficient for the model to generate accurate queries when combined with code execution, significantly reducing token consumption and computational overhead.
3Ease of operation
If pre-trained models use byte-pair-encoding or WordPiece tokenization, then language processing capability is improved, but alignment with tabular structure deteriorates
Solution Approach 1:
Instead of attempting to make the tokenization align with tabular structures (the conventional approach), the patent inverts the problem by making the tabular schema information conform to standard NLP tokenization patterns. The schema is processed through conventional byte-pair-encoding or WordPiece, and the model learns to map these tokens back to meaningful table structures through training on schema-question-answer pairs.
4Adaptability or versatility
If external models are used for table question-answering, then model capability is improved, but data confidentiality requirements worsen
Solution Approach 1:
The patent introduces an intermediary code-generation layer that acts as a mediator between the language model and the actual data. The model generates executable code (SQL queries, Python code) based on schema information, and this code is then executed against the hidden data to retrieve answers. This intermediary mechanism enables accurate querying without direct model access to sensitive data.
Solution Approach 2:
The system creates a simplified copy of the tabular structure (schema and column headers) that can be safely shared with external models without containing actual sensitive data. This copied structural information is sufficient for the model to generate accurate queries, while the real data remains protected and is only accessed through controlled code execution in the original environment.
Data Source
AI summary
A method for using a large language model to perform complex table question-answering tasks while preserving data security and privacy with respect to the underlying data included in a table is provided. The method includes: receiving a user query that relates to a table; generating, based on the user query, a prompt that includes a role, at least one instruction, schema, and a question; transmitting the prompt to an external processor that does not have access to the underlying data included in the table; receiving, from the external processor, a set of executable code which has been generated by the external processor in response to the prompt and which is applicable to the table; generating an answer to the user query by executing the set of executable code; and transmitting, to the user, the answer to the user query.