Dynamic Question Generation from Tables and Text
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current question generation in Natural Language Processing (NLP) faces challenges such as low quality and diversity of questions, dependence on high-quality input text, difficulty in generating questions for complex texts, and lack of understanding of abstract concepts, particularly when using fixed templates.
Innovation Solution
A system that receives a corpus of documents containing tables and passages, employs a path sampler, reasoning path, and transformer generator to generate questions without fixed templates, supporting various input types like SQuaD, HotpotQA, WikiSQL, and HybridQA, optimizing query generation and improving machine learning processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If fixed templates are used for question generation, then the generation process is simple and efficient, but the quality and diversity of generated questions deteriorate
Solution Approach 1:
The patent replaces fixed templates with dynamic question generation using transformer models that can adapt to different input contexts. The system dynamically selects and generates questions based on the specific table and passage content, rather than using predetermined templates, thereby improving question quality and diversity while maintaining efficiency through automated generation.
Solution Approach 2:
The system changes the generation approach from static template filling to dynamic transformer-based generation. By using neural networks to transform input contexts into questions, the system can produce varied and high-quality questions that adapt to different domains and complexities, resolving the contradiction between efficiency and quality.
2Ease of manufacture
If fixed templates are used for question generation, then the implementation is straightforward, but the ability to handle complex texts and abstract concepts deteriorates
Solution Approach 1:
The patent replaces the mechanical template-filling approach with a neural network-based transformer model. This substitution enables the system to handle complex texts and abstract concepts by learning patterns from training data, while the automated nature of the system keeps the implementation relatively straightforward through standard NLP pipelines.
Solution Approach 2:
The transformer-based system provides universality by capable of handling multiple types of inputs (tables, passages, hybrid) and generating appropriate questions across different domains. This single model can adapt to various text complexities and domains, unlike template-based systems that require domain-specific templates.
3Reliability
If template-based question generation is used, then the system is less dependent on input text quality, but the generated questions lack coherence and relevance
Solution Approach 1:
The transformer model incorporates feedback mechanisms where the generated question is evaluated for coherence and relevance, and the model can iteratively refine its output. This feedback loop ensures that questions maintain high coherence and relevance to the input text, while the model's training on diverse data provides robustness to varying input qualities.
Data Source
AI summary
One or more systems, devices, computer program products and/or computer-implemented methods of use provided herein relate to a process to facilitate a Question Generation System. A system can comprise a memory that stores computer executable components, and a processor that executes the computer executable components stored in the memory, wherein the computer executable components can comprise a receiving component that receives a corpus of documents that contain Tables (Ts) and Passages (Ps) for performing natural language processing (NLP); an executing component that executes the NLP by employing the tables (sT) and passages (Ps) as primary inputs; and a query component that generates an output Question (Q) based on a subset of the tables Ts and passages (Ps).