Neural Key Phrase Detection for Automatic Question Generation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing automatic question generation systems for assessing reading comprehension rely heavily on rule-based methods that fail to scale across different domains and writing styles, and require significant human effort, especially in massive open online courses (MOOCs), and struggle to capture the diversity and variance of natural language questions.

Innovation Solution

An end-to-end trainable neural model is employed to identify interesting phrases in a document and generate questions, using a neural entity selection model and a pointer network to automatically predict key phrases and condition a question generation model, trained on a large-scale crowd-sourced question-answering dataset like SQuAD to learn the importance and appropriateness of answers.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If rule-based methods are used for automatic question generation, then implementation is straightforward with templates and syntactic transformations, but the system fails to scale across different domains and writing styles, and cannot capture the diversity of natural language questions

Engineering Contradiction:

Improveease of implementationVSAvoidadaptability to different domains and writing styles

Core Design Contradiction:

Ease of manufactureVSAdaptability or versatility

Solution Approach 1:

The patent replaces rule-based mechanical systems (templates, syntactic transformations) with a neural network-based automatic question generation system. The neural model learns to generate questions directly from text passages without relying on hand-crafted rules, enabling it to adapt to different domains and writing styles while maintaining ease of implementation through automated training.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Reliability

If manual human effort is used to formulate questions for reading comprehension assessment, then question quality and appropriateness are high, but the process requires significant human effort that is difficult to scale to massive open online courses

Engineering Contradiction:

Improvequestion qualityVSAvoidscaling capability

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent implements a self-service system where the neural network automatically generates questions from text passages without requiring human intervention for each question. The model is trained once on a dataset and then autonomously generates questions for any input text, eliminating the need for continuous human effort while maintaining question quality and enabling unlimited scaling.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent uses a neural network that learns from example question-answer pairs in the training data and generates new questions by applying learned patterns to new text passages. This copying of learned generation patterns enables the system to produce high-quality questions at scale without replicating human effort for each individual question.

Inventive Principle:

Principle #26Copying

3Device complexity

If simple rule-based techniques such as slot-filling with templates are used, then the system is computationally efficient and easy to implement, but the techniques are inadequate to capture the diversity and high variance of natural language questions

Engineering Contradiction:

Improvesystem simplicityVSAvoidcapability to capture natural language diversity

Core Design Contradiction:

Device complexityVSReliability

Solution Approach 1:

The patent replaces simple rule-based techniques with a neural network model that automatically learns complex patterns in natural language questions. The neural model processes text passages and generates diverse questions by learning from training data, capturing the high variance of natural language without requiring complex hand-crafted rules.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

4Measurement precision

If crowd-sourcing is used to rate the acceptability of computer-generated questions, then quality assessment is improved, but the process is expensive and time consuming

Engineering Contradiction:

Improvequality assessment accuracyVSAvoidtime and cost for quality rating

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent implements self-service quality assessment through automated metrics that evaluate generated questions without human intervention. The system uses computational methods to assess question quality, eliminating the need for expensive and time-consuming crowd-sourcing while maintaining adequate quality control.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS20240404421A1Neural models for key phrase detection and question generation

Publication Date: 2024.12.05 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

A method, system, and storage device storing a computer program, for generating questions based on provided content, such as, for example, a document having words. The method comprises automatically estimating the probability of interesting phrases in the provided content, and generating a question in natural language based on the estimating. In one example embodiment herein, the estimating includes predicting the interesting phrases as answers, and the estimating is performed by a neural model. The method further comprises conditioning a question generation model based on the interesting phrases predicted in the predicting, the question generation model generating the question. The method also can include training the neural model. In one example, the method further comprises identifying start and end locations of the phrases in the provided content, and the identifying includes performing a dot product attention mechanism parameterizing a probability distribution.