Contextual Query Generation for QA Models

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional question answering (QA) models face limitations in providing accurate responses due to the lack of domain-specific training data, struggling with noisy or unfamiliar documents, and requiring large amounts of labeled data for supervised learning.

Innovation Solution

A content processing system uses in-context learning to condition a language model, generating contextual queries based on semantic contexts of input documents, which are then used to improve the accuracy of QA models by asking the right questions to extract relevant information.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If conventional QA models are trained using supervised learning with large sets of domain specific training data, then the models can provide accurate responses in familiar domains, but the training data is not readily available and computationally demanding to generate

Engineering Contradiction:

Improveaccuracy of QA model responsesVSAvoidamount of labeled training data required

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent applies preliminary action by pre-training the language model on general language data before deploying it for domain-specific QA tasks. This pre-training establishes fundamental language understanding capabilities that can be adapted to various domains without requiring extensive domain-specific labeled data, thus reducing the quantity of training data needed while maintaining response accuracy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces an intermediary language model that acts as a bridge between the question and the document. This language model generates contextual queries that improve the effectiveness of the QA system, reducing reliance on large amounts of domain-specific training data while maintaining or improving response accuracy through better query formulation.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If QA models are trained with domain specific data, then they perform well in familiar domains, but they perform poorly when used in unfamiliar domains

Engineering Contradiction:

Improveperformance consistency across domainsVSAvoidability to handle unfamiliar domains

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The patent implements universality by creating a language model that can function across multiple domains without requiring domain-specific retraining. The model uses general language understanding capabilities combined with contextual query generation to adapt to unfamiliar domains, making the system versatile while maintaining reliable performance across different domain contexts.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent applies preliminary action through pre-training on general language data, which establishes transferable skills that enable the model to adapt to unfamiliar domains. This preliminary preparation allows the model to maintain reliability across domains by leveraging fundamental language understanding rather than relying on domain-specific training that would limit adaptability.

Inventive Principle:

Principle #10Preliminary action

3Productivity

If conventional QA models process inputs quickly, then they can analyze large bodies of text efficiently, but they are limited by availability of labeled training data

Engineering Contradiction:

Improvespeed of text processingVSAvoidavailability of labeled training data

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The patent introduces an intermediary language model that enhances the QA system's ability to generate effective queries without requiring additional labeled training data. This intermediary component improves the quality of query generation using general language understanding, maintaining productivity while overcoming the limitation of limited labeled data availability.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent applies self-service by enabling the language model to generate its own contextual queries based on general language understanding rather than relying on externally provided labeled training data. The model serves itself by adapting to different domains through its pre-trained capabilities, maintaining processing speed while reducing dependency on scarce labeled data.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS20240427998A1Contextual query generation

Publication Date: 2024.12.26 ADOBE INC

AI summary

Contextual query generation techniques are described that enable generation of a contextual query for output to a question-answering (QA) model. A content processing system, for instance, configures a language model using in-context learning to generate queries based on semantic contexts of input documents, e.g., based on one or more linguistic cues from text of the input documents. The content processing system receives an input that includes a document having text and a reference query. The content processing system leverages the language model to generate a contextual query based on a semantic context of the text of the document and the reference query. The content processing system then outputs the contextual query and the document to a QA model. Using the QA model, the content processing system generates a response as an answer to the contextual query based on the contextual query and the document.