Candidate Answer Variant Engine for Structured Table Data

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Cognitive systems, such as QA systems, are unable to effectively process structured content like table data structures, often ignoring them in favor of natural language processing, which limits their ability to identify candidate answer variants from structured portions of content.

Innovation Solution

A method is implemented in a data processing system that generates candidate answer variants by analyzing structured content, such as table data structures, by creating groupings of elements and calculating similarity measures between these groupings and the input question, using vector representations and similarity calculations like cosine similarity to identify relevant elements as candidate answers.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Difficulty of detecting and measuring

If cognitive systems focus on natural language processing, then processing capability for unstructured content is improved, but ability to process structured content like tables deteriorates

Engineering Contradiction:

Improveprocessing capabilityVSAvoidability to process structured content

Core Design Contradiction:

Difficulty of detecting and measuringVSAdaptability or versatility

Solution Approach 1:

The patent segments the content processing task by separating structured content (tables) from unstructured content (natural language). It applies different processing pipelines: one for natural language understanding and another for structured data analysis. This allows the system to handle each type of content with specialized techniques, resolving the contradiction between optimizing for natural language while maintaining structured content capability.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces an intermediary component that bridges structured and unstructured content processing. This intermediary extracts meaningful information from tables and converts it into a format that can be integrated with natural language processing results. By acting as a mediator, it enables the system to leverage both structured and unstructured content without compromising either processing capability.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Device complexity

If QA systems ignore structured portions of content, then processing complexity is reduced, but information completeness deteriorates

Engineering Contradiction:

Improveprocessing complexityVSAvoidinformation completeness

Core Design Contradiction:

Device complexityVSLoss of information

Solution Approach 1:

The patent applies preliminary action by pre-processing structured content before the main QA processing. It extracts key information from tables in advance, organizes it into structured formats, and prepares it for integration with natural language processing results. This preliminary preparation reduces the complexity of the main processing while ensuring no information is lost.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent merges the results from structured content analysis with unstructured content analysis. It combines candidate answers derived from tables with those from natural language processing, integrating multiple information sources. This merging approach maintains information completeness while managing complexity through systematic integration.

Inventive Principle:

Principle #5Merging (Combining)

3Productivity

If the system processes only natural language content, then processing speed is improved, but answer accuracy deteriorates

Engineering Contradiction:

Improveprocessing speedVSAvoidanswer accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent applies partial action by selectively processing only the relevant portions of structured content that are necessary for answering the question. Instead of processing entire tables unnecessarily, it identifies and processes only the specific rows and columns that contain relevant information. This maintains processing speed while improving answer accuracy by focusing computational resources on pertinent data.

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The patent substitutes traditional mechanical text-processing approaches with specialized techniques for structured data. Instead of applying general-purpose natural language processing to tables, it uses dedicated structured query and analysis methods. This substitution maintains efficiency while significantly improving accuracy for structured content interpretation.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS11132370B2Generating answer variants based on tables of a corpus

Publication Date: 2021.09.28 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

Mechanisms are provided for implementing a candidate answer variant engine. The mechanisms receive an indication of a structured portion of content in a corpus, generate a plurality of groupings of elements of the structured portion of content, and generate, for each grouping of elements in the plurality of groupings of elements, and for each element in the grouping of elements, a corresponding grouping vector representation, corresponding to the element. The mechanisms, for each grouping vector representation of each grouping of elements in the plurality of groupings of elements perform a similarity measure calculation between the grouping vector representation and a vector representation of an input question, and select an element corresponding to the grouping vector representation for inclusion as a candidate answer variant based on results of the similarity measure calculation. The mechanisms perform question answering operations based on an analysis of one or more candidate answer variants.