Supporting-Evidence Extraction for Two-Hop NLI Explanations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Natural Language Inference (NLI) models struggle to accurately determine the inferential relationship between a premise and a hypothesis when insufficient information is provided, requiring follow-up hypotheses to be generated and verified.
Innovation Solution
A method is developed to train an inference model on two-hop NLI problems, using a sequence model to determine extraction starting positions and lengths from both premises and hypotheses, filling output slots with word combinations, and applying these to a single-hop NLI classifier to generate explanations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If simple entailment models use a single sentence or concatenated sentences to judge hypotheses, then the model complexity is low, but the information sufficiency is insufficient leading to unsupported outputs
Solution Approach 1:
The patent segments the NLI task into multiple hops, where each hop processes a specific premise-hypothesis pair. The two-hop NLI model divides the reasoning process into distinct stages: first processing the relationship between the first premise and hypothesis, then processing the relationship between the second premise and the intermediate hypothesis, ultimately determining the final inferential relationship. This segmentation allows the model to handle complex reasoning tasks while maintaining manageable complexity at each stage.
Solution Approach 2:
The patent introduces an intermediate hypothesis as a mediator between the premises and the final conclusion. The intermediate hypothesis serves as a bridge that connects the first premise to the second premise, allowing the model to verify background information and generate follow-up hypotheses systematically. This intermediary structure enables multi-step reasoning without requiring the entire complex relationship to be processed simultaneously.
2Measurement precision
If two-hop NLI models process multiple premises and hypotheses to verify background information, then the accuracy of inferential relationship determination is improved, but the computational complexity increases
Solution Approach 1:
The patent applies preliminary action by training the model on two-hop NLI problems before deploying it for single-hop NLI tasks. The training phase involves processing multiple premises and hypotheses to learn complex inferential relationships, extraction patterns, and reasoning strategies. Once trained, the model can efficiently handle simpler single-hop NLI problems, as the complex reasoning capabilities have already been established during training. This preliminary training on complex tasks prepares the model to handle accuracy-critical applications.
3Loss of information
If extraction output slots are filled with word combinations from premises and hypotheses, then the explainability of the model is improved, but the processing time increases
Solution Approach 1:
The patent extracts key words and phrases from premises and hypotheses by filling extraction output slots with relevant word combinations. This extraction process identifies and isolates the most important elements that contribute to the inferential relationship, creating concise explanations that capture the essential reasoning. By taking out only the critical words rather than processing entire sentences, the model achieves explainability while reducing the amount of data that needs to be processed.
Data Source
AI summary
A method trains an inference model on two-hop NLI problems that include a first and second premise and a hypothesis, and further includes generating, by the model using hypothesis reduction, an explanation from an input premise and an input hypothesis, for an input single hop NLI problem. The learning step determines a distribution over extraction starting positions and lengths from within the first premise and hypothesis of a two-hop NLI problem. The learning step k extraction output slots with combinations of words from the first premise of the two-hop NLI problem and fills another extraction output slots with combinations of words from the hypothesis of the two-hop NLI problem. The learning step trains a sequence model by using the extraction output slots and the other extraction output slots together with the second premise as an input to a single-hop NLI classifier to output a label of the two-hop NLI problem.