Multi-Hop QA Retrieval With Knowledge Graph Reasoning for LLMs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) struggle to provide satisfactory performance for multi-hop questions due to entangled knowledge retrieval and reasoning operations, as they are trained for general applicability and fail to disentangle the necessary information gathering and processing required for these complex queries.
Innovation Solution
The implementation of a multi-hop question answer retrieval and reasoning (MQARR) system that disentangles knowledge retrieval and reasoning operations by generating sub-questions, building a relevant knowledge graph, and using it as a structured intermediary search space to answer multi-hop questions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If LLMs are trained for general applicability across various domains, then flexibility and broad knowledge coverage are improved, but performance on specific complex tasks like multi-hop questions deteriorates due to entangled knowledge retrieval and reasoning operations
Solution Approach 1:
The patent segments the multi-hop question answering process into distinct components: knowledge retrieval (obtaining facts from knowledge base) and reasoning operations (generating affirmative sentences and reasoning sentences). This segmentation allows each component to be optimized independently, resolving the contradiction between general flexibility and specialized performance.
Solution Approach 2:
The patent introduces an intermediary search space consisting of affirmative sentences and reasoning sentences that bridge the gap between raw knowledge base facts and final answers. This intermediary structure enables systematic reasoning over multi-hop questions while maintaining the flexibility of general-purpose LLMs.
2Adaptability or versatility
If LLMs process multi-hop questions using general-purpose training, then broad applicability is maintained, but accuracy deteriorates due to inability to disentangle information gathering and processing
Solution Approach 1:
The patent divides the question answering process into discrete steps with clear boundaries: entity extraction, sub-question generation, knowledge retrieval, affirmative sentence generation, and reasoning sentence generation. This segmentation enables precise control over each operation while maintaining overall system flexibility.
Solution Approach 2:
The patent performs preliminary actions by generating sub-questions and retrieving relevant knowledge before attempting to answer the main multi-hop question. This preliminary processing disentangles information gathering from reasoning, improving accuracy without sacrificing broad applicability.
3Measurement precision
If a structured intermediary search space with knowledge graphs is introduced to improve multi-hop question answering, then answer accuracy is improved, but system complexity increases
Solution Approach 1:
The patent uses an intermediary search space with affirmative sentences and reasoning sentences as a mediator between the knowledge base and final answers. This intermediary structure organizes information systematically, improving accuracy while keeping the added complexity manageable through clear structural boundaries.
Solution Approach 2:
The patent segments the complex reasoning process into manageable components that can be processed sequentially. Each segment (entity extraction, sub-question generation, knowledge retrieval, sentence generation) handles a specific aspect, making the overall complex system more tractable and maintainable.
Data Source
AI summary
Mechanisms are provided for answering a multi-hop question. The mechanisms extract one or more entities included in the multi-hop question and generate, for each entity, a plurality of sub-questions to help answer the multi-hop question. The mechanisms obtain an answer to each sub-question from a knowledge base to convert each pair of the answer and the sub-question into each affirmative sentence. The mechanisms generate one or more reasoning sentences to answer the multi-hop question by using one or more affirmative sentences and determine whether the multi-hop question is answerable or not by using the one or more reasoning sentences. The mechanisms, in response to a positive determination, output an answer to the multi-hop question by using the one or more affirmative sentences and the one or more reasoning sentences.