RAG Passage Association Retrieval for More Self-Contained Responses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In retrieval-augmented generation, the retrieved text group is often not self-contained, leading to suboptimal quality of the finally generated response.
Innovation Solution
An information processing apparatus and method that calculates and stores association information between passages using a language model, allowing for retrieval of additional passages based on this information to enhance the quality of the generated response.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If only initial passage retrieval is performed without considering passage associations, then retrieval processing is simple and fast, but the retrieved text group is not self-contained and response quality is poor
Solution Approach 1:
The system pre-calculates and stores association information between passages before retrieval operations. This preliminary action creates a ready-to-use association structure that enables enhanced retrieval without adding computational complexity during the actual retrieval process, thus improving self-containedness while maintaining processing simplicity
Solution Approach 2:
Association information serves as an intermediary element that connects initial passages with additional relevant passages. This intermediary structure enables the retrieval system to automatically identify and incorporate supplementary passages without requiring complex real-time analysis, thereby improving the completeness of the retrieved text group
2Reliability
If association information is calculated and stored for all passages, then additional passages can be retrieved to improve response quality, but processing time and computational resources increase
Solution Approach 1:
Association information between passages is pre-calculated and stored in advance, transforming a potentially time-consuming runtime operation into a static data structure. This allows the retrieval system to quickly access passage associations without performing complex calculations during query processing, thus improving response quality while minimizing time loss
Solution Approach 2:
The system dynamically adjusts the retrieval process by using pre-stored association information to selectively retrieve additional passages based on the specific query and initial passage results. This dynamic approach ensures that only necessary additional passages are retrieved, optimizing the balance between response quality and processing time
Data Source
AI summary
An object of the present disclosure is to provide a technology capable of improving quality of a finally generated response in retrieval-augmented generation and supporting decision making. An information processing apparatus includes: a first acquisition unit configured to acquire a query; a first retrieval unit configured to retrieve an initial passage related to the query from a passage set including a plurality of passages; a second retrieval unit configured to retrieve an additional passage from the passage set with reference to association information including a strength of association between the passages included in the passage set, and the initial passage; and a third retrieval unit configured to perform retrieval processing using the initial passage and the additional passage.