RAG Knowledge Base Image Retrieval for Accurate LLM Responses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Retrieval-Augmented Generation (RAG) chatbots lack the ability to provide graphical content to supplement their responses, relying on additional image generative AI models that incur processing costs and introduce inaccuracies.
Innovation Solution
Incorporating image path strings into the knowledge base of Large Language Models (LLMs) to replace images, enabling direct integration of images from the original reference files without the need for additional processing, thus providing accurate and efficient graphical content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If image generative AI models (e.g., diffusion models) are used to provide graphical content in chatbot responses, then graphical content capability is improved, but processing resources consumption increases and inaccuracies are introduced
Solution Approach 1:
The patent extracts images from the original reference files (knowledge base) and stores them separately with path strings. When a query is processed, the system retrieves relevant images from storage using the path strings rather than regenerating them. This extraction approach eliminates the need for computationally intensive image generation while maintaining graphical content capability.
Solution Approach 2:
The patent creates copies of images from the knowledge base and stores them in a separate retrieval storage system. These image copies are then retrieved and displayed in responses without requiring regeneration. The copying approach allows efficient retrieval of graphical content while avoiding the high processing costs of generative models.
2Adaptability or versatility
If image generative AI models are used to provide graphical content in chatbot responses, then graphical content capability is improved, but response accuracy deteriorates due to introduced inaccuracies
Solution Approach 1:
By extracting images directly from the original reference files (knowledge base) rather than generating them through AI models, the system eliminates the source of inaccuracies inherent in generative models. The images are taken verbatim from the authoritative source material, ensuring accuracy while maintaining graphical content capability.
Solution Approach 2:
The patent creates accurate copies of images from the knowledge base and stores them for retrieval. These copies faithfully represent the original images without introducing the hallucinations or inaccuracies that plague generative image models, thereby maintaining high response accuracy.
3Device complexity
If links to references are provided instead of integrated images, then implementation complexity is reduced, but user time consumption increases for reviewing references
Solution Approach 1:
The patent performs preliminary action by extracting and storing images from the knowledge base in advance. When a query is processed, the system can immediately retrieve and display relevant images without requiring users to manually search through reference files. This preliminary preparation eliminates the need for users to spend time reviewing references for images.
Solution Approach 2:
The patent merges the image retrieval function with the existing RAG query processing flow. Images are automatically retrieved based on query relevance and integrated into responses alongside textual content. This merging approach provides graphical content without requiring separate reference review steps, reducing user time consumption while maintaining manageable implementation complexity.
Data Source
AI summary
Images are identified in reference files and removed from the reference files. Image data for the removed images are stored in at least one memory and the removed images are replaced with image path strings indicating storage locations in the at least one memory of image data corresponding to the removed images. The reference files are stored in a data storage device including image path strings for the removed images as a knowledge base for a Large Language Model (LLM) in responding to queries. In one aspect, a response to a query is received from the LLM and an image path string is identified in the response. Image data is retrieved from the at least one memory using the identified image path string, and the received response is displayed with an image replacing the identified image path string using the retrieved image data.