LLM Question Answering With Type-Specific Chunking and Dictionary Context
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing large language models (LLMs) used in chatbots often provide inaccurate answers due to the possibility of presenting incorrect information, and the trade-offs in document chunking for retrieval-augmented generation (RAG) techniques lead to reduced accuracy, especially in documents with low section correlation, such as command manuals.
Innovation Solution
A system that cuts documents into chunks based on predefined overlap amounts specific to document types, associates these chunks with dictionaries, and uses an LLM to generate answers by inputting user questions along with identified document and dictionary chunks, enhancing accuracy through Retrieval-Augmented Generation (RAG).
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the overlap amount between chunks is increased to improve answer accuracy, then the connection between sentences is improved, but noise with respect to the question increases
Solution Approach 1:
The patent applies local quality by setting different overlap amounts for different document types. Documents with high section correlation (e.g., narratives) use larger overlap amounts to maintain sentence connectivity, while documents with low section correlation (e.g., command manuals) use smaller or zero overlap amounts to avoid noise. This localized parameter adjustment resolves the contradiction by optimizing overlap for each document's specific characteristics.
Solution Approach 2:
The patent changes the overlap amount parameter based on document type classification. By dynamically adjusting this parameter rather than using a fixed value, the system achieves both reduced noise for low-correlation documents and maintained sentence connectivity for high-correlation documents, thereby resolving the accuracy-noise trade-off.
2Object-affected harmful factors
If the overlap amount between chunks is decreased to reduce noise, then the noise with respect to the question is reduced, but the connection between sentences is lost and answer accuracy decreases
Solution Approach 1:
The patent applies local quality by setting different overlap amounts for different document types. Documents with high section correlation (e.g., narratives) use larger overlap amounts to maintain sentence connectivity, while documents with low section correlation (e.g., command manuals) use smaller or zero overlap amounts to avoid noise. This localized parameter adjustment resolves the contradiction by optimizing overlap for each document's specific characteristics.
Solution Approach 2:
The patent changes the overlap amount parameter based on document type classification. By dynamically adjusting this parameter rather than using a fixed value, the system achieves both reduced noise for low-correlation documents and maintained sentence connectivity for high-correlation documents, thereby resolving the accuracy-noise trade-off.
3Ease of operation
If LLM is used to provide natural language answers, then the responsiveness and natural language capability are improved, but the accuracy of answers decreases due to possibility of incorrect information
Solution Approach 1:
The patent merges RAG (Retrieval-Augmented Generation) with LLM by combining retrieved document chunks as context with the LLM's generative capabilities. This integration allows the LLM to ground its natural language answers in actual document content, significantly improving accuracy while maintaining the ease of natural language interaction that LLMs provide.
Solution Approach 2:
The patent introduces retrieved document chunks as an intermediary between the user question and the LLM. These chunks serve as evidence-based context that mediates the LLM's answer generation, ensuring that responses are grounded in actual document content rather than relying solely on the LLM's pre-trained knowledge, thereby improving accuracy.
Data Source
AI summary
An overlap amount is defined for each document type. For each document, a system cuts the document into a plurality of document chunks according to the overlap amount corresponding to the type of document, and if the document type of the document is a predetermined document type, for each document type, the system associates the document chunk with a dictionary chunk that contains descriptions about a word that matches a word in the document chunk. When the system receives a user question, the system generates and provides an answer by inputting the question, as well as a document chunk identified based on the question and a dictionary chunk which the document chunk is associated with, into an LLM.