Dual-Adapter Embedding for Precise Question-Chunk Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models, particularly large language models, face challenges in embedding questions and corresponding document chunks effectively, leading to discrepancies and hallucinations due to the use of a single adapter for both, which hinders precise retrieval and generation tasks.
Innovation Solution
Employing dual adapter-based embedding by using separate question and document adapters to generate embeddings, with contrastive learning techniques to draw positive pairs closer and push negative pairs apart in the embedding space, reducing discrepancies and enhancing retrieval accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single adapter is used for both question and document chunk embedding, then the model structure is simplified, but retrieval precision deteriorates due to embedding discrepancies and hallucinations
Solution Approach 1:
The patent divides the single adapter into two separate adapters: a question adapter for embedding questions and a document adapter for embedding document chunks. This segmentation allows each adapter to be optimized for its specific input type, resolving the contradiction by sacrificing minor structural complexity to achieve superior retrieval precision and eliminate hallucinations.
Solution Approach 2:
The patent applies local quality by giving different adapters different optimization objectives and training data. The question adapter focuses on question understanding while the document adapter focuses on document content representation. This localized optimization ensures each component performs its specific function optimally, improving overall retrieval precision despite the increased structural complexity.
2Measurement precision
If contrastive learning is applied to draw positive pairs closer and push negative pairs apart, then retrieval accuracy is enhanced, but computational complexity increases
Solution Approach 1:
The patent implements contrastive learning with feedback mechanisms that use labeled pairs of questions and document chunks. Positive pairs (relevant matches) are drawn closer in embedding space while negative pairs (irrelevant matches) are pushed apart. This feedback-driven approach enhances retrieval accuracy by continuously optimizing the embedding representations based on relevance judgments.
Solution Approach 2:
The patent changes the optimization parameters and learning objectives during training to accommodate contrastive learning. By adjusting loss functions, learning rates, and optimization strategies, the system achieves high retrieval accuracy. The parameter changes enable the model to handle the increased computational complexity efficiently through techniques like batch processing and gradient descent optimization.
Data Source
AI summary
Systems and techniques are provided for retrieving data. For example, a method can include obtaining, using a question adapted embedding model, a question, the question adapted embedding model being configured to embed one or more questions into an embedding space, generating, using the question adapted embedding model, a question embedding based on the question, determining, from an embedding space comprising a plurality of chunk embeddings, one or more chunk embeddings associated with the question embedding, and retrieving one or more chunks associated with the one or more chunk embeddings. The plurality of chunk embeddings can be generated by a document adapted embedding model.