RAG Relevancy Annotation for More Accurate Content Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for content retrieval and presentation lack optimal allocation and relevancy annotation, leading to suboptimal user experiences in search and recommendation processes.
Innovation Solution
A Retrieval-Augmented Generation (RAG) system that utilizes large language models (LLMs) to generate intermediate relevancy annotations, which are then processed by statistical inference and allocation systems to optimize content presentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional keyword and vector search retrieval systems are used to produce candidate content items, then the system can retrieve content efficiently, but the allocation and relevancy annotation are suboptimal leading to poor user experience
Solution Approach 1:
The patent introduces large language models as an intermediary component between the traditional retrieval systems and the final content allocation. The LLMs generate semantic relevance annotations that bridge the gap between efficient keyword/vector search and accurate relevancy assessment, allowing the system to maintain retrieval efficiency while significantly improving annotation precision through the mediating role of generative AI
Solution Approach 2:
The system combines multiple retrieval methods (keyword search and vector search) with a new component (LLM-based semantic annotation) to create a composite content allocation system. This composite approach integrates the strengths of traditional retrieval efficiency with the emergent capabilities of generative AI for semantic understanding, achieving both productivity and measurement precision
2Measurement precision
If large language models are used to generate semantic relevance annotations, then the relevancy annotation accuracy is improved, but the system complexity increases due to integrating multiple retrieval systems and LLMs
Solution Approach 1:
The patent segments the content allocation system into distinct functional modules: keyword retrieval subsystem, vector search subsystem, LLM-based annotation subsystem, and final allocation subsystem. This segmentation allows each component to be optimized independently and simplifies the integration process, managing system complexity through modular architecture while maintaining high relevancy annotation accuracy
Solution Approach 2:
The large language models serve multiple functions within the system: they generate semantic relevance annotations, understand user intent, and bridge different retrieval methodologies. This multi-functionality reduces the need for separate specialized components, thereby managing system complexity while achieving high measurement precision through a versatile core component
3Productivity
If the system generates intermediate output for statistical inference and allocation, then the content allocation is optimized, but the processing time and computational resources increase
Solution Approach 1:
The system performs preliminary retrieval of candidate content items using efficient keyword and vector search methods before applying the computationally intensive LLM-based semantic annotation. This preliminary action filters the content pool to a manageable subset, reducing the overall processing time while maintaining allocation optimization through subsequent detailed semantic analysis of only the most promising candidates
Data Source
AI summary
Methods and systems provide content searching and retrieval using generative artificial intelligence (AI) Models. The system is configured to receive a user search for content, media or item listings. The system receives a natural language-based input associated with a client device of a user. The system generates a search criterion for the received natural language-based input. The system provides a data set of retrieved content items to one or more large language models that annotate each of the content items in the data set. The system receives a new data set with the content items each including a relevancy annotation. Based on the relevancy annotations in the new data the system determines what additional processing to perform.