LLM In-Context Learning via Semantic Retrieval and CRF Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) face challenges such as biased results due to training data and annotator biases, uncontrollable and unexplainable generative content, and ethical and legal risks in certain applications. Additionally, LLMs may struggle with understanding local culture and customs, leading to inappropriate suggestions.
Innovation Solution
The system enhances LLMs by using in-context learning combined with online knowledge. It employs a semantic retrieval component to extract information from online sources, generating in-context learning inputs for the LLM. This system uses Simple Contrastive Learning of Sentence Embeddings (simCSE), tagging, and semantic searching to refine candidate knowledge and constrain responses within a semantic knowledge scope using a Conditional Random Field (CRF) model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If LLMs use training data from diverse sources, then knowledge coverage is improved, but bias and uncontrollability increase
Solution Approach 1:
The patent extracts and filters biased information from training data through a filtering component that uses CRF models to identify and remove harmful content, while preserving useful knowledge. This separation allows the system to access diverse knowledge sources without being contaminated by bias.
Solution Approach 2:
The patent introduces an intermediary filtering layer between the LLM and training data that acts as a mediator. This filtering component uses CRF models to intervene in the data flow, selectively allowing or blocking information based on predefined ethical and legal criteria, thus controlling the LLM's learning process.
2Adaptability or versatility
If LLMs generate content freely, then creativity is improved, but explainability and control decrease
Solution Approach 1:
The patent implements feedback mechanisms where the filtering component continuously monitors LLM outputs and provides feedback to adjust generation. This allows the system to maintain creativity while ensuring explainability through real-time monitoring and adjustment of generated content.
Solution Approach 2:
The patent performs preliminary actions by pre-defining ethical and legal criteria in the CRF models before the LLM generates content. This preliminary setup establishes clear guidelines that make the generation process explainable and controllable while preserving creative freedom within acceptable boundaries.
3Productivity
If LLMs respond quickly, then inferencing efficiency is improved, but accuracy and stability may decrease
Solution Approach 1:
The patent performs preliminary actions by pre-processing training data through filtering and organizing it into structured formats. This preparation allows the LLM to quickly access well-organized information during inference, achieving both speed and accuracy without compromising reliability.
Solution Approach 2:
The patent applies local quality by providing different levels of filtering and validation for different types of content and contexts. This allows the system to optimize inferencing speed for straightforward queries while maintaining high accuracy and stability for complex or sensitive topics through enhanced filtering.
Data Source
AI summary
One or more systems, devices, computer program products and/or computer-implemented methods of use provided herein relate to enhancing LLMs using in-context learning and online knowledge. The computer-implemented system can comprise a memory that can store computer executable components. The computer-implemented system can further comprise a processor that can execute the computer executable components stored in the memory, wherein the computer executable components can comprise a semantic retrieval component that can extract information from an online source, according to a query, to generate an in-context learning input utilized by an LLM for responding to the query.