Automated Prompt Tuning With Feedback Loops for LLM Quality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning systems, such as large language models (LLMs), require inefficient manual reformatting of requests to improve prompt quality, lacking clear indicators of response improvement, and fail to effectively utilize user feedback for prompt tuning.
Innovation Solution
A prompt tuning system that uses objective and subjective measures, including evaluation scores based on metrics like factuality, coherence, completeness, and bias, to iteratively refine prompts, incorporating user feedback to update and improve LLM responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If manual reformatting of requests is used to improve prompt quality, then response quality may improve, but efficiency deteriorates due to time-consuming manual intervention
Solution Approach 1:
The system automatically evaluates and refines prompts using machine learning models without requiring manual reformatting. The LLM evaluates its own outputs and iteratively improves prompts based on evaluation scores, enabling self-service that eliminates manual intervention while maintaining or improving prompt quality.
Solution Approach 2:
The system implements an automated feedback loop where evaluation scores from machine learning models are fed back into the prompt generation process. This feedback mechanism allows the system to automatically adjust and improve prompts based on objective metrics, replacing manual reformatting with systematic automated refinement.
2Measurement precision
If automated evaluation metrics are implemented to track prompt quality, then measurement precision improves, but system complexity increases
Solution Approach 1:
The machine learning model serves multiple functions: it generates responses, evaluates prompt quality, and guides iterative improvement. This multi-functionality reduces the need for separate specialized components, managing system complexity while providing comprehensive automated evaluation across multiple metrics simultaneously.
3Reliability
If iterative prompt refinement is performed to meet quality thresholds, then response reliability improves, but processing time increases
Solution Approach 1:
The system performs preliminary evaluation of prompts before final generation, identifying quality issues early in the process. By evaluating and refining prompts iteratively before commitment to final output, the system ensures quality thresholds are met while minimizing rework, balancing reliability with time efficiency.
Data Source
AI summary
Disclosed herein are system, method, and computer program product embodiments for feedback based learning and automated prompt tuning. A system queries a large language model (LLM) with a natural language prompt request to obtain a first prompt responsive to the natural language prompt request. The system then generates an evaluation score for the first prompt via a first machine learning model. The system then obtains a second prompt generated by the LLM responsive to the natural language prompt request, the first prompt, and the evaluation score. The system identifies a review for the second prompt via a second machine learning model. The system then obtains a third prompt generated by the LLM responsive at least to the natural language prompt request, the second prompt, the evaluation score, and the review for the second prompt.