LLM Verifier Ranking for More Accurate Reasoning Answers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The training and use of large language models (LLMs) for verification tasks are challenging due to the scarcity of comprehensive training data and limitations during the inference stage, leading to inaccurate verification results.
Innovation Solution
A data pipeline framework is provided for training and inferencing a verifier LLM to improve accuracy by using a training dataset that includes both correct and incorrect solutions, and integrating language and code solutions, with a method to convert input data formats before verification, and scoring solutions based on their probabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a verifier LLM is used to verify LLM-generated answers, then the accuracy of verification improves, but the complexity of the system increases due to requiring multiple LLMs and coordination between them
Solution Approach 1:
The patent combines the generator LLM and verifier LLM into a coordinated system where the generator produces candidate answers and the verifier assesses their correctness. This merging of functions allows the system to achieve high verification accuracy while managing complexity through integrated design.
Solution Approach 2:
The verifier LLM acts as an intermediary between the generated answer and the final output, assessing correctness before presenting results to users. This intermediary role improves verification accuracy while maintaining system organization through clear functional separation.
2Measurement precision
If comprehensive training data including both correct and incorrect solutions is used, then the verifier LLM accuracy improves, but the data processing complexity and time increase
Solution Approach 1:
The system performs preliminary actions by pre-processing training data to identify and categorize correct and incorrect solutions before verifier training. This preparation reduces the complexity of the actual training process while maintaining high verifier accuracy.
Solution Approach 2:
The training data is segmented into distinct categories (correct solutions and incorrect solutions with errors), allowing the verifier to learn from structured examples. This segmentation improves training efficiency while maintaining comprehensive coverage for accurate verification.
3Measurement precision
If multiple candidate solutions are generated and verified, then the quality of the final answer improves, but the computational resources and time required increase
Solution Approach 1:
The system generates multiple candidate solutions (excessive action) but verifies only the most promising ones based on initial filtering. This partial verification approach maintains high answer quality while improving processing efficiency by avoiding unnecessary verification of all candidates.
Solution Approach 2:
The verifier provides feedback on candidate solution quality, allowing the system to prioritize verification resources on the most promising candidates. This feedback mechanism improves processing efficiency while maintaining high answer quality through intelligent resource allocation.
Data Source
AI summary
A method for building an artificial intelligence (AI) agent. The method includes: receiving a training query; generating, by a first neural network based language model, a training dataset comprising a correct solution and an incorrect solution to the training query; generating, by a second neural network based language model, a first candidate score in response to the correct solution and a second candidate score in response to the incorrect solution; and training the second neural network based language model, based on a training objective. The method also includes building, at a server, an AI agent through a first application programming interface (API) to a third neural network based language model configured to generate a plurality of candidate solutions in response to the user utterance, and through a second API to the trained second neural network based language model configured to generate scores conditioned on the plurality of candidate solutions; ranking.