ML Output Verification Using Reference Data Attestation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generative machine learning models, such as large language models, often produce outputs that are incorrect, non-responsive, or not accurate, lacking attestation to their veracity, which can lead to misleading information.
Innovation Solution
A system and method to verify the veracity of model outputs by parsing them into verifiable statements, using additional trained models and reference data to corroborate their accuracy, and generating verification scores or counterfactuals to assess reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If generative machine learning models produce outputs without verification, then productivity is improved, but reliability deteriorates due to hallucinations and inaccurate information
Solution Approach 1:
The patent introduces a verification system as an intermediary component between the generative ML model and the user. This verification system includes a parser that breaks down model outputs into verifiable statements, a verification module that checks these statements against reference data, and a scorer that generates veracity scores. This intermediary layer maintains high productivity by allowing rapid generation while ensuring reliability through systematic verification of the generated content.
2Reliability
If verification processes are implemented to improve reliability, then veracity of outputs is improved, but device complexity increases due to additional models and data processing
Solution Approach 1:
The verification system is segmented into distinct functional modules: a parser module that divides model outputs into individual statements, a verification module that checks each statement against reference data, and a scoring module that aggregates verification results. This segmentation allows the complex verification process to be managed through specialized, independent components rather than a monolithic system, reducing overall complexity while maintaining reliability.
Solution Approach 2:
The patent introduces a verification system as an intermediary component between the generative ML model and the user. This verification system includes a parser that breaks down model outputs into verifiable statements, a verification module that checks these statements against reference data, and a scorer that generates veracity scores. This intermediary layer maintains high productivity by allowing rapid generation while ensuring reliability through systematic verification of the generated content.
3Measurement precision
If multiple datastores are queried to verify unverified statements, then measurement precision is improved, but loss of time increases due to additional data access operations
Solution Approach 1:
The verification system implements partial action by selectively verifying only the necessary statements. The parser identifies verifiable statements, and the verification module checks these against reference data. If statements are already verified or cannot be verified with available data, the system moves on without exhaustive checking of all possible data sources. This approach achieves sufficient verification accuracy without the time cost of exhaustive verification.
Solution Approach 2:
The system performs preliminary actions by first attempting verification with readily available reference data before querying additional datastores. The verification module checks statements against existing reference data, and only when verification fails does it proceed to query additional datastores. This preliminary verification step reduces the overall time required while maintaining accuracy by ensuring thorough verification when necessary.
Data Source
AI summary
The techniques described herein relate to techniques for verifying veracity of machine learning outputs. An example method includes receiving input comprising one or more verifiable statements in text, verifying, using first reference data stored in at least one first datastore, the one or more verifiable statements to produce first verification results indicating which of the one or more verifiable statements has been verified, when it is determined that at least one of the one or more verifiable statements remains unverified based on the first verification results, identifying at least one second datastore having second reference data attesting to veracity of the input, and verifying, using the second reference data, the at least one unverified statement to produce second verification results, and providing output indicating whether one or more of the one or more verifiable statements have been verified based on at least one of the first or second verification results.