大规模语言模型生成答案可靠性检测方法
By constructing a robust discriminator RelD and a comprehensive training dataset RelQA, and combining a pre-trained language model with a weighted average probability method, the limitations of large-scale language model-generated answer reliability detection are addressed, achieving more accurate and scalable detection results.
CN117390409BActive Publication Date: 2026-07-17FUDAN UNIVERSITY
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- FUDAN UNIVERSITY
- Filing Date
- 2023-09-17
- Publication Date
- 2026-07-17
AI Technical Summary
Technical Problem
Existing technologies have limitations in detecting the reliability of answers generated by large-scale language models. They cannot fully consider semantic and syntactic information, and are costly and lack scalability.
Method used
A robust discriminator, RelD, is constructed by building a comprehensive training dataset, RelQA, combining a pre-trained language model and a weighted average probability method to evaluate the reliability of the generated answers through binary classification. ELECTRA is used as the base model for training.
Benefits of technology
It provides more accurate, comprehensive and scalable answer reliability detection, applicable to different large-scale language models and application scenarios, improving the accuracy and robustness of detection.
✦ Generated by Eureka AI based on patent content.
Smart Images
Figure CN117390409B_ABST
Abstract
本发明属于大规模语言模型技术领域，具体为一种大规模语言模型生成答案可靠性检测方法。本发明通过构建一个鲁棒的判别器RelD来检测大规模语言模型生成答案的可靠性，包括构建训练数据集RelQA，该数据集包括现有多个数据集的问题、上下文和大规模语言模型生成的答案以及多种评估指标；将RelQA作为输入，结合预训练语言模型，使用加权平均概率方法拟合生成答案的人工标注，来训练判别器RelD；判别器RelD对大规模语言模型生成的答案进行二分类，以此判断生成的答案的可靠性。本发明能够提供更全面、准确的评估结果，更好地反映出生成答案的质量；可以适用于不同的大型语言模型和应用场景，具有较强的可扩展性。
Need to check novelty before this filing date? Find Prior Art