Text Detection Model Using Confidence Thresholds and Generative Reasoning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting the matching degree between answers and queries in large language models often suffer from overfitting, leading to inaccurate results when input deviates slightly from the training data distribution, and fail to effectively utilize deep features like logical correlation.
Innovation Solution
A method involving the use of a discriminative model to detect text pairs with low confidence levels, followed by a generative model trained with discrimination results and reasons to improve accuracy, utilizing deep features such as logical correlation between query and answer texts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single discriminative model is used for text detection, then the detection process is simple and fast, but the accuracy deteriorates when input deviates from training data distribution due to overfitting
Solution Approach 1:
The detection system is segmented into two specialized models: a discriminative model for initial detection and a generative model for reasoning. Each model focuses on specific strengths, preventing overfitting while maintaining accuracy. The discriminative model handles pattern recognition while the generative model handles logical reasoning, dividing the detection task into specialized components.
Solution Approach 2:
A confidence level threshold acts as an intermediary mechanism between the discriminative model and generative model. When the discriminative model's confidence is below the threshold, the system invokes the generative model for secondary detection. This intermediary approach ensures accurate detection for difficult cases without unnecessarily complicating simple detections.
2Measurement precision
If a discriminative model is used for text detection, then the detection speed is fast, but the ability to utilize deep features like logical correlation is insufficient
Solution Approach 1:
The system applies partial action by using the generative model only when necessary (when confidence level is below threshold). This avoids the excessive computational cost of always using the generative model while ensuring deep feature utilization for difficult cases. The discriminative model handles most cases efficiently, and the generative model provides enhanced analysis only when needed.
3Reliability
If the generative model processes all text pairs, then the detection accuracy is high, but the computational cost and time consumption increase significantly
Solution Approach 1:
The system applies local quality by assigning different processing qualities to different text pairs based on their confidence levels. High-confidence text pairs receive quick discriminative model processing, while low-confidence text pairs receive thorough generative model processing. This localized approach ensures high accuracy for difficult cases while maintaining fast processing for easy cases.
4Adaptability or versatility
If a single model is used for text detection, then the system is simple to implement, but it suffers from overfitting and inaccurate results when input deviates from training data
Solution Approach 1:
The system uses confidence level as feedback to dynamically adjust processing depth. When the discriminative model produces low-confidence results, the system automatically triggers the generative model for secondary detection. This feedback mechanism enables the system to adapt to different input complexities, improving generalization capability while maintaining reasonable system complexity through conditional processing.
Data Source
AI summary
A method of detecting a text, a training method, a device, and a medium are provided, which relate to a field of artificial intelligence, in particular to large language models, deep learning, text processing, etc. The training method includes: when a confidence level of a first sample detection result output by a first model is less than a threshold, processing a sample text pair and a sample label by using an initial model to obtain a second sample detection result, the second sample detection result includes a discrimination result configured to characterize a response matching degree between a sample answer text and a sample query text included in the sample text pair, and a discrimination reason; obtaining a first loss value according to the second sample detection result and the sample label; and adjusting a model parameter of the initial model based on the first loss value.