LLM Verbal Input Paraphrasing for Classification Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current large language models (LLMs) face challenges in correctly addressing queries due to limitations in multi-class classification tasks, where generated text is often a single word or phrase, and sampling techniques are not suitable for classification tasks.
Innovation Solution
A method and apparatus that involve obtaining a verbal input, assigning a meaning classification and confidence score, generating paraphrases using an LLM if the confidence score is low, concatenating the input and paraphrases with their classifications and scores, and selecting the final interpreted input based on the LLM's output.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If LLMs are used for straightforward classification tasks with single-word output, then the task simplicity is maintained, but the classification accuracy deteriorates due to inability to capture nuanced meanings
Solution Approach 1:
The patent segments the classification process into multiple stages: initial classification of the verbal input, generation of paraphrases when confidence is low, classification of each paraphrase, and aggregation of results. This segmentation allows the system to maintain task simplicity while improving accuracy through multiple classification opportunities.
Solution Approach 2:
The system performs preliminary actions by generating paraphrases before final classification when the initial confidence score is low. This preliminary paraphrasing action enables the system to explore alternative interpretations before committing to a final classification, thereby improving accuracy without complicating the overall task structure.
2Adaptability or versatility
If sampling techniques are used for diverse generation, then the diversity of output is improved, but the suitability for classification tasks deteriorates because words with highest probabilities should be selected
Solution Approach 1:
The patent introduces paraphrases as an intermediary between the original verbal input and the final classification. Instead of directly sampling from the original input's probability distribution, the system generates paraphrases that serve as intermediaries, allowing diverse exploration of meaning while ultimately selecting the classification with the highest aggregated probability.
Solution Approach 2:
The system changes the parameter being classified by transforming the original verbal input into multiple paraphrased versions. This parameter change allows the system to explore different semantic representations while maintaining the ultimate goal of selecting the most probable classification, thus reconciling diversity with classification suitability.
3Measurement precision
If paraphrases are generated and aggregated for low-confidence inputs, then the classification accuracy is improved, but the processing time and computational complexity increase
Solution Approach 1:
The patent applies partial action by generating paraphrases only when the initial confidence score is below a threshold. For high-confidence inputs, the system skips the paraphrasing step entirely, thus avoiding unnecessary processing time. This selective application of paraphrasing ensures accuracy improvement only where needed, minimizing time loss.
4Reliability
If multiple paraphrases are generated and classified, then the misclassification errors are reduced, but the device complexity increases due to additional processing steps
Solution Approach 1:
The patent segments the complex classification process into manageable sub-tasks: initial classification, confidence assessment, selective paraphrase generation, individual paraphrase classification, and result aggregation. This segmentation reduces device complexity by organizing the processing into distinct, modular steps that can be independently optimized and managed.
Data Source
AI summary
A method of interpreting a verbal input, may include: assigning a meaning classification to the verbal input, and a confidence score to the meaning classification; and based on the confidence score corresponding to the meaning classification of the verbal input being less than or equal to a threshold, generating at least one paraphrase of the verbal input using at least one large language model (LLM); assigning the meaning classification to the at least one paraphrase, and the confidence score to the meaning classification; and concatenating the verbal input, the at least one paraphrase, the meaning classification, and the confidence score to generate a concatenated input; inputting the concatenated input into the at least one LLM.