LLM-Guided Text Classification Model With Explainable Labeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional classification models, particularly those using Machine Learning (ML), fail to provide clear, understandable explanations for their classification decisions, especially when dealing with complex data types such as text, lacking sufficient insights into the reasoning behind the classifications.
Innovation Solution
Utilize a Large Language Model (LLM) to generate textual reasonings that support pre-labeling of textual items, compile these into a Unified List of classification indicators, and create an Instruction Prompt to automatically train an ML classification model that provides human-readable explanations for classifications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If conventional Machine Learning classification models are used, then classification of textual items can be performed, but clear and understandable explanations for classification decisions are not provided
Solution Approach 1:
The patent introduces Large Language Models as an intermediary component between the classification model and the user. The LLM generates human-readable explanations that mediate between the model's internal decision-making process and user understanding, allowing the system to maintain classification accuracy while improving explainability without requiring changes to the core classification algorithm
Solution Approach 2:
The system segments the explanation generation task from the classification task itself. The classification model first performs the classification, then the LLM separately generates explanations based on the classification results and input data. This segmentation allows each component to specialize in its function while working together to solve the overall problem of providing both accurate classification and understandable explanations
2Manufacturing precision
If manual labeling and explanation creation is performed for training data, then high quality labeled datasets can be obtained, but significant manual intervention and time are required
Solution Approach 1:
The system enables self-service by using the LLM to automatically generate labels and explanations for training data without human intervention. The LLM processes unlabeled textual items, generates appropriate labels based on classification criteria, and creates explanatory text, allowing the system to prepare its own training data autonomously and scale efficiently
Solution Approach 2:
The patent applies preliminary action by pre-training the LLM on existing labeled datasets and explanation patterns before it is used to generate new training data. This preliminary training equips the LLM with the necessary skills to generate high-quality labels and explanations, ensuring that the automatically created training data meets quality standards without requiring manual verification of each item
3Measurement precision
If LLM is used to generate textual reasonings and automatically train ML models, then classification transparency and accuracy are enhanced, but computational resources and processing time increase
Solution Approach 1:
The system applies partial action by using the LLM to generate explanations and reasoning for only the most critical or ambiguous classification cases, rather than for every single item. This selective approach maintains high classification accuracy for important cases while reducing the overall computational burden and energy consumption of the system
Solution Approach 2:
The patent implements parameter changes by adjusting the LLM's generation parameters such as temperature, token limits, and explanation detail levels based on the specific classification task and data characteristics. These parameter optimizations allow the system to achieve the necessary classification accuracy while minimizing unnecessary computational energy expenditure
Data Source
AI summary
A computerized method includes: obtaining a first dataset of pre-labeled textual items, wherein each pre-labeled textual item is associated with a pre-label; feeding each of the pre-labeled textual items into a Large Language Model (LLM), and prompting it to generate textual reasoning that supports the pre-label of each pre-labeled textual item; collating the generated textual reasonings, and generating therefrom a textual instruction prompt; obtaining a second dataset of not-yet-labeled textual items; feeding each of the not-yet-labeled textual items into the LLM, and commanding it to utilize the textual instruction prompt and to generate a textual label for each of the not-yet-labeled textual items; collecting those textual items, that were labeled by the LLM, into a third dataset of LLM-labeled textual items; automatically training a Machine Language (ML) classification model on that third dataset of LLM-labeled textual items; deploying that ML classification model in a platform for classification of textual items.