Language Model Prompt Compression for Topic Labeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current topic modeling techniques face limitations in efficiently processing large volumes of unstructured data to generate meaningful inferences and topic descriptions.
Innovation Solution
A system and method involving an unsupervised machine learning model to generate topics from a set of documents, selecting and compressing topic terms using weight values and inverse document frequency, and inputting these into a language model to produce topic labels and descriptions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If all topic terms are included in the prompt for language models, then the completeness of topic information is improved, but the prompt length and processing complexity increase
Solution Approach 1:
The patent extracts only the most important topic terms from the complete set of topic terms. It uses weighting mechanisms (TF-IDF, topic model weights) to identify and extract the subset of topic terms that contribute most to representing each topic, thereby reducing prompt length while maintaining information completeness.
Solution Approach 2:
The patent applies different quality criteria to different topic terms based on their importance. High-weight topic terms are retained in the prompt while low-weight terms are excluded. This creates a non-uniform selection where each topic term is evaluated individually based on its local contribution to topic representation.
2Measurement precision
If comprehensive topic analysis is performed on large volumes of unstructured data, then the accuracy of topic modeling is improved, but the processing time and computational resources increase
Solution Approach 1:
The patent extracts essential features from large volumes of unstructured data through automated topic term extraction and weighting. By focusing computational resources on identifying and analyzing only the most significant topic terms rather than processing all text data equally, it achieves accurate topic modeling with reduced processing time.
Solution Approach 2:
The patent transforms the problem by changing parameters from analyzing all text data to analyzing weighted topic term frequencies. It uses mathematical transformations (TF-IDF weighting, topic model probability distributions) to convert unstructured text into structured numerical representations that can be processed more efficiently while maintaining analytical accuracy.
3Manufacturing precision
If multiple weighting mechanisms are applied to select topic terms, then the precision of topic representation is improved, but the computational complexity increases
Solution Approach 1:
The patent merges multiple weighting mechanisms (TF-IDF weighting, topic model weights, frequency analysis) into a unified topic term selection process. By combining these different weighting approaches, it achieves more precise topic representation that accounts for both term importance within documents and term discriminative power across the corpus, while managing computational complexity through integrated processing.
Data Source
AI summary
A data processing system and method include receiving a set of documents having unstructured data, executing the unsupervised machine learning model for outputting topics, selecting a first subset of topic terms, computing an inverse document frequency weight value for each topic term in the first subset of topic terms, computing a second weight value for each topic term in the first subset of topic terms, selecting a second subset of topic terms from the first subset of topic terms, generating a compressed representation of the set of documents from the second subset of topic terms to include in a prompt, inputting the prompt into a language model, and executing the language model based on the prompt to generate the topic label and the topic description.