Probabilistic Generative Model Cluster Deletion for Text Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for processing text struggle to capture semantic meaning due to the formation of small, non-generalizable clusters of conceptually related words during the training phase of probabilistic generative models, leading to reduced processing efficiency and memory consumption without providing semantic benefits.
Innovation Solution
A system that selectively deletes less-useful clusters from a probabilistic generative model by determining the number of outgoing links and firing frequency of cluster nodes, and iteratively updates the model using expectation maximization to introduce new cluster nodes and links, thereby improving cluster quality and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If clusters of conceptually related words are formed during training phase, then semantic meaning is captured, but small non-generalizable clusters reduce processing efficiency and consume memory
Solution Approach 1:
The patent changes the parameter of cluster size by deleting small clusters that do not meet a minimum size threshold. This removes non-generalizable clusters that consume memory and reduce processing efficiency while preserving larger, more meaningful clusters that capture semantic meaning effectively.
Solution Approach 2:
The patent discards small, non-useful clusters during the deletion phase, removing them from the generative model. This selective discarding improves processing efficiency and memory usage by eliminating clusters that do not contribute meaningfully to semantic understanding.
2Quantity of substance
If small clusters are present in the generative model, then memory is consumed, but they do not provide semantic benefits
Solution Approach 1:
The patent applies a minimum size threshold parameter to filter clusters. Clusters smaller than this threshold are deleted, directly reducing memory consumption while ensuring that only clusters with sufficient semantic meaning (adequate size) are retained in the model.
3Productivity
If small clusters are present in the generative model, then processing efficiency is reduced, but no semantic benefits are provided
Solution Approach 1:
The patent uses a minimum cluster size parameter to eliminate small clusters that hinder processing efficiency. By setting and enforcing this threshold, the system improves productivity while maintaining semantic quality through the retention of larger, more meaningful clusters.
Solution Approach 2:
The patent systematically discards small clusters that do not meet the minimum size requirement. This removal process eliminates computational overhead from ineffective clusters while preserving the semantic integrity of the model through retention of quality clusters.
4Device complexity
If clusters with few outgoing links are present, then model complexity increases, but they do not contribute to semantic meaning
Solution Approach 1:
The patent applies a minimum outgoing links threshold parameter to evaluate cluster usefulness. Clusters with fewer outgoing links than this threshold are deleted, reducing model complexity by removing clusters that do not sufficiently connect to and contribute semantic meaning through other words or clusters.
Data Source
AI summary
One embodiment of the present invention provides a system that selectively deletes clusters of conceptually-related words from a probabilistic generative model for textual documents. During operation, the system receives a current model, which contains terminal nodes representing random variables for words and contains one or more cluster nodes representing clusters of conceptually related words. Nodes in the current model are coupled together by weighted links, so that if an incoming link from a node that has fired causes a cluster node to fire with a probability proportionate to a weight of the incoming link, an outgoing link from the cluster node to another node causes the other node to fire with a probability proportionate to the weight of the outgoing link. Next, the system processes a given cluster node in the current model for possible deletion. This involves determining a number of outgoing links from the given cluster node to terminal nodes or cluster nodes in the current model. If the determined number of outgoing links is less than a minimum value, or if the frequency with which the given cluster node fires is less than a minimum frequency, the system deletes the given cluster node from the current model.