Distributable Clustering Model Training Using Variational Bayesian Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The Dirichlet process Gaussian mixture model (DPGMM) is inefficient for clustering large datasets due to its high modeling complexity, leading to prohibitive clustering times and difficulties in parallelization with distributed data, especially when processing big data.
Innovation Solution
A variational Bayesian (VB) inference method is employed to train a clustering model, using a computer-readable medium with instructions to compute mean vectors, inverse precision matrices, and responsibility parameter vectors, updating parameters iteratively until convergence, and determining cluster membership for unclassified observation vectors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If Dirichlet process Gaussian mixture model (DPGMM) is used for clustering, then clustering accuracy is improved, but clustering time becomes prohibitively long for large datasets
Solution Approach 1:
The patent segments the large dataset into multiple smaller partitions that can be processed independently and in parallel. Each partition is clustered separately using DPGMM, and then the results are merged. This segmentation allows the computationally intensive DPGMM algorithm to run faster on smaller data subsets while maintaining clustering accuracy through proper integration of results across partitions.
Solution Approach 2:
The patent performs preliminary actions by pre-processing the data to identify potential cluster structures and using these to guide the DPGMM clustering process. This preliminary analysis helps reduce the search space for the algorithm, allowing it to converge faster while maintaining accuracy on large datasets.
2Reliability
If DPGMM is applied to big data, then clustering efficacy is improved, but parallelization with distributed data becomes difficult
Solution Approach 1:
The patent divides the big data into multiple independent partitions that can be processed in parallel across distributed computing nodes. Each node runs DPGMM on its local partition independently, eliminating complex inter-node communication requirements while maintaining clustering efficacy through consistent model parameters and result integration strategies.
Solution Approach 2:
The patent designs a universal DPGMM implementation that can function both in single-node and distributed multi-node environments. The same core algorithm is used across all nodes, with standardized interfaces for data input and result output, making the system adaptable to different computing architectures without requiring complex parallelization logic.
3Measurement precision
If supervised learning is used to label training data, then prediction accuracy is improved, but complexity and cost increase due to required human labor
Solution Approach 1:
The patent applies semi-supervised learning by using only a small portion of labeled data to initialize the clustering model, then leveraging the unlabeled majority of the data to refine and improve the clustering results. This partial labeling approach achieves high prediction accuracy without requiring extensive human labeling effort, as the algorithm iteratively improves using the structure discovered in the unlabeled data.
Solution Approach 2:
The clustering algorithm performs self-service by automatically discovering cluster structures and assigning labels to data points based on the learned patterns. Once initialized with minimal labeled data, the system autonomously refines its own performance by iteratively improving cluster assignments using the unlabeled data, eliminating the need for continuous human intervention or extensive pre-labeling.
Data Source
AI summary
A computing system trains a clustering model. A responsibility parameter vector is initialized for each observation vector and includes a probability value of a cluster membership. The observation vectors include a plurality of classified observation vectors and a plurality of unclassified observation vectors. (A) Beta distribution parameter values are computed for each cluster. (B) Parameter values are computed for a normal-Wishart distribution for each cluster. (C) Each responsibility parameter vector is updated using the beta distribution parameter values, the parameter values, and a respective observation vector. (D) A convergence parameter value is computed. (E) (A) to (D) are repeated until the computed convergence parameter value indicates the responsibility parameter vector defined for each observation vector of the plurality of unclassified observation vectors is converged. A cluster membership is determined and output for each observation vector using a respective, updated responsibility parameter vector.