Distributed Clustering Model Training via Mini-Batch Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The Dirichlet process Gaussian mixture model (DPGMM) is inefficient for clustering large datasets due to its high modeling complexity, leading to prohibitive clustering times and difficulties in parallelization with distributed data.
Innovation Solution
A clustering model training method that computes a mean vector, inverse precision matrix, and responsibility parameter vector, updates beta distribution and normal-Wishart distribution parameters using mini-batches, and iteratively converges the responsibility parameter vector to determine cluster membership, utilizing stochastic variational Bayesian inference and a linked model with common covariance for clusters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If Dirichlet process Gaussian mixture model is used for clustering, then clustering accuracy is improved, but clustering time becomes prohibitive for large datasets
Solution Approach 1:
The patent divides the large dataset into multiple mini-batches and processes them in parallel across distributed computing nodes. Each node performs stochastic variational Bayesian inference on its assigned mini-batch, and results are aggregated iteratively until convergence. This segmentation enables the algorithm to scale to large datasets while maintaining clustering accuracy.
Solution Approach 2:
The patent employs stochastic variational Bayesian inference with dynamic parameter updates. Instead of fixed iterative updates, the algorithm uses stochastic gradients with mini-batches to dynamically adjust responsibility parameters and model parameters (mean vectors, precision matrices) during training. This dynamic approach accelerates convergence compared to traditional batch processing.
2Adaptability or versatility
If Dirichlet process Gaussian mixture model is applied to big data, then clustering capability is enhanced, but computational complexity increases prohibitively
Solution Approach 1:
The patent segments the computational workload by distributing mini-batch processing across multiple computing nodes in a distributed system. Each node independently computes stochastic gradients and updates local model parameters, reducing the computational burden on any single machine and enabling scalable processing of big data.
Solution Approach 2:
The patent changes the optimization approach from exact batch inference to stochastic variational Bayesian inference with mini-batches. This parameter change in the computational method reduces per-iteration complexity while maintaining asymptotic convergence to the same solution, making the algorithm feasible for large-scale datasets.
3Measurement precision
If traditional DPGMM is used, then model accuracy is maintained, but parallelization with distributed data becomes difficult
Solution Approach 1:
The patent divides the dataset into distributed mini-batches that can be independently processed by different computing nodes. The stochastic variational Bayesian framework naturally supports this segmentation by using mini-batch stochastic gradients, allowing parallel execution while maintaining model accuracy through iterative aggregation of results across nodes.
Solution Approach 2:
The patent implements iterative feedback loops where model parameters are updated based on aggregated results from distributed nodes. Responsibility parameters and model parameters are refined through multiple rounds of stochastic gradient updates, with convergence determined by feedback on parameter changes. This feedback mechanism ensures accuracy is maintained while enabling parallelization.
Data Source
AI summary
A computing system trains a clustering model. (A) Beta distribution parameter values are computed for each cluster using a mass parameter value and a responsibility parameter vector of each observation vector. (B) Parameter values are computed for a normal-Wishart distribution for each observation vector included in a batch of a plurality of observation vectors. (C) Each responsibility parameter vector defined for each observation vector of the batch is updated using the beta distribution parameter values, the parameter values for the normal-Wishart distribution, and a respective observation vector of the selected batch of plurality of observation vectors. (D) A convergence parameter value is computed. (E) (A) to (D) are repeated until the convergence parameter value indicates the responsibility parameter vector defined for each observation vector is converged. A cluster membership is determined for each observation vector using the responsibility parameter vector. The determined cluster membership is output for each observation vector.