Hilbert Curve Partitioning for Parallel DBSCAN Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
DBSCAN clustering algorithm is not parallelizable across multiple computing nodes, leading to long running times on large data sets, and existing solutions for parallelization, such as Enhanced DBSCAN, can produce suboptimal results due to inappropriate partitioning and random initialization of cluster centers.
Innovation Solution
The use of Hilbert curve indexing to pre-process data sets, allowing for intelligent initialization of cluster centers and partitioning, which reduces the number of iterations required for partitional clustering algorithms like CLARANS, and enables parallel processing of DBSCAN by merging intermediate results from parallel partitions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If DBSCAN is applied to large data sets without parallelization, then clustering accuracy is maintained, but running time becomes excessively long
Solution Approach 1:
The patent divides the large data set into multiple partitions using Hilbert curve indexing, allowing DBSCAN to be applied in parallel to each partition. This segmentation enables the algorithm to process large data sets faster while maintaining clustering accuracy through subsequent merging of intermediate results.
Solution Approach 2:
The patent performs preliminary Hilbert curve indexing and partitioning before applying DBSCAN. This preliminary action organizes the data in a way that preserves spatial locality, enabling efficient parallel processing while maintaining the accuracy requirements of the original DBSCAN algorithm.
2Loss of time
If Enhanced DBSCAN is used for parallelization, then running time is reduced, but clustering accuracy deteriorates due to suboptimal partitioning and random initialization
Solution Approach 1:
The patent applies Hilbert curve indexing as a preliminary step to determine intelligent initial cluster centers based on data density distribution, rather than using random initialization. This preliminary action ensures that parallel processing maintains clustering accuracy while reducing running time.
Solution Approach 2:
The patent changes the initialization parameter from random selection to Hilbert curve-based density-aware selection. This parameter change improves clustering accuracy in parallel processing by ensuring that initial cluster centers are positioned in high-density regions, leading to better convergence.
3Productivity
If data is partitioned for parallel DBSCAN processing, then productivity increases, but partitioning quality may deteriorate leading to suboptimal clustering results
Solution Approach 1:
The patent uses Hilbert curve, a space-filling curve with specific geometric properties, to partition the data. This curved indexing method preserves spatial locality better than linear partitioning, ensuring that points close in the original space remain close in partitioned form, thus maintaining partitioning quality while enabling parallel processing.
Solution Approach 2:
The patent transforms the partitioning problem from a simple linear split into a multi-dimensional space-filling approach using Hilbert curve. This dimensionality transformation ensures that partitions maintain spatial coherence, preserving clustering quality while enabling parallel processing across multiple computing nodes.
Data Source
AI summary
DBSCAN clustering analyses can be improved by pre-processing of a data set using a Hilbert curve to intelligently identify the centers for initial partitional analysis by a partitional clustering algorithm such as CLARANS. Partitions output by the partitional clustering algorithm can be process by DBSCAN running in parallel before intermediate cluster results are merged.