Parallel Data Clustering via Sample Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Parallel processing of large-scale data clustering analytics is hindered by computational overhead, such as data copying and synchronization across processors, making it impractical for certain types of computational tasks, especially those requiring comparisons between large and small datasets.
Innovation Solution
A system that divides samples and attributes into subsets corresponding to processing cores, allowing for parallel comparison of samples with clusters, and merges results to populate clusters efficiently, utilizing a cluster selector, attribute divider, joint operator, and merger to reassess and stabilize cluster assignments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If parallel processing is used to process large datasets, then processing speed is improved, but computational overhead increases due to data copying and synchronization
Solution Approach 1:
The patent segments the large dataset into multiple smaller datasets that can be distributed across different processing cores. Each processing core receives a specific subset of data, allowing parallel processing without requiring all data to be copied to each core. This segmentation approach reduces the computational overhead associated with data copying while maintaining parallel processing speed benefits.
2Ease of operation
If all data is copied to each processor for parallel processing, then processing can proceed independently, but data copying overhead becomes significant
Solution Approach 1:
The patent divides the complete dataset into multiple segments and assigns each segment to a specific processing core. This allows each core to process its assigned data independently without requiring copies of the entire dataset. The segmentation enables independent processing while minimizing data copying overhead to only the necessary subset of data for each core.
3Loss of time
If datasets are divided for parallel processing, then processing time is reduced, but synchronization delays occur
Solution Approach 1:
The patent segments the processing task into independent units assigned to different cores, where each core processes its assigned segment without requiring continuous synchronization. This segmentation approach reduces processing time by enabling simultaneous computation while minimizing synchronization delays by only requiring coordination at the boundaries of segments rather than throughout the entire dataset.
4Measurement precision
If complete datasets are processed in parallel, then comprehensive analysis is achieved, but computational resources are consumed excessively
Solution Approach 1:
The patent segments the large dataset into smaller manageable portions that are distributed to different processing cores. Each core processes only its assigned segment, reducing the computational resources required at each core while collectively achieving comprehensive analysis of the entire dataset. This segmentation maintains measurement precision through complete data processing while significantly reducing overall computational resource consumption.
Data Source
AI summary
A cluster selector may determine a plurality of sample clusters, and may reproduce the plurality of sample clusters at each of a plurality of processing cores. A sample divider may divide a plurality of samples stored in a database with associated attributes into a number of sample subsets corresponding to a number of the plurality of processing cores, and may associate each of the number of sample subsets with a corresponding one of the plurality of processing cores. A joint operator may perform a comparison of each sample of each sample subset at each corresponding core of the plurality of processing cores with respect to each of the plurality of sample clusters reproduced at the corresponding processing core, based on associated attributes thereof.