Parallel Data Mining Model Generation Across Distributed Computing Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Locating specific data from large datasets is typically time-consuming, necessitating a more efficient procedure.
Innovation Solution
A data mining method that divides data into multiple subsets and uses a computing system with multiple CPUs to generate and compare data mining models simultaneously, selecting the best model based on chosen techniques such as lift chart or root mean squared techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If data mining is performed on large datasets using a single computing device, then model accuracy can be maintained, but the time required to locate specific data and generate models increases significantly
Solution Approach 1:
The patent divides the large dataset into multiple data subsets and distributes them across multiple computing devices. Each computing device independently processes its assigned data subset to generate local data mining models. This segmentation enables parallel processing, significantly reducing the time required to locate specific data and generate models while maintaining comprehensive coverage of the entire dataset.
2Productivity
If data is divided into multiple subsets and processed simultaneously on multiple computing devices, then processing speed increases, but system complexity increases
Solution Approach 1:
The system segments both the data and the processing tasks across multiple computing devices. Each device handles a specific data subset independently, which simplifies the complexity at each individual device while achieving high overall productivity through parallel processing. The segmentation approach allows each computing device to operate autonomously on its assigned subset.
Solution Approach 2:
After individual computing devices generate local data mining models from their respective data subsets, the patent merges these local models to create a comprehensive global data mining model. This merging process consolidates the results from multiple parallel processing operations, achieving high productivity while managing system complexity through a structured combination approach.
3Measurement precision
If multiple data mining models are generated simultaneously from different data subsets, then model selection accuracy improves, but computational resources required increase
Solution Approach 1:
The patent generates multiple data mining models from different data subsets simultaneously, which exceeds the minimum requirement of a single model. This partial or excessive action improves model selection accuracy by providing multiple candidates for comparison and selection, while the distributed computing approach manages computational resource consumption across multiple devices rather than concentrating all resources in a single system.
Data Source
AI summary
A computing system and method for generating and selecting data mining models. The computing system comprises a computer readable medium and computing devices electrically coupled through an interface apparatus. A data mining modeling algorithm is stored on the computer readable medium. Each of the computing devices comprises at least one central processing unit (CPU) and an associated memory device. Each of the associated memory devices comprises a data subset from a plurality of data subsets. A technique is selected for generating a data mining model applied to each of the data subsets. The data mining modeling algorithm is run simultaneously, on each of the computing devices, using the selected technique to generate an associated data mining model on each of the computing devices. A best data mining model from the generated data mining models is determined in accordance with the selected technique.