Spatial Join Partitioning in MPP Database Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional spatial join techniques are not highly adaptable or effective in Massively Parallel Processing (MPP) database systems, where a large number of processing units are used to process data in parallel, and existing methods like Partition Based Spatial-Merge join (PBSM) face challenges such as grid determination, efficient duplication elimination, and suitable row distribution strategies.
Innovation Solution
The approach involves determining the intersection of bounds of data sets as a join bounding space, partitioning it into sub-partitions, and assigning these to multiple processing units for parallel processing, while using distribution cost information to distribute and redistribute join pairs to avoid high distribution costs, optimizing the processing of Spatial Joins in MPP systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional spatial join techniques are used in MPP database systems, then the processing can be performed using existing methods, but the adaptability and effectiveness are poor due to challenges in grid determination, duplication elimination, and row distribution
Solution Approach 1:
The patent divides the join bounding space into multiple sub-partitions and assigns them to different processing units. This segmentation allows each processing unit to handle a specific portion of the spatial join independently, improving adaptability to MPP architectures while reducing the complexity of global grid determination and duplication elimination.
2Productivity
If data is distributed across multiple processing units for parallel processing, then processing speed improves, but distribution costs become high due to inefficient row distribution strategies
Solution Approach 1:
The patent performs preliminary partitioning of the join bounding space into sub-partitions before distributing data to processing units. This preliminary action enables efficient row distribution strategies by pre-establishing the spatial framework, reducing the computational overhead and distribution costs during the actual parallel processing phase.
Solution Approach 2:
The patent implements local distribution strategies within each sub-partition, allowing each processing unit to efficiently manage its assigned portion of data. This local quality approach reduces global distribution costs by limiting data movement to local contexts while maintaining high parallel processing productivity.
3Productivity
If the join bounding space is partitioned into sub-partitions and assigned to multiple processing units, then parallel processing efficiency improves, but the system complexity increases
Solution Approach 1:
The patent segments the join bounding space into manageable sub-partitions that can be independently processed. This segmentation improves parallel processing efficiency by enabling concurrent operations across multiple processing units while managing system complexity through modular, hierarchical organization of the spatial data structure.
Data Source
AI summary
Improved techniques for performing Spatial Joins multi-processing computing systems and environments are disclosed. One or more intersection of bounds (or limits) of data sets is determined as a join bounding space. The join bounding space is in a space (Global space or Global universe) where a spatial join between (or for) the data can be performed. The determined join bounding space can be partitioned into sub-partitions of the join bounding space. The sub-partitions of the join bounding space can assigned respectively to multiple processing unit for processing in parallel in. In addition, distribution cost information associated with the cost of distribution of the datasets (and/or their components) to the processing units of a multi-processing system can be provided and/or used to effectively distribute and/or redistribute processing of the Spatial Join between the processing units of a multi-processing system.