Spatial Join Skew Detection in Parallel Database Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional spatial join algorithms in parallel database management systems face challenges in efficiently partitioning spatial objects to avoid overlap and redundancy, leading to suboptimal distribution of input spatial data.
Innovation Solution
A method and system for executing spatial queries in a distributed database that involves mapping spatial objects to spatial grids, redistributing them based on cell skewness, and subdividing skewed cells to improve data distribution and facilitate efficient spatial joins, utilizing modules for object mapping, skew determination, redistribution, and table joining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If conventional space decomposition is used to partition spatial data, then spatial objects can be assigned to regions, but overlap and redundancy occur leading to uneven distribution
Solution Approach 1:
The patent segments spatial data partitioning into multiple phases: initial partitioning using conventional space decomposition, skew detection to identify unevenly distributed regions, and secondary partitioning of skewed cells. This multi-stage segmentation approach allows the system to first apply simple partitioning rules, then refine specific problematic regions to achieve overall even distribution.
Solution Approach 2:
The patent applies local quality by treating skewed cells differently from normal cells. Instead of applying a uniform partitioning strategy across all spatial regions, the system identifies cells with uneven data distribution through skew detection and applies specialized secondary partitioning only to those specific regions, preserving the simplicity of conventional partitioning for well-distributed areas.
2Stability of the object's composition
If static data structure like N×N grid is used for partitioning, then disjoint decomposition is achieved, but spatial object duplication occurs
Solution Approach 1:
The patent introduces an intermediary mechanism - the skew detection and secondary partitioning module - that sits between the static N×N grid partitioning and the final data distribution. This intermediary detects when spatial objects would be duplicated across partitions and triggers additional partitioning of the affected cells, thereby eliminating duplication while preserving the stable grid structure.
3Adaptability or versatility
If dynamic data structure like R-tree is used for partitioning, then spatial objects are assigned to single region, but region overlap occurs leading to redundancy
Solution Approach 1:
The patent implements feedback by detecting skewness in partitioned cells and using this information to trigger secondary partitioning. The skew detection mechanism provides feedback about uneven data distribution, and this feedback drives the adaptive refinement of specific cells, thereby eliminating redundancy caused by region overlap while maintaining the flexible R-tree structure for spatial object mapping.
Data Source
AI summary
There is provided a method, a system and a machine readable medium to execute a spatial query against a plurality of input tables of a distributed database. The method comprises: mapping spatial objects of a first input table to a first spatial grid, the first spatial grid including a plurality of first spatial cells; redistributing the spatial objects of the first input table to a first intermediate table of one or more access module processors based on the first spatial cell of the first spatial grid to which each of the spatial objects is mapped; determining whether one or more first spatial cells of the first spatial grid are skewed; subdividing each of the skewed one or more first spatial cells to a second spatial grid, the second spatial grid including a plurality of second spatial cells; mapping spatial objects of the first intermediate file to the second spatial grid, the second spatial grid including a plurality of second spatial cells; redistributing the spatial objects of the first intermediate file to one or more access module processors based on the second spatial cell of the second spatial grid to which each of the spatial objects is mapped; mapping spatial objects of a second input table to the first spatial grid and the second spatial grid; redistributing the spatial objects of the second input table to a second intermediate file of one or more access module processors based on the first spatial cell of the first spatial grid and the second spatial cell of the second spatial grid to which each of the spatial objects is mapped; and joining the spatial objects of the first intermediate file to spatial objects of the second intermediate file for each first spatial cell of the first spatial grid or the second spatial cell of the second spatial grid.