Spatial Join Query Indexing to Filter Invalid Partitions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Spark-based spatial join query methods are inefficient due to the inclusion of invalid data in spatial partitions, leading to resource wastage and prolonged calculation times, as they rely on sampling to approximate spatial distribution which may not accurately represent the full dataset.
Innovation Solution
Generate multi-tree spatial indexes, such as quad-trees, to collect global spatial distribution information, determine intersecting spatial partitions, and perform spatial join calculations only on data within these partitions, filtering out invalid data during the partitioning stage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If sampling is used to approximate spatial distribution, then calculation time is reduced, but measurement precision of spatial distribution deteriorates
Solution Approach 1:
The patent generates multi-tree spatial indexes (quad-tree, R-tree, KD-tree) in advance to organize spatial data into hierarchical structures. This preliminary indexing allows the system to quickly determine spatial distribution characteristics without needing to sample the data, thereby maintaining high measurement precision while reducing calculation time through efficient index-based queries.
Solution Approach 2:
The patent replaces the mechanical sampling process with an information-based approach using multi-tree spatial indexes. Instead of physically sampling spatial data to estimate distribution, the system uses pre-built index structures to accurately query and determine spatial distribution information, substituting a computational information system for a statistical sampling mechanism.
2Reliability
If all spatial partitions are processed, then completeness of spatial join query is improved, but productivity deteriorates due to processing invalid data
Solution Approach 1:
The patent extracts and processes only the valid spatial partitions that contain intersecting geometric objects by using global spatial distribution information to identify and filter out partitions that do not contain relevant data. This extraction principle allows the system to maintain complete query results for valid partitions while eliminating the productivity loss caused by processing invalid partitions.
Solution Approach 2:
The patent applies different processing strategies to different spatial partitions based on their local characteristics. Partitions identified as containing valid data through global spatial distribution analysis receive full processing attention, while partitions determined to be invalid are skipped. This local quality approach ensures completeness where needed while improving overall productivity by avoiding unnecessary processing.
3Measurement precision
If multi-tree spatial indexes are generated, then spatial distribution information accuracy is improved, but device complexity increases
Solution Approach 1:
The patent segments the spatial indexing task into multiple specialized tree structures (quad-tree for hierarchical spatial subdivision, R-tree for minimum bounding rectangles, KD-tree for k-dimensional space partitioning). Each tree type handles specific aspects of spatial organization, and their combined use provides comprehensive and accurate spatial distribution information while distributing the complexity across multiple specialized components rather than one monolithic structure.
Data Source
AI summary
A spatial join query method and apparatus, an electronic device, and a storage medium. The spatial join query method comprises: obtaining a first resilient distributed dataset of first spatial data and a second resilient distributed dataset of second spatial data; generating multi-tree spatial indexes for spatial partitions, and collecting statistics about spatial distribution information of geometric objects according to the multi-tree spatial indexes to obtain global spatial distribution information; determining an overlapping spatial partition according to the global spatial distribution information; and setting data, in the first spatial data and the second spatial data, corresponding to the overlapping spatial partition as target data, and performing spatial join calculation on the target data.