Spatial Data Mapping via Space-Filling Curves in Column Stores

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Traditional column-store architectures struggle to efficiently manage and query spatial point data due to the overhead of tree-based index structures, which are not suitable for in-memory databases and result in poor query performance for geographic data.

Innovation Solution

The use of space-filling curves, such as Hilbert curves, to map spatial data sets to physical storage, enabling efficient querying by transforming points into a positive coordinate space, sorting them by Hilbert distance, and using bit-compressed vectors for compact storage and quick data retrieval.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If tree-based index structures (R-Tree, K-d Tree, Quadtree) are used to manage spatial data in column stores, then spatial query capability is provided, but space overhead and time complexity increase due to managing linked index nodes

Engineering Contradiction:

Improvespatial query capabilityVSAvoidindex structure overhead

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The patent extracts the spatial indexing function from traditional tree-based structures and relocates it to the storage layer by using space-filling curves to reorder spatial points according to their curve positions. This eliminates the need for separate index structures while preserving spatial query capabilities through the inherent ordering property of space-filling curves.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The space-filling curve approach serves multiple functions simultaneously: it provides spatial indexing, enables efficient range queries through continuous scanning, and maintains data locality. The same reordered storage structure supports both point queries and range queries without requiring separate index mechanisms.

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Ease of operation

If tree-based index structures are used for spatial data, then spatial access is enabled, but continuous scanning of in-memory structures is not supported due to linked node requirements

Engineering Contradiction:

Improvespatial data accessVSAvoidcontinuous scan performance

Core Design Contradiction:

Ease of operationVSProductivity

Solution Approach 1:

Instead of organizing data in continuous storage and adding index structures on top, the patent inverts the approach by reordering the data itself according to space-filling curve positions. This inversion allows the base storage structure to provide both random access and continuous scanning capabilities without requiring additional index layers.

Inventive Principle:

Principle #13The other way round (Inversion)

3Adaptability or versatility

If geographic coordinates are stored in standard columns as floating point numbers, then integration with column-store systems is achieved, but spatial data is not handled in a native way reducing query efficiency

Engineering Contradiction:

Improvecolumn-store integrationVSAvoidspatial query efficiency

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent changes the organizational parameter of spatial data from arbitrary floating-point coordinate storage to space-filling curve ordered positions. This parameter change maintains compatibility with column-store architectures while dramatically improving spatial query efficiency by preserving spatial locality in the stored order.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS10929501B2Managing and querying spatial point data in column stores

Publication Date: 2021.02.23 SAP SE

AI summary

A query of spatial data is received by a database comprising a columnar data store storing data in a column-oriented structure. Thereafter, a spatial data set is mapped to physical storage in the database using a space-filling curve. The spatial data set is then compacted and such compacted data can be used to retrieve data from the database that is responsive to the query. Related apparatus, systems, techniques and articles are also described.