In-Memory Spatial Database for Real-Time Geocoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional GIS technologies face challenges in real-time data processing due to complex and time-consuming extract, transform, load (ETL) processes, especially when converting non-spatial data into spatial data for geographic processing, leading to slow performance in real-time or near-real-time analysis.
Innovation Solution
An in-memory database system with an extraction module for real-time data extraction and transformation, a geocoder for geocoding address data using an internal reference table, and a geoprocessing module for spatial data processing, all integrated within the same database tier to enhance calculation performance and reduce network communication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional ETL processes are used to extract, transform, and load data for GIS processing, then data can be processed through standard procedures, but the process becomes complex and time-consuming, preventing real-time analysis
Solution Approach 1:
The patent merges the ETL process with the database loading operation by implementing incremental loading that automatically extracts, transforms, and loads data in a single integrated process. The extraction module, transformation module, and loading module work together seamlessly within the database system, eliminating the need for separate ETL procedures and reducing processing time while maintaining data completeness.
Solution Approach 2:
The system performs preliminary data transformation and validation during the data extraction phase itself, rather than waiting for a separate ETL process. By preparing data in advance and maintaining it in a ready-to-use format within the database, the system eliminates the time-consuming sequential ETL steps and enables immediate real-time analysis.
2Reliability
If complex spatial calculations are performed on large amounts of data using conventional methods, then geographic processing can be completed, but the processing speed becomes relatively slow, especially for real-time or near-real-time analysis reports
Solution Approach 1:
The patent changes the storage parameter of spatial data from traditional row-based storage to columnar storage format. This parameter change enables more efficient compression and faster retrieval of spatial data, significantly improving the speed of spatial calculations while maintaining processing accuracy. The columnar format allows the system to quickly access only the necessary spatial parameters for each calculation.
Solution Approach 2:
The system creates optimized copies of spatial data in multiple formats and precisions within the database. By maintaining copies of spatial data at different levels of detail and in various coordinate systems, the system can quickly serve real-time analysis requests without performing complex transformations on the original data, thereby improving processing speed while maintaining accuracy.
3Reliability
If data is stored in traditional database formats, then data can be persisted reliably, but network communication overhead increases and calculation performance decreases when processing large datasets
Solution Approach 1:
The patent introduces a new dimension to data storage by implementing a hierarchical storage architecture that combines traditional disk-based storage with in-memory columnar storage. This dimensional change allows the system to maintain reliable persistent storage while providing high-performance access paths through memory-resident data structures, thereby improving calculation performance without sacrificing data persistence reliability.
Data Source
AI summary
The embodiments provide an in-memory database system having an extraction module configured to extract data (e.g., business data and address data) from one or more external data sources and transform the data into a standard format, a geocoder configured to geocode the address data including obtaining spatial data based on the address data using an internal reference table, and an internal database configured to store the internal reference table, the business data, the address data, and the spatial data.