Vehicle Journey Analysis Using Geohash Event Window Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The automotive industry faces challenges in processing and analyzing high-volume vehicle location event data in real-time, particularly in identifying individual vehicle journeys and destinations, due to the vast amount of data generated and the need for accurate anonymization.
Innovation Solution
A system and method that ingest location event data, encode it using geohashing to identify proximity, and process it to determine vehicle journeys by identifying journey segments, dwell times, and minimum travel distances, using connected components algorithms for active vehicle detection, and store data in a data warehouse for analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional systems process location event data, then data processing capability is limited, but processing speed and real-time analysis capability deteriorate when facing high-volume data up to 600,000 records per second
Solution Approach 1:
The system segments the high-volume data stream into manageable processing units using event windows and micro-batch processing. Location events are grouped into windows with configurable time durations, allowing the system to process data in discrete, manageable chunks rather than attempting to handle the entire high-volume stream simultaneously, thus maintaining processing speed while handling increased data volume.
Solution Approach 2:
The system implements periodic processing through event windows that continuously slide through the data stream at configured intervals. This periodic action allows the system to maintain a steady processing rhythm, ingesting and analyzing data in regular cycles rather than attempting continuous real-time processing of all 600,000 records per second, thereby balancing processing speed with data volume capacity.
2Measurement precision
If the system processes individual vehicle data for journey identification, then journey analysis accuracy improves, but data processing complexity and computational resources increase
Solution Approach 1:
The system performs preliminary filtering and aggregation of location events before detailed journey analysis. By pre-processing data to identify candidate journey segments based on basic criteria (time windows, location changes), the system reduces the complexity of subsequent individual vehicle analysis while maintaining accurate journey identification. This preliminary action prepares data in advance, making the complex analysis step more manageable.
Solution Approach 2:
The system applies partial processing to individual vehicle data by focusing computational resources only on events that meet journey identification criteria within event windows. Rather than analyzing every single data point for every vehicle, the system selectively processes only relevant segments, reducing overall system complexity while maintaining sufficient journey identification accuracy through targeted analysis of excessive data subsets.
3Loss of information
If the system encodes location data to proximity using geohashing, then data privacy and anonymization improve, but location precision and detail are reduced
Solution Approach 1:
The system applies different levels of geohashing precision to different analytical needs. For privacy-sensitive operations, higher-level geohashes (coarser proximity) are used to protect individual location data. For analysis requiring greater precision, the system can access the original location data or use lower-level geohashes. This local quality approach allows the system to optimize privacy protection in appropriate contexts while maintaining location precision where needed for accurate journey analysis.
Data Source
AI summary
Embodiments are directed to a system and method for ingesting location event data and identifying a journey for a vehicle from the event data. The journey identification includes identifying whether a given vehicle is moving for purposes of driving to a journey destination.