Multi-Level Parallel Buffering for Vehicle Telematics Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional internet-based frameworks for processing vehicle telematics data face challenges due to unstable GPS/3G/4G signals and the need for fast, large shared memory access to handle multiple requests from moving vehicles, which are not independent and require parallel processing to avoid delivering stale data.
Innovation Solution
A vehicle domain multi-level parallel buffering and context-based streaming data pre-processing system that includes a first-level buffer and an electronic task-queue-dictionary (TQD) module, which separates serial data streams into individual data values and assigns them to specific second-level buffers based on context, enabling parallel processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional internet-based frameworks (Hadoop, Storm, Spark) are used to process vehicle telematics data, then the system can handle data from stationary terminals, but the system fails to handle constantly moving vehicles with unstable GPS/3G/4G signals and delivers stale data due to lack of fast shared memory access
Solution Approach 1:
The system segments the processing architecture into multiple levels: edge computing nodes in vehicles perform local data preprocessing and filtering, while cloud-based frameworks handle aggregate analysis. This segmentation enables fast local response to maintain data freshness while distributing processing workload to maintain productivity.
Solution Approach 2:
The system performs preliminary actions by pre-processing and filtering telematics data at the edge computing nodes before transmission to cloud frameworks. This preliminary filtering reduces data volume and ensures critical data is processed immediately, maintaining both data freshness and processing efficiency.
2Productivity
If a single-level buffering system is used, then the system structure is simple, but the system cannot achieve parallel processing and delivers stale data during peak data periods
Solution Approach 1:
The system transitions from a single-level buffering architecture to a multi-level hierarchical buffering system across different spatial and functional dimensions. This includes vehicle-level buffers, edge computing buffers, and cloud-based buffers, enabling parallel processing while managing complexity through structured organization.
Solution Approach 2:
The buffering system implements a nested hierarchical structure where vehicle-level buffers are contained within edge computing buffers, which are in turn contained within cloud-based buffering systems. This nested architecture enables parallel processing at multiple levels while organizing complexity in a manageable hierarchical manner.
3Productivity
If serial data stream processing is used, then the system implementation is simple, but the system cannot handle multiple simultaneous requests from vehicles and delivers stale data
Solution Approach 1:
The system segments the serial data stream into parallel processing streams at multiple levels. Edge computing nodes process data in parallel from multiple vehicles simultaneously, and cloud frameworks process aggregate data in parallel, thereby increasing throughput while reducing latency through concurrent operations.
Solution Approach 2:
The system implements dynamic parallel processing where the degree of parallelism adapts based on data volume and processing requirements. During peak periods, more parallel processing resources are allocated, while during low-traffic periods, resources are consolidated, optimizing both throughput and resource utilization.
Data Source
AI summary
A vehicle domain multi-level parallel buffering and context-based streaming data pre-processing system includes a first data processing level and a second data processing level. The first data processing level includes a first-level buffer configured to buffer data provided from a plurality of raw data streams output from a plurality of vehicles. The second data processing level includes an electronic task-queue-dictionary (TQD) module and a plurality of second-level data processing buffers. The TQD module is configured to create a plurality of tasks in response to receiving a serial data stream output from the first-level buffer. The TQD module is further configured to assign each task to a corresponding second-level buffer, and separate the serial data stream into individual data values that are delivered to a specific second-level buffer based on the task so as to generate a multi-level parallel context-based buffering operation.