Vehicle Data Streaming Architecture for Real-Time Response
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies struggle to effectively manage and process large-scale vehicle data integration for real-time and non-real-time applications, lacking efficient methods for data capture, analysis, and response generation.
Innovation Solution
A data container platform combining a distributed event streaming system and Lambda Architecture Framework for batch and stream processing, utilizing MQTT messaging and metadata enrichment to parse and transform vehicular data, enabling near-real-time data collaboration and response generation across multiple applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If large scale vehicle data is captured and processed using traditional methods, then data volume increases, but processing efficiency and response time deteriorate
Solution Approach 1:
The patent segments vehicle data into multiple data streams based on different attributes (e.g., sensor type, time sensitivity, source). Each data stream is processed independently through specialized computing applications, allowing parallel processing that maintains efficiency while handling large volumes. The data container platform divides the overall processing task into smaller manageable units that can be distributed across multiple nodes.
Solution Approach 2:
The patent introduces a new dimension of processing by implementing a multi-layered architecture with batch processing and stream processing layers. This dimensional transformation allows data to be handled through different processing modes simultaneously - batch processing for non-time-sensitive data and stream processing for time-sensitive data - thereby resolving the contradiction between data volume and processing efficiency.
2Device complexity
If traditional data processing frameworks are used, then system complexity is reduced, but collaboration efficiency among computing applications deteriorates
Solution Approach 1:
The patent implements a universal data container platform that serves multiple functions: data ingestion, validation, parsing, transformation, routing, and analysis. This multi-functional platform enables different computing applications to collaborate efficiently through a common interface and shared data structures, improving collaboration efficiency without requiring each application to be overly complex.
Solution Approach 2:
The patent introduces computing nodes and data containers as intermediary elements between data sources and computing applications. These intermediaries manage data flow, coordinate between different applications, and handle the complexity of collaboration, allowing applications to work together efficiently without direct complex interactions between them.
3Loss of time
If data processing is performed in real-time, then response time is reduced, but system resource consumption increases
Solution Approach 1:
The patent implements dynamic processing where the system adapts its resource allocation based on data characteristics and time sensitivity requirements. Time-sensitive data triggers stream processing with higher resource allocation and faster response, while non-time-sensitive data uses batch processing with lower resource consumption. This dynamic adjustment resolves the contradiction between response time and resource consumption.
Solution Approach 2:
The patent changes processing parameters such as batch size, processing frequency, and resource allocation based on data attributes. For example, critical safety data receives immediate processing with maximum resources, while routine telemetry data is processed in larger batches with fewer resources. This parameter adjustment allows the system to optimize the trade-off between response time and resource consumption for different data types.
Data Source
AI summary
Approaches, techniques, and mechanisms are disclosed for large scale vehicle data collaborative analysis. According to one embodiment, a large amount of data streams is received from a multitude of vehicles. A distributed event streaming system is applied to parse the data streams based on an attribute, such as time sensitive data, location-specific data, or vehicle maintenance, operational, or fault-prevention data. A data container platform instance hosts applications that receive the parsed data streams. Output of an application is transformed into data streams having a common topic. Other applications may access the data streams by topic. Upon receiving an indication that an application has processed a data stream by topic, a decision point may be reached and executed by an external application, triggering an action on a vehicle from the multitude of vehicles. A vehicle computing system accesses topic data streams and performs an action on the vehicle.