Trip Feature Integration Pipeline for Single-Pass Sensor Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing workflows that rely on raw sensor data from mobile devices are processor intensive and time-consuming, requiring multiple passes through the data for information discovery, which leads to network bandwidth and processor throughput issues.
Innovation Solution
A features integration pipeline system that collects raw sensor data, evaluates it to identify trips, and processes metadata separately. The system normalizes trip data, processes it in parallel through feature enhancing apps, and integrates features into granular level tables stored in a feature store, reducing the need for additional data passes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If multiple passes are made through raw sensor data to discover information, then information completeness is improved, but processor throughput deteriorates and processing time increases
Solution Approach 1:
The patent segments the processing workflow into distinct passes with specific responsibilities: a first pass that extracts and stores intermediate results (features, metadata, trip information), and subsequent passes that consume these pre-processed results. This segmentation allows different applications to access processed data without requiring multiple passes through the raw sensor data, thereby improving processor throughput while maintaining information completeness.
Solution Approach 2:
The system performs preliminary processing actions in the first pass by extracting features, generating metadata, and storing trip information before subsequent applications need the data. This preliminary action ensures that all necessary information is available upfront, eliminating the need for multiple passes through raw data and improving overall processing efficiency.
2Adaptability or versatility
If multiple applications process raw sensor data for the same purposes, then application-specific information needs are met, but resource load increases and processing efficiency decreases
Solution Approach 1:
The patent creates a universal processing framework where a single first-pass workflow performs multiple functions: extracting features, generating metadata, identifying trips, and storing results in a centralized data structure. This universal processing result can then be consumed by multiple different applications for their specific needs, eliminating redundant processing while maintaining adaptability to various application requirements.
3Loss of information
If raw sensor data is processed repeatedly for information discovery, then data evaluation completeness is improved, but network bandwidth and processor throughput are overwhelmed
Solution Approach 1:
The system extracts critical information (features, metadata, trip data) from raw sensor data in the first pass and stores it in a structured format. Subsequent applications extract only the specific information they need from this pre-processed data rather than reprocessing the entire raw data set. This extraction approach maintains data evaluation completeness while significantly reducing network bandwidth and processor throughput requirements.
Data Source
AI summary
Raw sensor data from a device of a user is collected. Trips from the sensor data are identified as raw trip data. Metadata for the trips is identified, linked to the trips, and maintained separately from the trip data. The trips are staged, and the corresponding trip data normalized when obtained from a storage or memory location. Normalized trip data is piped or made accessible to feature enhancing applications (apps), each app associating one or more features and events with a given trip. The features and events are maintained for the trips in event and feature level of detail tables. The tables are processed by consuming apps for purposes of updating user-level attributes associated with the user. In an embodiment, custom apps process the raw trip data to add additional features and events, which are directly integrated and updated in the tables for access by the consuming apps.