Cascaded Vehicle Data Aggregation for Faster AI Model Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for training machine learning models for semi-autonomous/autonomous vehicular operations are computationally expensive and time-consuming due to the large volume of raw vehicle data, especially when exploring different combinations of variables, transformations, and levels of resolution, which are often unknown in advance.
Innovation Solution
A method and system that transform raw vehicle data into cascaded data structures by iteratively segmenting and applying transformation functions at multiple levels of resolution, generating intermediate data structures that can be combined to create a cascaded data structure, allowing for efficient access and use in training machine learning models without repetitive re-processing of raw data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If raw vehicle data is directly used for training machine learning models, then model training can be performed, but computational resources and time required are excessive
Solution Approach 1:
The patent applies preliminary action by pre-processing raw vehicle data into multiple intermediate data structures with different levels of granularity before model training. The system segments raw data into various resolutions (e.g., trip-level, day-level, week-level aggregations) in advance, so that when training machine learning models, the appropriate pre-processed data structure can be directly used without repetitive re-processing, significantly reducing computational resources and training time.
Solution Approach 2:
The patent applies segmentation by dividing raw vehicle data into multiple intermediate data structures with different levels of granularity. The system segments data at various hierarchical levels (individual sensor readings, trip-level aggregations, day-level summaries, week-level trends), allowing flexible selection of appropriate data resolution for different model training scenarios and reducing the overall computational burden.
2Reliability
If different combinations of variables, transformations, and levels of resolution are explored, then optimal model performance can be achieved, but the process becomes extremely time-consuming
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing multiple intermediate data structures with different transformations, variable combinations, and resolution levels before model exploration begins. This allows researchers to quickly experiment with different model configurations by selecting from pre-prepared data structures rather than re-processing raw data for each exploration scenario.
Solution Approach 2:
The patent applies universality by creating a multi-functional intermediate data structure system that serves multiple purposes: it supports exploration of different variable combinations, multiple transformation types, and various resolution levels all within a unified framework. This universal structure accelerates model exploration by eliminating the need for separate data processing pipelines for each exploration scenario.
3Productivity
If comprehensive data processing is performed to create intermediate data structures at various resolutions, then efficient model training is enabled, but data processing complexity increases
Solution Approach 1:
The patent applies segmentation by organizing comprehensive data processing into distinct hierarchical levels (raw data, trip-level intermediates, day-level intermediates, week-level intermediates). Each level is processed and stored separately with its own structure optimized for specific query types, making the overall complex system manageable through modular organization of data processing tasks.
Data Source
AI summary
Systems and methods are provided for generating cascaded data structures that can be used, for example, for training models for use in artificial intelligence applications, for example, in for vehicular operation and vehicle usage modeling and analyses. Examples include obtaining raw sensor data from vehicles comprising a first characteristic and iteratively generating intermediate data structures by segmenting data items of an input data structure according to levels of the first characteristic and, for each level, executing one or more transformation functions on the segmented data items to generate a respective intermediate data structure. The input data structure for a first iteration may be the raw data and the input data structure for subsequent iterations may be an intermediate data structure generated by a preceding iteration. Examples also include combining the intermediate data structures to generate the cascaded data structure.