Distributed Model-Building for IoT Time Series Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data modeling techniques in distributed computing environments require domain knowledge and struggle to efficiently process large, diverse data sets from IoT devices, often missing significant variables and correlations without prior information.
Innovation Solution
A method that generates data models by applying variable transformations to time series data, partitioning it across multiple dimensions, and evaluating variable importance using distributed computational units, without relying on domain knowledge, to identify significant variables and correlations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If distributed computing environments process large data sets from IoT devices using existing data modeling techniques, then processing capability is improved, but domain knowledge requirements and difficulty in identifying significant variables worsen
Solution Approach 1:
The system performs self-service by automatically identifying significant variables and correlations through computational units that evaluate variable importance scores without requiring external domain knowledge. The model building process serves itself by using the data to generate the model structure autonomously.
Solution Approach 2:
The system changes parameters by transforming variables into different representations and evaluating them through multiple computational units. Each computational unit applies different transformations and evaluations to the same data, changing the parameter space to discover significant variables that would be missed by a single approach.
2Speed
If distributed computational units evaluate variable importance independently, then processing speed is improved, but completeness of variable identification worsens
Solution Approach 1:
The system merges results from multiple independent computational units by combining their variable importance evaluations. The central computational unit aggregates scores and findings from distributed units to produce a comprehensive model that includes all significant variables identified across the distributed processing.
Solution Approach 2:
The system adds another dimension by evaluating variables across multiple computational units simultaneously rather than sequentially. Each unit processes data from a different dimensional perspective through variable transformations, ensuring comprehensive coverage without sacrificing processing speed.
3Measurement precision
If traditional data modeling requires prior domain knowledge, then model accuracy is improved, but adaptability to new data sources worsens
Solution Approach 1:
The system achieves universality by creating a domain-agnostic model building process that can handle any data source including IoT devices, sensors, and other time-series data. The computational units use universal variable transformations and importance evaluation methods that work across different data types without requiring domain-specific knowledge.
Solution Approach 2:
The system substitutes the mechanical requirement for domain knowledge with an automated computational process. Instead of relying on human experts to identify significant variables based on their knowledge, the system uses computational units to automatically evaluate variable importance through mathematical transformations and statistical analyses.
Data Source
AI summary
In some implementations, a computer-implemented method for generating computer-readable data models includes receiving time series data; applying a plurality of variable transformations to the time series data to generate a variable matrix with first and second dimensions; partitioning the variable matrix along a first one of the first and second dimensions to generate a plurality of data sets; partitioning the plurality of data sets along a second one of the first and second dimensions to generate a plurality data subsets; providing each of the plurality of data subsets to a respective computational unit in a distributed computing environment for evaluation; receiving, from the respective computational units, scores for a plurality of variables as determined by the respective computational units from the plurality of data subsets; and selecting a portion of the plurality of variables as having at least a threshold level of accuracy in modeling the time series data.


