Time Series Data Imputation Using Extremeness Scores
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data imputation systems are ineffective in handling time-series data with periodic, frequent, or multiple extreme events, leading to inaccurate predictions due to their assumptions about smooth data without extreme events.
Innovation Solution
A system that generates substitute values for missing data elements in time-series data by identifying and scoring extremeness using a Recurrent Neural Network (RNN) with bi-directional Long Short-Term Memory (LSTM) networks, processing both the original data and extremeness scores to reconstruct the dataset.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional data imputation systems are used, then the system is simple to implement, but the prediction accuracy deteriorates when data contains extreme events
Solution Approach 1:
The patent segments the data processing into distinct components: an extremeness detection module that identifies extreme events, and a data imputation module that handles missing values. This segmentation allows the system to treat extreme events differently from normal data, improving prediction accuracy while maintaining manageable complexity through modular architecture.
Solution Approach 2:
The patent changes the parameter representation by introducing extremeness scores that quantify the severity of extreme events. This parameter transformation enables the imputation algorithm to adapt its behavior based on the detected extremeness, improving prediction accuracy for datasets containing extreme events while keeping the system structure relatively simple.
2Loss of information
If data imputation is performed on sparse time series data, then the completeness of the dataset is improved, but the accuracy of imputed values deteriorates when extreme events are present
Solution Approach 1:
The patent applies preliminary action by detecting and scoring extreme events before performing data imputation. The extremeness detection module processes the time series data first, identifying extreme events and assigning scores. This preliminary step allows the subsequent imputation algorithm to account for extreme events, improving imputation accuracy while maintaining data completeness.
Solution Approach 2:
The patent implements feedback by using the detected extremeness scores to guide the imputation process. The system continuously monitors for extreme events and adjusts the imputation strategy based on the extremeness scores, ensuring that imputed values are more accurate in the presence of extreme events while maintaining complete datasets.
Data Source
AI summary
In various examples, a system can obtain a first time series data set, the first time series data set including a plurality of data elements. Each data element can include value data and corresponding time data. Based on the first time series data set, the system can generate a second data set and a third dataset. The second dataset can indicate one or more data elements with missing value data and the third dataset can include extremeness data. The extremeness data can indicate an extremeness score for each data element of the plurality of data elements. Additionally, based on the first time series data set, the second data set and a third dataset, the system can implement a set of operations that generate a substitute value data for each data element of the one or more data elements that is missing value data.