Time-Series Data Confidence Computation for KPI Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems fail to accurately compute confidence values for corrected data in time-series data, particularly for erroneous data instances related to key performance indicators (KPIs), leading to potential errors in energy management and other applications.
Innovation Solution
A system and method that determine predicted replacement values for potentially erroneous data instances in time-series data streams, using a confidence value computation process involving data inspection, KPI characterization, snapshot generation, and confidence measurement to resolve errors and provide accurate KPI values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing systems process time-series data without confidence value computation, then processing speed is maintained, but data accuracy and reliability deteriorate due to undetected erroneous data instances
Solution Approach 1:
The system segments the data processing workflow into distinct modules: data inspection module that identifies potentially erroneous data instances, snapshot generation module that creates historical context, KPI characterization module that defines confidence criteria, and confidence measurement module that computes confidence values. This segmentation allows each module to specialize in one aspect of data quality assurance, improving overall measurement precision while managing complexity through modular design.
Solution Approach 2:
The system performs preliminary actions by generating snapshots of historical data and pre-characterizing KPIs before confidence computation. The data inspection module proactively identifies potentially erroneous data instances before they affect downstream processing. This preliminary preparation enables faster confidence value computation and reduces the need for complex real-time analysis, thereby improving data accuracy without proportionally increasing system complexity.
2Reliability
If confidence value computation is implemented for all data instances, then data reliability improves, but processing time and computational resources increase
Solution Approach 1:
The system applies confidence value computation selectively rather than uniformly to all data instances. The data inspection module identifies specifically those data instances that are potentially erroneous based on local data quality indicators. Confidence measurement is then applied only to these identified instances, ensuring data reliability where needed while avoiding unnecessary processing of already-valid data, thus minimizing time loss.
Solution Approach 2:
The system changes the parameter of confidence computation from a constant operation applied to all data to a conditional operation applied only when needed. The confidence value computation is triggered based on indicators from the data inspection module, such as data anomalies, missing values, or deviations from expected patterns. This parameter change enables the system to maintain high data reliability for problematic instances while significantly reducing overall processing time.
3Measurement precision
If multiple predicted replacement values are generated for erroneous data instances, then data accuracy improves through selection, but device complexity increases due to multiple value management
Solution Approach 1:
The system uses feedback mechanisms to manage multiple predicted replacement values. The confidence measurement module evaluates each predicted replacement value and provides feedback in the form of confidence values indicating the reliability of each replacement. This feedback allows the system to select the most reliable replacement value systematically, improving data accuracy while managing complexity through evidence-based selection rather than arbitrary choices.
Solution Approach 2:
The confidence value acts as an intermediary that mediates between multiple predicted replacement values and the final selected value. Instead of directly comparing complex replacement value structures, the system uses confidence values as an intermediate metric to evaluate and rank replacement options. This intermediary approach simplifies the selection process, improving data accuracy through systematic evaluation while reducing the complexity of managing multiple replacement values.
Data Source
AI summary
A system, computer program product, and method are presented for providing replacement data for data in a time series data stream that has issues indicative of errors, where the data issues and the replacement data are related to one or more KPIs. The method includes determining one or more predicted replacement values for potentially erroneous data instances in the time series data stream. The method further includes resolving the potentially erroneous data instances with one predicted replacement value of the one or more predicted replacement values in the time series data stream.