Automated VEE Processor for Energy Data Anomaly Handling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Real-time and near real-time energy consumption systems face challenges in accurately processing energy consumption data due to intermittent errors and anomalies, leading to suboptimal validation, estimation, and editing (VEE) techniques, which are often labor-intensive and less accurate, especially when handling a large number of data streams.
Innovation Solution
An automated VEE processor and configuration engine dynamically optimize VEE techniques for individual energy consumption data streams, automatically configuring detection and estimation rules based on anomaly durations and stream types, reducing the need for manual data analyst intervention and enhancing accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated VEE techniques are implemented to process energy consumption data streams, then processing speed and productivity improve, but system complexity increases
Solution Approach 1:
The system segments the energy consumption data processing into distinct modules: data reception module, validation module, estimation module, and editing module. Each module handles specific aspects of the VEE process independently, allowing parallel processing of multiple data streams while maintaining manageable system complexity through functional decomposition
Solution Approach 2:
The automated VEE system implements universal processing algorithms that can handle multiple types of energy consumption data streams (electrical, gas, water) using the same validation, estimation, and editing techniques. This multi-functional approach enables a single system to process diverse data streams without requiring separate specialized systems for each resource type
2Device complexity
If manual data analyst intervention is used for VEE techniques, then system complexity is reduced, but labor costs and processing time increase
Solution Approach 1:
The system implements self-service automation where the VEE processor automatically performs validation, estimation, and editing operations on energy consumption data streams without requiring manual data analyst intervention. The processor autonomously identifies anomalies, applies appropriate estimation algorithms, and corrects errors, eliminating the need for human analysts to manually process each data stream while significantly reducing processing time
3Ease of operation
If generic VEE techniques are applied to all data streams, then ease of operation improves, but measurement precision deteriorates
Solution Approach 1:
The system applies local quality by selecting and applying specific estimation algorithms based on the characteristics of each individual data stream and the type of anomaly detected. Different validation rules, estimation techniques, and editing strategies are applied to different segments of data streams according to their specific needs, such as applying linear interpolation for short gaps and seasonal patterns for longer anomalies, thereby maintaining high data accuracy while preserving ease of operation through automated algorithm selection
Data Source
AI summary
A method for performing validation, estimation, and editing (VEE), including: displaying real-time electrical usage for a building on a controllable video display; executing VEE rules on each of the streams to generate and store a corresponding post VEE readings, the post VEE readings comprising tagged energy consumption data sets each associated with a corresponding one of the streams, each of the data sets comprising groups of contiguous interval values tagged as having been validated and corresponding to correct data; for the each of the data sets, creating anomalies having different durations using only the groups of contiguous interval values; generating estimates for the anomalies by employing estimation techniques; for each of the durations, selecting one of the estimation techniques for subsequent employment when performing VEE of subsequent energy consumption data for the corresponding one of the streams; and executing functions on the streams translated by the generating and directing the controllable video display to display a weather normalized usage baseline recommendation for action regarding current energy usage.