ML-Based Data Compression for Industrial Control System Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage strategies for industrial automation control systems and power systems are inadequate for detailed long-term analysis, leading to loss of valuable information and excessive storage requirements, as they aggregate data at a coarse granularity, making it difficult to extract meaningful patterns and relations.
Innovation Solution
A machine learning approach is employed to compress data from various sources, optimizing storage by automatically selecting compression techniques based on data characteristics, reducing storage space and communication bandwidth, and enabling detailed long-term analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is aggregated and stored at a coarse granularity of 15 minutes, then storage space requirements are reduced, but the ability to perform meaningful analysis and extract patterns from past data is lost
Solution Approach 1:
The patent segments data storage into multiple granularities: fine-grained data is stored for recent periods while coarser aggregations are stored for longer periods. This allows the system to maintain detailed data when needed for analysis while reducing storage requirements for historical data, thereby resolving the contradiction between storage space and information preservation.
Solution Approach 2:
The patent combines multiple compression techniques including machine learning-based compression, statistical compression, and traditional compression algorithms. By merging these different approaches, the system achieves superior compression ratios that preserve fine-grained data details while significantly reducing storage space requirements.
2Productivity
If detailed data is stored for longer periods to enable thorough analysis of weekly, monthly, seasonal, and yearly patterns, then analysis capability is improved, but storage requirements become excessive
Solution Approach 1:
The patent implements dynamic data retention policies where the granularity and retention period of stored data adapt based on analysis needs. Recently accessed or frequently analyzed data maintains finer granularity, while less accessed historical data is automatically aggregated to coarser levels, enabling long-term storage of detailed data patterns without excessive storage requirements.
Solution Approach 2:
The patent changes the parameter of data granularity over time, storing fine-grained data for short-term analysis and automatically aggregating to coarser granularities for long-term storage. This parameter transformation allows the system to maintain analysis capability for various time scales while managing storage requirements effectively.
3Quantity of substance
If current aggregation strategies are used to reduce storage requirements, then storage space is conserved, but the ability to determine grid extension strategies and perform retro-perspective analysis is compromised
Solution Approach 1:
The patent performs preliminary compression and aggregation operations on data before it is stored, using machine learning techniques to identify and preserve important patterns and relationships. This preliminary processing allows the system to store compressed data that retains sufficient detail for various future analysis scenarios including grid extension planning and retro-perspective analysis.
Solution Approach 2:
The patent introduces machine learning-based compression algorithms as an intermediary between raw data and stored data. This intermediary process intelligently compresses data while preserving essential information needed for diverse analysis purposes, enabling both storage efficiency and analysis versatility to coexist.
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
To facilitate storage of data from plural data sources (31-33) of an industrial automation control system, power distribution system or power generation system, a decision making device (20) executes a machine learning algorithm to determine a compression technique in dependence on the data source (31-33) from which data originates.