Historian Server Signal Cache for Fast Data Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for storing and accessing process data in historian servers are memory-intensive and inefficient, particularly when dealing with large volumes of data, leading to slow read operations and the overwriting of historical data, which renders previously calculated aggregates unusable until manually recalculated.
Innovation Solution
Assigning each process value a separate signal cache for raw values and internal signal-specific aggregate accumulators, which are incrementally updated, allowing for immediate aggregation and storage without re-reading historical data, utilizing a ring memory configuration to maintain aggregate data efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If all process data are stored in a single table with signal index and timestamp as primary indexes, then all signals can be archived in a unified structure, but reading individual signal time series becomes very slow as the number of signals increases
Solution Approach 1:
The patent divides the unified table structure into multiple signal-specific tables, each containing only the data for one signal. This segmentation allows direct access to individual signal data without scanning the entire table, dramatically improving read speed while maintaining organized storage.
2Measurement precision
If aggregate values are calculated by reading the entire raw data stock, then complete aggregate determination is achieved, but the operation becomes very time-consuming and involves high data throughput requirements
Solution Approach 1:
The patent pre-calculates and stores aggregate values (such as minimum, maximum, sum) directly in the signal tables alongside the raw data. This preliminary action eliminates the need to re-read and re-process entire data stocks when aggregates are needed, significantly reducing calculation time while maintaining completeness.
3Quantity of substance
If historical data are overwritten to make space for new data, then storage capacity is optimized, but previously created aggregate values become unusable and require manual recalculation
Solution Approach 1:
The patent implements continuous incremental update of aggregate values whenever raw data is added or overwritten. The system maintains the invariant that aggregate values are always consistent with the current raw data state, eliminating gaps in data availability and removing the need for manual recalculation after overwrites.
4Ease of operation
If all signals are recorded at the same fixed speed (e.g., every 5 seconds), then uniform data collection is achieved, but slowly changing signals are recorded multiple times while rapidly changing signals may be lost
Solution Approach 1:
The patent implements dynamic sampling intervals for each signal based on its rate of change. Signals that change rapidly are sampled more frequently, while slowly changing signals are sampled less frequently. This dynamic adjustment maintains uniform data collection operations while significantly improving signal capture accuracy for different signal types.
Data Source
AI summary
Exemplary embodiments of the disclosure relate to a method and a system for the storage and processing of process values of a technical process or a technical installation in a historian server which can be integrated into a process control system of a technical installation or of a technical process. In such a method and system a dedicated signal cache can be associated with every process value for its raw values provided by the process. The raw values provided by the process are stored in the respective signal caches and additional internal signal-individual aggregate accumulators for the desired aggregates are associated with every raw value. The aggregate accumulators are incrementally modified for the defined interval when a new raw value is received.