Historian Data Synchronization Using Bitmask Metadata Extraction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Synchronizing historical data between primary and secondary historian systems in complex industrial operations like pipeline systems is challenging due to the large volume of data and the need for bidirectional synchronization, especially during intermittent outages where both systems may go down at different times.

Innovation Solution

The method involves using short-term and long-term repositories in both historian systems, where data values are stored as tags with timestamps and IDs, and upon availability, these tags are copied to the long-term archives. If the secondary system is unavailable, flags are set to synchronize data later, ensuring efficient data management and redundancy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If complete historical archive replication is performed between primary and secondary historian systems, then data redundancy and fault tolerance are improved, but data synchronization time and computational resources are excessively consumed due to the large volume of terabytes of data

Engineering Contradiction:

Improvefault toleranceVSAvoidsynchronization time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent extracts only the essential synchronization metadata (bitmask files, tag ID status records, archive file headers) from the complete historical archive, rather than replicating all terabytes of data. This allows the system to determine what data needs synchronization without transferring the entire archive, significantly reducing synchronization time while maintaining fault tolerance capabilities.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The system performs preliminary actions by maintaining bitmask files and tag ID status records that pre-indicate which data elements need synchronization. When a historian system comes online after a outage, these pre-prepared metadata structures allow for rapid identification and transfer of only the necessary data changes, avoiding the need to process complete historical archives.

Inventive Principle:

Principle #10Preliminary action

2Reliability

If bidirectional synchronization is implemented between primary and secondary historians, then data consistency is improved, but system complexity and difficulty of implementation increase due to intermittent outages where both systems may go down at different times

Engineering Contradiction:

Improvedata consistencyVSAvoidsynchronization system complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent uses copying of lightweight metadata structures (bitmask files, tag ID status records, archive file headers) rather than complex bidirectional data transformation. Each historian system maintains copies of these synchronization metadata files, allowing them to independently determine what data needs to be synchronized when both systems are online, greatly simplifying the bidirectional synchronization process.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The bitmask files and tag ID status records serve as intermediary structures that mediate the bidirectional synchronization process. These metadata files act as a common language between primary and secondary historians, allowing them to exchange synchronization information in a standardized, simple format that reduces implementation complexity.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Loss of information

If complete historical archive data is transferred during synchronization, then data completeness is improved, but network bandwidth and system resources are excessively consumed

Engineering Contradiction:

Improvedata completenessVSAvoidnetwork resource consumption

Core Design Contradiction:

Loss of informationVSUse of energy by moving object

Solution Approach 1:

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The system performs partial synchronization by transferring only the metadata necessary to identify data changes (bitmask files, tag ID status records) rather than performing excessive action by transferring complete historical archives. This partial action approach ensures data completeness is maintained while dramatically reducing network resource consumption.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS7853569B2Synchronizing historical archive data between primary and secondary historian systems

Publication Date: 2010.12.14 AIR LIQUIDE LARGE IND US LP

AI summary

Techniques for operating an archival database, referred to as a historian, are disclosed. Embodiments of the invention provide techniques for synchronizing historical archive files and/or archival data between a primary and a secondary historian used to archive data values from a real-time database. The real-time database may be used to monitor the current operational state of a large industrial system, such as a pipeline. In turn, the historian may be used to archive the data stored by the real-time database. In the event that either the primary or the secondary system (or both) goes down and then comes back online at a later time, embodiments of the invention may be used to synchronize data between the primary and secondary historian systems.