Blockchain Event ETL for Secure External Data Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing blockchain systems, particularly permissioned networks like Hyperledger Fabric, face challenges in efficiently extracting and analyzing diverse transactional data due to their decentralized and permissioned nature, lacking integrated mechanisms for data extraction and analysis suitable for external data structures.
Innovation Solution
Implementing an ETL-peer within the blockchain network to execute model-driven extraction of event data using ETL-code, which transforms and loads data into external data structures while preserving privacy and adhering to the data model, utilizing event schemas and cryptographic security.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If blockchain networks use decentralized permissioned architecture to ensure security and privacy, then data security and privacy are improved, but data extraction and analysis capabilities deteriorate
Solution Approach 1:
The patent introduces an ETL-peer as an intermediary component within the blockchain network that mediates between the decentralized blockchain architecture and external data structures. This ETL-peer executes ETL-code to extract event data from blockchain events, transform the data according to event schemas and data models, and load it into external data structures, thereby enabling data extraction capabilities without compromising the decentralized secure architecture
Solution Approach 2:
The patent segments the data extraction function into a separate ETL-peer component that operates independently within the blockchain network. This segmentation allows the core blockchain to maintain its security properties while the ETL-peer handles data extraction and transformation tasks, resolving the contradiction between security architecture and extraction capability
2Stability of the object's composition
If blockchain networks maintain tamper-proof immutable ledgers to ensure data integrity, then data integrity is improved, but data flexibility and adaptability for external systems deteriorate
Solution Approach 1:
The ETL-peer acts as an intermediary that receives immutable event data from the blockchain and transforms it into flexible formats suitable for external data structures. The transformation process uses event schemas to interpret the immutable event data and applies data models to adapt it for different external systems, thereby maintaining data integrity while achieving adaptability
Solution Approach 2:
The patent changes the parameters of data representation through transformation. The immutable event data with its fixed structure is transformed into flexible data structures with adjustable parameters that can adapt to different external data models, resolving the contradiction between integrity and versatility
3Productivity
If blockchain networks implement comprehensive event data extraction to enable advanced analytics, then analytical capability is improved, but system complexity and resource consumption deteriorate
Solution Approach 1:
The patent extracts only the necessary event data from blockchain events using event schemas that define what data to capture. This selective extraction enables advanced analytics by providing relevant data to external systems without requiring the entire blockchain state, thereby improving analytical capability while controlling complexity
Solution Approach 2:
The ETL-peer provides multi-functional capabilities by handling event detection, data extraction, transformation according to schemas, and loading into external structures. This universal component reduces overall system complexity by consolidating multiple functions into a single mechanism that supports various analytical applications
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The invention relates to a method for a model-driven extraction of event data representing an event occurring on a blockchain network by a computational device with access to the blockchain network. The computational device is configured as an ETL-device for executing an ETL-code to modify a data content of an external data structure external of the blockchain network using the extracted event data. The method comprising detecting the event, determining an event schema, providing and executing a the ETL-code. The ETL- code comprises machine-executable instructions for extracting the event data representing the detected event, transforming the extracted event data using the event schema to comply with a data model defining a logical structure of the external data structure and loading the transformed data to the external data structure to modify the data content of the external data structure.