Homogeneous Transaction Data Store for Heterogeneous OLTP Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current OLTP databases and data warehouses are not optimized for tasks such as comparing data between periods, tracing specific activities, showing data evolution, identifying interesting transactions, and providing historical operational changes, due to limitations in maintaining comprehensive historical information and transactional context.
Innovation Solution
A system that creates a homogeneous transaction data store from heterogeneous sources by converting committed transaction records into a common format, allowing for efficient querying and storage of transaction history, with a built-in search system and metadata repository for efficient data retrieval and analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If OLTP databases store comprehensive historical information, then historical analysis capability is improved, but storage cost and system complexity increase
Solution Approach 1:
The system segments historical data management by separating OLTP databases from the transaction data store. OLTP databases maintain only current operational data while the transaction data store preserves comprehensive historical transaction records, allowing historical analysis without burdening operational systems
Solution Approach 2:
The transaction data store acts as an intermediary between OLTP systems and analytical applications. It captures transaction context from OLTP databases and provides historical data to analytical tools, eliminating the need for OLTP databases to store comprehensive history
2Loss of time
If data is moved periodically from OLTP to data warehouse, then data freshness is improved, but historical transaction context is lost
Solution Approach 1:
The system performs preliminary capture of transaction context at the point of transaction commit in OLTP databases. Transaction context is captured and stored in the transaction data store immediately when transactions occur, preserving complete historical context before periodic data warehouse updates
Solution Approach 2:
The transaction data store continuously captures and preserves transaction context from OLTP systems in real-time, providing unbroken historical continuity. This continuous capture ensures no transaction context is lost between periodic ETL cycles
3Adaptability or versatility
If heterogeneous data sources are integrated, then data comprehensiveness is improved, but data format complexity increases
Solution Approach 1:
The system applies homogeneity by converting all transaction records from heterogeneous data sources into a unified transaction context format. The standardized schema normalizes diverse source formats while preserving essential transaction details, enabling consistent analysis across multiple sources
Data Source
AI summary
A computer readable storage medium includes executable instructions to gather first committed transactional record from a first transaction log associated with a first data source. The first committed transactional record is converted to a homogeneous format to form a first homogeneous transactional record with a common transaction record header containing transactional context information from the first data source. A second committed transactional record is collected from a second transaction log associated with a second data source with a format different than the first data source. The second committed transactional record is changed to the homogeneous format to form a second homogeneous transactional record with a common transaction record header containing transactional context information from the second data source. The first homogeneous transactional record is combined with the second homogeneous transactional record to form a homogeneous transaction data store.