Standby Database Replication for Real-Time Encrypted Data Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Running complex queries on a transactional database is a time-consuming process that impacts the performance of live applications, particularly in environments with multiple data sources and encrypted data objects.
Innovation Solution
Implement a system with a standby database configured to store rule and data schemas, using a change data capture process to monitor updates, extract encrypted data, decrypt it, and reload it to the standby database in real-time, utilizing a data extraction engine like Pega BIX for efficient replication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If complex queries are run on the transactional database to extract data, then data extraction capability is improved, but performance of live applications deteriorates
Solution Approach 1:
The system divides the database into two separate instances: a transactional database that handles live application operations and a standby database that handles data extraction. This segmentation allows complex extraction queries to run on the standby database without impacting the performance of live applications on the transactional database.
Solution Approach 2:
A standby database is created as a copy of the transactional database, storing replicated data that can be used for extraction purposes. The standby database maintains the same data structure and content but serves a different function, allowing extraction operations without affecting the original transactional database.
2Reliability
If encrypted data is extracted and decrypted in real-time, then data security is improved, but processing time increases
Solution Approach 1:
The system performs data decryption in advance by extracting encrypted data from the standby database and decrypting it before it is needed for analysis. This preliminary action ensures that decrypted data is readily available when needed, reducing processing time during actual data retrieval operations while maintaining security through proper encryption storage.
Data Source
AI summary
An example computer system includes a transactional database configured to store multiple data sources each corresponding to one of multiple applications, a standby database configured to store multiple rule schemas and multiple data schemas, and processor hardware configured to execute instructions to monitor for a change to stored data in one of the multiple data sources, via a change data capture process, replicate data from the transactional database to the standby database in response to an update indicated by the change data capture process, identify encrypted data in at least one data object of an updated data table of one of the data schemas of the standby database, extract data from the at least one data object to store decrypted data, wherein the standby database is an external database for the data extraction engine, and reload the decrypted data to one of the data schemas of the standby database.