Automated Snapshot Reconciliation With Database-Triggered Support Pipelines

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Reconciling large quantities of continually updating data in enterprises is time-consuming, resource-intensive, and prone to errors due to a lack of automation and manual processes, particularly in data management systems.

Innovation Solution

An automated snapshot reconciliation process with multi-threading and logging enhancements, including concurrent execution of database pools, dynamic data masking, and support pipelines to automate repetitive tasks, ensuring efficient data management and reconciliation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If manual processes are used for data reconciliation, then human oversight and control are maintained, but the process becomes time-consuming and resource-intensive

Engineering Contradiction:

Improvedata reconciliation accuracyVSAvoidreconciliation time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system implements self-service automation where the data reconciliation process executes automatically without manual intervention. The automated system performs data comparison, discrepancy identification, and reconciliation operations autonomously, eliminating the need for manual human processes while maintaining reliability through systematic validation protocols

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent replaces manual mechanical processes with automated computational systems. Human operators are substituted by computer-executable instructions that systematically perform data reconciliation tasks, using algorithmic logic to compare datasets, identify discrepancies, and execute corrections without human physical intervention

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Adaptability or versatility

If manual data reconciliation processes are used, then flexibility in handling complex scenarios is maintained, but errors increase and automation is lacking

Engineering Contradiction:

Improveprocess flexibilityVSAvoiderror rate

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The automated system incorporates feedback mechanisms that systematically validate data against predefined criteria and rules. The system continuously monitors reconciliation processes, compares actual results against expected outcomes, and automatically adjusts operations to maintain accuracy, providing consistent error prevention without sacrificing adaptability

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system maintains flexibility by allowing dynamic adjustment of reconciliation parameters, data sources, and validation rules through configurable settings. These parameter changes enable the system to adapt to different scenarios and data types while maintaining automated execution, preventing human error through consistent parameter application

Inventive Principle:

Principle #35Parameter changes

3Productivity

If concurrent processing across multiple database pools is implemented, then processing speed increases, but system complexity increases

Engineering Contradiction:

Improvereconciliation speedVSAvoidsystem architecture complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The system segments the data reconciliation workload into multiple independent database pools, each handling specific portions of the data. This segmentation enables concurrent processing where multiple pools operate simultaneously, increasing overall productivity while managing complexity through modular, independent units that can be scaled and maintained separately

Inventive Principle:

Principle #1Segmentation

4Reliability

If data stewards perform repetitive manual tasks, then data quality control is maintained, but time consumption and resource usage increase

Engineering Contradiction:

Improvedata qualityVSAvoidoperational efficiency

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The system implements self-service automation where automated processes perform data quality control tasks that previously required manual intervention by data stewards. The system autonomously validates data quality, identifies discrepancies, and executes corrections, maintaining reliability through systematic validation while dramatically improving operational efficiency by eliminating repetitive manual labor

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS20250298802A1Computer System And Method For Automating Support Operations In A Data Management System

Publication Date: 2025.09.25 THE TORONTO DOMINION BANK

AI summary

A system and method are provided for executing supporting operations in a data management system. The method includes assigning a support pipeline to each of at least one repetitive data treatment task; automatically generate a database query for each support pipeline, each database query applying a corresponding operation to data in a database used by the data management system; and initiating each support pipeline to be triggered by database operations.