Attribute-Level Data Scrubbing for Golden Copy Reconciliation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methodologies for Corporate Action Announcement processes face inconsistencies and inaccuracies due to multiple heterogeneous sources providing varying reliability of information, lacking a robust solution for automating data cleansing and scrubbing at the field level, and failing to present a unified view of unformatted and unstructured data from diverse sources, making it difficult to identify the best data for the Golden Copy.
Innovation Solution
A computer-implemented method and system for data scrubbing at the attribute level, involving the receipt of data from distributed sources, applying a ranking matrix process to compute combined weights, and determining promotion and confirmation thresholds to decide which attributes are promoted to the Golden Copy, enabling users to manage narratives and conflicts from a single screen.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple heterogeneous sources are used to procure data for Corporate Action Announcements, then data coverage and source diversity are improved, but data consistency and accuracy deteriorate due to varying reliability and formatting across sources
Solution Approach 1:
The system segments data processing at the field level by creating separate data elements for each attribute (e.g., Record Date, Rate of Interest) and processing them independently through standardized templates. This allows heterogeneous sources to be handled uniformly at the field level while maintaining overall data consistency across the Golden Copy.
Solution Approach 2:
The system changes the parameter of data representation by transforming all incoming data from heterogeneous sources into a standardized set of fields with consistent formatting. Each data element is mapped to a standardized template structure, converting varying source formats into uniform representations that ensure consistency in the Golden Copy.
2Ease of operation
If source level precedence is applied for incoming messages, then processing simplicity is improved, but the ability to choose the best data from multiple sources deteriorates
Solution Approach 1:
The system segments the data selection process by evaluating each field independently rather than applying uniform source precedence to entire messages. This allows the system to choose the best data for each attribute based on its specific reliability and quality, rather than being constrained by overall source ranking.
Solution Approach 2:
The system incorporates feedback mechanisms through configurable thresholds and validation rules that continuously monitor data quality and source reliability. This feedback allows dynamic adjustment of data selection based on real-time assessment of data quality, enabling the system to choose the best available data rather than relying on fixed source precedence.
3Productivity
If automated data processing is implemented, then processing efficiency is improved, but the ability to manage and reconcile conflicting narratives from multiple sources deteriorates
Solution Approach 1:
The system segments the reconciliation process by handling each data field independently through standardized templates and validation rules. This segmentation allows automated processing to efficiently handle routine fields while complex narrative conflicts are identified and managed through the same standardized framework, reducing overall system complexity.
Solution Approach 2:
The system changes the parameter of data representation by transforming all narratives into standardized fields with consistent formatting and validation rules. This parameter transformation simplifies the reconciliation process by providing a uniform structure for comparing and validating data from multiple sources, making automated processing more effective.
4Reliability
If field level data scrubbing is implemented, then data quality and consistency are improved, but the complexity of configuring and managing the scrubbing process deteriorates
Solution Approach 1:
The system changes the parameter of data representation by providing standardized templates for each field that define the expected format, validation rules, and scrubbing criteria. This parameter standardization simplifies the configuration process by providing pre-defined templates rather than requiring custom scrubbing rules for each field, while maintaining high data quality through consistent validation.
Data Source
AI summary
A system and method enabling automated data cleansing and scrubbing at the attribute level is disclosed. A consolidated view may be provided of the scrubbed data or narratives that gets promoted to a final copy and the data or narratives received from multiple sources on a single user interface.