Data Quality Assessment for Financial Transaction Databases
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Business entities, such as financial institutions, face difficulties in defining critical attributes for transactions and assessing data quality due to missing, invalid, or inaccurate data, which necessitates a method to determine and address data quality issues effectively.
Innovation Solution
The method involves collecting data samples from a database, determining critical data elements through analytics and expert input, building data quality rules, monitoring data quality, identifying outliers, and developing corrective actions using statistical and analytical techniques like correlation analysis, regression analysis, and root cause analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If many thousands of data quality rules are created to assess data quality, then data quality assessment completeness is improved, but system complexity and resource consumption worsen
Solution Approach 1:
The patent extracts and identifies critical data elements from the overall data set using analytics and expert input. By focusing only on critical data elements rather than all data elements, the system reduces the number of data quality rules needed while maintaining assessment completeness for the most important data.
Solution Approach 2:
The patent applies different levels of data quality assessment to different data elements based on their criticality. Critical data elements receive comprehensive assessment with dedicated rules, while non-critical elements receive minimal or no assessment, optimizing resource allocation.
2Measurement precision
If comprehensive data quality rules are implemented for all attributes, then data quality assessment accuracy is improved, but processing time and computational resources worsen
Solution Approach 1:
The patent segments the data quality assessment process into distinct phases: critical data element identification, data quality rule building, and monitoring. This segmentation allows the system to focus computational resources on the most critical assessment tasks rather than uniformly processing all data.
Solution Approach 2:
The patent performs preliminary analytics including correlation and regression analysis to identify critical data elements before implementing full data quality rules. This preliminary action reduces the scope of subsequent detailed assessment, saving processing time while maintaining accuracy for critical elements.
3Reliability
If critical data elements are identified using analytics and expert input, then data quality monitoring effectiveness is improved, but analysis complexity worsens
Solution Approach 1:
The patent introduces data quality rules as an intermediary layer between raw data and monitoring outcomes. These rules translate complex analytics and expert knowledge into actionable monitoring criteria, simplifying the overall system while maintaining effectiveness.
4Reliability
If continuous data quality monitoring is implemented, then data reliability is improved, but system resource consumption worsens
Solution Approach 1:
The patent implements continuous data quality monitoring specifically for critical data elements rather than all data. This continuous monitoring maintains high data reliability for critical elements while consuming fewer resources by excluding non-critical elements from continuous assessment.
Data Source
AI summary
Methods and systems for assessing data involve, collecting samples of data elements from a database storing a population of data elements representing attributes of each numerous different financial transactions. Critical data elements from the collected samples are determined. Data quality rules are built and data dimensions are calculated for the critical data elements. A quality of data within the critical data elements for different data quality dimensions is monitored. Critical data elements that produce a high number of outliers are identified and causes for the outliers are identified. Thereafter, a corrective action plan to address a solution for the causes for the outliers may be developed and executed.