Data Quality Check Module for Automated Trade Validation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data quality issues such as duplicates, under/over representation, missing, and incorrect data in trades and valuations lead to wrong valuation results, often requiring manual fixing and resulting in data loss in subsequent feeds, causing inefficiencies and inaccuracies in collateral valuations.
Innovation Solution
A data quality check module that uses processors and memories to receive distribution event data, access data sources, calculate statistical analysis, create dynamic rules, and verify data against these rules to determine if it's in good order, automatically flagging data as acceptable or not for Straight Through Processing (STP) without manual intervention, allowing for editable rules at runtime without code changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual fixing is performed to correct data quality issues, then data accuracy is improved, but time consumption and operational complexity increase
Solution Approach 1:
The patent implements preliminary data quality checks before processing to identify and flag potential issues in advance. The system performs validation on incoming data streams, checking for duplicates, completeness, and format compliance before the data enters the main processing pipeline, thereby preventing downstream errors and reducing manual intervention needs.
Solution Approach 2:
The system implements automated self-correction mechanisms where common data quality issues are automatically detected and corrected without human intervention. The patent describes automated processes that identify duplicates, fill missing values based on patterns, and correct format errors, allowing the system to service itself and maintain data quality continuously.
2Measurement precision
If manual fixing is performed to correct data quality issues, then data accuracy is improved, but device complexity and operational difficulty increase
Solution Approach 1:
The system implements automated self-correction mechanisms where common data quality issues are automatically detected and corrected without human intervention. The patent describes automated processes that identify duplicates, fill missing values based on patterns, and correct format errors, allowing the system to service itself and maintain data quality continuously.
Solution Approach 2:
The patent implements feedback loops where data quality metrics are continuously monitored and fed back to the processing system. When quality thresholds are not met, the system automatically adjusts processing parameters or triggers re-validation, creating a closed-loop control system that maintains data quality without requiring complex manual oversight.
3Measurement precision
If manual intervention is used to fix data issues, then data quality is improved, but productivity decreases
Solution Approach 1:
The system implements automated self-correction mechanisms where common data quality issues are automatically detected and corrected without human intervention. The patent describes automated processes that identify duplicates, fill missing values based on patterns, and correct format errors, allowing the system to service itself and maintain data quality continuously.
Solution Approach 2:
The patent implements preliminary data quality checks before processing to identify and flag potential issues in advance. The system performs validation on incoming data streams, checking for duplicates, completeness, and format compliance before the data enters the main processing pipeline, thereby preventing downstream errors and reducing manual intervention needs.
4Measurement precision
If data is manually fixed, then data accuracy is improved, but data loss occurs in subsequent feeds
Solution Approach 1:
The patent implements preliminary data quality checks before processing to identify and flag potential issues in advance. The system performs validation on incoming data streams, checking for duplicates, completeness, and format compliance before the data enters the main processing pipeline, thereby preventing downstream errors and reducing manual intervention needs.
Solution Approach 2:
The patent implements feedback loops where data quality metrics are continuously monitored and fed back to the processing system. When quality thresholds are not met, the system automatically adjusts processing parameters or triggers re-validation, creating a closed-loop control system that maintains data quality without requiring complex manual oversight.
Data Source
AI summary
Various methods, apparatuses/systems, and media for implementing a data quality check module for determining whether data is acceptable or not are disclosed. A processor collects all facts data corresponding to an agreement based on corresponding received distribution event data in accordance with margin requirements associated with the agreement and calculates statistical analysis data based on historical data points of the facts data for the agreement. The processor also creates and configures dynamic rules that are required to be applied for determining whether the agreement is in good order; verifies the statistical analysis data against the rules to determine anomaly data; marks the agreement as a good order agreement flag when it is determined that the anomaly data is within a predetermined threshold value; and automatically executes, in response to marking the agreement as a good order agreement flag, a straight through processing of the agreement.