Data Governance System for Accuracy and Speed Trade-offs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Entities face challenges in controlling and ensuring the accuracy and consistency of significant volumes of data in real-time, necessitating an improved system for data governance that integrates data from multiple platforms and meets emerging data aggregation requirements.
Innovation Solution
A data governance system that receives data from source systems, verifies it against set rules, generates metadata and control statements, and transfers data to authorized sources, while classifying and validating data to ensure accuracy and consistency, and implements actions to eliminate inconsistencies and redundancies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data is received from multiple source systems without verification, then data volume and processing speed are maintained, but data accuracy and consistency deteriorate
Solution Approach 1:
The system performs preliminary verification of data against predefined rules and criteria before data is fully integrated into the enterprise data warehouse. This advance validation ensures data accuracy is maintained without compromising processing speed, as the verification occurs as part of the initial data ingestion process rather than as a subsequent corrective measure.
Solution Approach 2:
The patent introduces an intermediary verification layer between data source systems and the enterprise data warehouse. This intermediary component validates data against predefined rules, schemas, and quality criteria, acting as a buffer that ensures data accuracy while maintaining the flow and processing speed of data through the system.
2Reliability
If data verification and validation processes are implemented, then data accuracy is improved, but system complexity increases
Solution Approach 1:
The system manages complexity by dynamically adjusting verification parameters and rule sets based on data source, data type, and enterprise requirements. Rather than implementing a single complex verification framework, the system modifies verification parameters to match specific data contexts, maintaining data consistency while keeping the overall system architecture manageable and adaptable.
Solution Approach 2:
The verification and validation processes are segmented into distinct, modular components that can be independently configured and maintained. Data verification is separated into multiple stages including initial validation, rule-based checking, and quality assessment, allowing each segment to be optimized independently and reducing overall system complexity through modular design.
3Adaptability or versatility
If data is classified and distributed to multiple authorized sources, then data accessibility and utility are improved, but data management complexity increases
Solution Approach 1:
The patent implements a universal classification framework that categorizes data into standardized domains and types that can be reused across multiple authorized sources and applications. This multi-functional classification system enables the same data structure and categorization logic to serve multiple purposes and data distribution scenarios, improving accessibility while avoiding the need for separate management systems for each data destination.
Solution Approach 2:
The system creates standardized data copies and metadata templates that can be replicated across multiple authorized data sources. Rather than managing unique data relationships for each destination, the system uses copy-based approaches where verified data and its classification metadata are replicated to authorized sources, simplifying management while maintaining data accessibility across the enterprise.
Data Source
AI summary
Embodiments of the invention are directed to systems, methods, and computer program products for verifying, storing, and transferring data within an entity. The system is configured for receiving data from one or more source systems, generating a metadata, exposure, and control statement for the data, transferring the data and the metadata, exposure, and control statement to a system of origination, logically classifying the data in the system of record into one or more domains, transferring the data into one or more authorized data sources associated with the one or more domains, receiving a request from a user associated with a target system to retrieve a set of data from the one or more authorized data sources, and transferring the set of data from the one or more authorized data sources to the target system.