Data Asset Collections for Dynamic Governance Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data governance systems struggle to efficiently verify that vast amounts of data managed by data processing systems comply with data governance policies, due to the dynamic nature of data assets and the manual effort required to associate data assets with applicable data standards.
Innovation Solution
The system automatically generates data asset collections based on predefined criteria, associates these collections with data standards, and identifies responsible users to attest compliance, using attribute-value pairs within the data assets to facilitate dynamic updates and efficient verification processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual methods are used to associate data assets with data standards, then flexibility and control are maintained, but the verification process becomes inefficient and time-consuming
Solution Approach 1:
The system enables data assets to automatically associate themselves with applicable data standards through self-service mechanisms. The data processing system autonomously identifies which data standards apply to which data assets based on predefined criteria, eliminating the need for manual association while maintaining accuracy and consistency.
Solution Approach 2:
The system performs preliminary actions by pre-configuring data standards with their associated criteria and rules before the verification process begins. This allows the system to automatically match data assets with appropriate standards without requiring manual intervention during the actual verification process, significantly improving efficiency.
2Productivity
If automated verification is implemented across vast amounts of data, then verification efficiency improves, but system complexity increases
Solution Approach 1:
The system segments the verification process into distinct components: data asset identification, criteria evaluation, standard matching, and compliance verification. Each component handles a specific aspect of the verification process, making the overall complex system manageable through modular design and reducing the cognitive load on users.
Solution Approach 2:
The system introduces an intermediary layer consisting of predefined criteria that act as mediators between raw data assets and data standards. This intermediary layer simplifies the matching process by providing clear, rule-based criteria that automatically determine which standards apply to which data assets, reducing system complexity while maintaining automation.
3Measurement precision
If data asset collections are dynamically updated to reflect current data, then verification accuracy improves, but processing overhead increases
Solution Approach 1:
The system implements periodic action by updating data asset collections at scheduled intervals or triggered by specific events rather than continuously. This approach maintains verification accuracy by ensuring data assets are regularly synchronized with current data while reducing processing overhead by avoiding constant updates.
Solution Approach 2:
The system maintains continuity of useful action by keeping data asset collections in a consistently usable state through efficient update mechanisms. Once updated, the collections remain valid and ready for verification without requiring constant reprocessing, balancing accuracy with processing efficiency.
Data Source
AI summary
Some embodiments relate to a method for use in connection with governance of a plurality of data assets managed by a data processing system, the method comprising: using at least one computer hardware processor to perform: accessing a data governance policy comprising a first data standard (e.g., by obtaining information about the first standard stored in a database system); generating a first data asset collection at least in part by automatically selecting, from among the plurality of data assets managed by the data processing system and using at least one data asset criterion, one or more data assets that meet the at least one data asset criterion; associating the first data asset collection with the first data standard; and verifying whether at least one of the one or more data assets in the first data asset collection complies with the first data standard.