Automated Metadata Validation via Explain Plan Inspection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Manual validation of metadata in large databases and data warehouses is time-consuming and resource-intensive, often leading to undetected errors until they cause invalid searches, which necessitates an inefficient iterative problem resolution process.
Innovation Solution
A method for automatically validating metadata by receiving a request, acquiring relevant metadata, generating operations that consume it, and validating these operations without execution, using an explain plan operation to inspect syntax and semantics, thereby identifying and addressing errors efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual validation of metadata is performed, then errors can be detected, but the process becomes time-consuming and resource-intensive
Solution Approach 1:
The patent applies preliminary action by generating and executing validation queries against the metadata before actual data operations are performed. The system proactively validates metadata definitions, relationships, and constraints in advance, so that errors are detected before they cause problems in data processing or analysis operations.
Solution Approach 2:
The patent implements self-service through automated validation processes that do not require manual intervention. The system automatically generates validation queries, executes them against the metadata, and identifies errors without human involvement, replacing the manual validation process with an autonomous automated system.
2Reliability
If manual construction of validation searches is performed, then metadata errors can be identified, but human resources and processing resources are limited
Solution Approach 1:
The system performs self-service by automatically generating validation queries and executing them without human intervention. The automated process continuously validates metadata definitions, relationships, and constraints, eliminating the need for manual construction of validation searches and significantly increasing validation throughput.
Solution Approach 2:
The patent replaces the mechanical manual process of constructing and executing validation queries with an automated computer-based system. The system automatically generates validation searches based on metadata definitions and executes them programmatically, substituting human manual work with automated computational processes.
3Productivity
If metadata validation is automated, then validation time is reduced, but the complexity of the validation system increases
Solution Approach 1:
The patent introduces an intermediary validation system that sits between the metadata definitions and the data operations. This intermediary automatically generates validation queries based on metadata definitions and executes them to detect errors, mediating between the metadata and operational systems while maintaining modularity and managing complexity.
Solution Approach 2:
The patent replaces complex manual validation processes with automated computational processes. The automated system uses computer algorithms to generate and execute validation queries, substituting the complexity of manual processes with manageable computational complexity that can be automated and scaled.
Data Source
AI summary
An application includes a variety of metadata and operations that may process the metadata. Selective portions of the metadata are identified and various combinations of available operations are automatically generated for the selective metadata. Each generated operation is then individually validated for syntax and semantics, without actually executing the operation.