Cloud-Based Global Data Validation Framework for Selective Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data verification and validation processes are resource-intensive and applied universally to all technical operations within a job task, leading to inefficient resource utilization and limited job task performance.
Innovation Solution
A cloud-based validation framework that selectively performs validation on user-configurable technical operations, generating a dictionary file for each operation and providing output in CSV or Parquet format, with options for file check, duplicate check, and custom validations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If verification and validation operations are coded and integrated directly into all job tasks, then data manipulation reliability is improved, but resource consumption and device complexity increase significantly
Solution Approach 1:
The patent segments verification and validation operations from job tasks into a separate, standalone framework. This framework can be selectively applied to specific technical operations rather than being universally integrated into all job tasks, thereby reducing system complexity while maintaining reliability where needed.
Solution Approach 2:
The patent extracts verification and validation coding from within job tasks and places it in an external, independent framework. This extraction allows the organization to control resource allocation and apply validation only to operations that require it, reducing overall system complexity and resource consumption.
2Reliability
If verification and validation operations are applied to all technical operations in a job task, then data quality is improved, but resource utilization efficiency deteriorates
Solution Approach 1:
The patent implements local quality by allowing verification and validation to be applied selectively to specific technical operations based on their individual requirements. Rather than uniformly applying validation to all operations, the framework enables targeted validation only where data quality concerns exist, optimizing resource utilization while maintaining necessary data quality standards.
3Reliability
If verification and validation coding is performed at the job task level, then comprehensive data validation is achieved, but the number of job tasks that can be performed is limited
Solution Approach 1:
The patent creates a universal validation framework that can serve multiple job tasks simultaneously. This standalone framework is designed to be reusable across different job tasks and technical operations, allowing comprehensive data validation to be achieved without limiting the organization's ability to perform multiple job tasks in parallel, thereby improving overall productivity.
Data Source
AI summary
A method for selective performing a validation in one or more technical operations by a cloud based validation framework is provided. The method includes identifying a job task to be performed, identifying technical operations associated with the job task; receiving user configurable input data for performing validation on at least one technical operation associated with the job task; and submitting the user configurable input data into the cloud network based validation framework. The method further includes determining, by the cloud network based validation framework, at least one technical operation specified for performing a validation function among the technical operations; generating a dictionary file for the at least one technical operation specified for performing the validation function; and performing the validation function specified by the dictionary file.