Modular Data Analysis Schema for Consistency Checks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data consistency analysis methods are labor-intensive, specific to particular scenarios, and often not reusable, making it difficult to detect and address data inconsistencies across data storage systems, which can lead to errors and loss of confidence in data integrity.
Innovation Solution
A modular data analysis schema that breaks down data analysis into separate components such as data sources, check methods, and data evaluators, allowing users to select and combine these components to create reusable analysis procedures, facilitating routine and automated checks for data consistency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual data consistency analysis is performed using custom scripts, then data consistency can be detected, but the process becomes labor-intensive and not reusable
Solution Approach 1:
The patent segments data analysis into separate reusable components including data sources, check methods, and data evaluators. This modular architecture allows each component to be independently developed, tested, and reused across multiple analysis scenarios, eliminating the need to create custom scripts for each analysis task and significantly reducing analysis time while maintaining consistency detection capability.
Solution Approach 2:
The patent creates a library of pre-built check methods that can be copied and reused across different data analysis scenarios. Instead of writing custom analysis scripts for each situation, users can select from existing check methods and combine them with relevant data sources, making the analysis process reusable and dramatically reducing the time required for data consistency detection.
2Reliability
If custom analysis scripts are created for specific scenarios, then data analysis can be performed, but the solutions are not reusable and require significant effort to create
Solution Approach 1:
The patent divides data analysis into discrete, reusable segments including data sources, check methods, and data evaluators. This segmentation allows users to build analysis solutions by assembling pre-fabricated components rather than creating custom scripts from scratch, significantly reducing creation effort while maintaining the ability to perform specialized analysis when needed.
Solution Approach 2:
The patent creates a universal framework where check methods can be applied to multiple different data sources and scenarios. A single check method can serve multiple purposes across different datasets and analysis requirements, eliminating the need to create separate custom scripts for each scenario and making the analysis creation process much easier.
3Reliability
If data analysis is performed manually, then specific data issues can be identified, but the process is not automated and requires continuous human intervention
Solution Approach 1:
The patent implements a self-service system where the data analysis framework automatically performs consistency checks without requiring continuous human intervention. Once check methods are configured and data sources are connected, the system autonomously executes analysis, identifies errors, and generates reports, maintaining high error detection capability while achieving full automation.
Solution Approach 2:
The patent incorporates feedback mechanisms where check methods automatically evaluate data against defined criteria and provide results back to the system. This automated feedback loop enables continuous monitoring and detection of data consistency issues without manual intervention, while the structured feedback format ensures reliable error identification.
Data Source
AI summary
An identifier of a data source specifying data is determined from a schema. The data source is called, and data to be analyzed is retrieved. From the schema, an identifier of a method is determined. The method associated with the identifier includes one or more rules. The method is called and used to determine data consistency with the one or more rules.