Data Verification System Using Parsing and Batching Modules
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data manipulation and verification systems face challenges in efficiently processing and verifying data across diverse storage platforms, requiring complex interactions and technical expertise, which hinders accessibility and resource efficiency for enterprises.
Innovation Solution
A system comprising a database, parsing module, batching module, comparison module, and reporting module that allows users to process, verify, and report data from multiple sources without programming knowledge, enabling easy comparison and execution of database queries, reducing repetitive interactions and enhancing data management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If employees without programming knowledge need to access and manipulate data, then accessibility is improved, but system complexity increases
Solution Approach 1:
The patent introduces an intermediary system comprising a parsing module, batching module, and comparison module that mediates between users and the database. This intermediary layer handles complex operations like SQL query generation, batch processing, and data verification automatically, allowing users to access data through simplified interfaces without needing to understand underlying programming complexities.
Solution Approach 2:
The system enables self-service data manipulation by automatically generating and executing database queries based on user inputs. The parsing module automatically converts user requirements into SQL queries, the batching module autonomously manages query execution batches, and the comparison module automatically verifies results, eliminating the need for users to manually write or understand programming code.
2Reliability
If multiple data sources need to be processed, then data verification capability is improved, but processing time increases
Solution Approach 1:
The system performs preliminary actions by pre-parsing data from multiple sources and pre-generating comparison queries before actual verification is needed. The parsing module pre-processes data files and stores parsed results, and the batching module pre-prepares batches of queries for execution, reducing the time required during actual data verification operations.
Solution Approach 2:
The system maintains continuity of useful action through the batching module that continuously processes and executes queries in batches. Instead of processing data sources sequentially with idle time between operations, the batching module maintains continuous processing flow by preparing and executing multiple queries in an optimized sequence, reducing total processing time while maintaining verification capability across multiple data sources.
Data Source
AI summary
In an exemplary embodiment, a system includes a database, a parsing module, a batching module, a comparison module, and a reporting module. The parsing module is operable to receive a first data file and a plurality of instructions for processing the data. The parsing module is operable to store at least a portion of data from the first data file in the database based at least in part on the plurality of instructions. The batching module can retrieve database queries from a second data file, receive a first indication that at least one database query is selected, and execute the at least one database query in the database. The comparison module is operable to retrieve a first data set from the database and retrieve a second data set, wherein the second data set is expected data, and determine whether the first data set matches the second data set.