Double-Blind Data Verification for Machine Learning Quality Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automated data quality control methods using machine learning and artificial intelligence face challenges due to limited data availability and inefficiencies in verifying data accuracy, requiring excessive time and resources.
Innovation Solution
A computer-implemented system for double blinded verification that includes a data extraction module, verification module, comparison module, and quality check module, utilizing machine learning to extract and verify data points across multiple levels by different users, enabling accurate and efficient quality control through a multi-level verification process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated technology such as machine learning and artificial intelligence techniques are used for quality control of data, then productivity is improved, but reliability deteriorates due to limited data availability and insufficient verification accuracy
Solution Approach 1:
The verification process is segmented into multiple independent levels: first-level verification by multiple first users, second-level verification by multiple second users, and third-level verification by multiple third users. Each level independently verifies different aspects of the data, with the number of verifiers at each level being different. This segmentation allows automated technology to process data efficiently at each stage while maintaining reliability through layered independent verification.
Solution Approach 2:
The system implements feedback mechanisms where verification results from each level are fed back to subsequent levels. Third users verify not only the original extracted data but also the verification results from first and second users. This multi-level feedback loop ensures that errors are detected and corrected at appropriate stages, maintaining high reliability while allowing automated processing to maintain productivity.
2Reliability
If multiple levels of verification are implemented to improve data accuracy, then reliability is improved, but loss of time increases due to excessive verification steps
Solution Approach 1:
The system applies partial verification actions at each level rather than requiring complete re-verification of all data. First users verify initial extracted data, second users verify results from first users, and third users verify results from both first and second users. This partial action approach ensures reliability through multiple verification levels while reducing time loss by avoiding redundant full verifications at each stage.
Solution Approach 2:
The system performs preliminary verification actions at earlier levels to catch and correct errors before they propagate. By having first users perform initial verification and second users perform intermediate verification, potential errors are addressed preliminarily, reducing the burden on third-level verification and overall verification time while maintaining reliability.
3Reliability
If manual data verification is performed to ensure data accuracy, then reliability is improved, but productivity deteriorates due to excessive time and expense requirements
Solution Approach 1:
The system merges automated data extraction technology with multi-level manual verification processes. Automated machine learning models extract data points from documents efficiently, while multiple users at different levels independently verify the extracted data. This combination maintains high productivity through automated extraction while ensuring reliability through structured manual verification at critical stages.
Solution Approach 2:
The verification system acts as an intermediary layer between automated data extraction and final data usage. Multiple users at different verification levels independently check extracted data points, serving as intermediaries that ensure accuracy without completely replacing automated processing. This intermediary verification maintains productivity by allowing automated extraction to proceed while ensuring reliability through targeted manual review.
Data Source
AI summary
A system to perform quality control of data by double blinded verification is disclosed. The system includes a processing subsystem which includes a data extraction module for parsing one or more documents to extract a plurality of data points by using machine learning model to generate a first transaction and a plurality of sub-transactions, a verification module verifies each of the plurality of sub-transactions individually by a first user and a second user respectively, a comparison module compares the verified results of the plurality of sub-transactions to identify a plurality of differences and filters and marks the plurality of differences in the verified results, a quality check module generates a second transaction by using the machine learning model, assigns the second transaction with the marked differences to a third user for a subsequent review, and updates the plurality of datapoints in response to the review made by a third user.