AI Model for Erroneous Cell Detection in Large Datasets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large databases cannot be manually analyzed for errors within a reasonable time, necessitating automated error detection methods to prevent the output of 'garbage data' with adverse downstream effects.
Innovation Solution
A computer-implemented method using an AI model trained to identify relationships between cells in a dataset to classify erroneous cells, employing an attention mechanism and neural network architecture to classify focus cells based on their context within a row.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual analysis is used to detect errors in large databases, then measurement precision can be maintained, but productivity becomes unacceptably low and analysis time becomes unreasonable
Solution Approach 1:
The patent introduces an AI model as an intermediary between the raw data and the error detection process. The model takes cell values and relationships as input, processes them through learned patterns, and outputs error classifications. This intermediary enables automated high-speed processing while maintaining accuracy through the model's ability to learn complex error patterns from training data.
Solution Approach 2:
The patent replaces manual mechanical analysis with an automated AI-based system. Instead of human analysts manually examining data cells, the system uses an AI model that automatically identifies relationships between cells and classifies errors, substituting the mechanical human analysis process with an automated computational approach that achieves both speed and accuracy.
2Productivity
If automated error detection methods are implemented, then productivity increases, but the complexity of the detection system increases
Solution Approach 1:
The patent segments the error detection task into distinct components: relationship identification between cells and error classification based on those relationships. The AI model processes these segments separately, first identifying how cells relate to each other, then using those relationships to classify whether a focus cell contains an error. This segmentation simplifies the overall system complexity by breaking down the complex detection problem into manageable parts.
3Measurement precision
If comprehensive relationship analysis between cells is performed, then measurement precision improves, but computing time and processing speed increase
Solution Approach 1:
The patent performs preliminary action by pre-training the AI model on extensive training data that encompasses various cell relationships and error patterns. During the actual error detection phase, the model already possesses learned knowledge about cell relationships, allowing it to quickly classify errors without performing exhaustive relationship analysis from scratch. This preliminary training action enables fast processing while maintaining high accuracy.
Data Source
AI summary
Classification of cell data includes obtaining a target dataset and an artificial intelligence (AI) model trained to identify relationship(s) between cells of a row and classify whether a focus cell of the row is erroneous based on the identified relationship(s), and applying the AI model to the target dataset to identify erroneous cell(s) thereof. The applying includes selecting a row of cells of the target dataset, inputting the selected row of cells to the AI model with an identification of a focus cell, the focus cell to be classified by the AI model, classifying the focus cell to obtain a classification of the focus cell, the classifying identifying whether the focus cell is erroneous, and outputting an indication of the classification of the focus cell.