Machine Learning Models for Tabular Data Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automated classification of electronic data stored in tables is challenging due to varied formats, syntax, and lack of metadata, making it difficult for conventional techniques to accurately classify sensitive data items.
Innovation Solution
The use of separate machine learning models trained on features extracted from both values and column headers, with iterative weight adjustment for aggregation, to predict data classifications, enabling accurate automated classification even in the absence of metadata.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional automated classification techniques are used, then the classification process is simple and fast, but the classification accuracy is low due to varied formats, syntax, and lack of metadata
Solution Approach 1:
The patent divides the classification task into two separate machine learning models: one model analyzes column headers while the other analyzes data values. This segmentation allows each model to specialize in specific features, improving overall classification accuracy while managing complexity through modular architecture
Solution Approach 2:
The patent extracts multiple features from both column headers and data values, adding dimensional information beyond simple text matching. This includes analyzing text content, data types, patterns, and relationships, thereby enhancing classification accuracy through multi-dimensional feature space
2Measurement precision
If separate machine learning models analyzing both column headers and values are used, then classification accuracy is improved, but computational complexity and processing time increase
Solution Approach 1:
The patent performs feature extraction from column headers and data values in advance, before the actual classification decision is made. This preliminary processing organizes and structures the input data, enabling the machine learning models to work more efficiently and reduce overall processing time
Solution Approach 2:
The patent combines the outputs from two separate machine learning models (one for column headers, one for values) into a unified classification decision. This merging approach leverages the strengths of both models while achieving accurate classification through integrated analysis
3Measurement precision
If manual classification is performed, then classification accuracy is high, but productivity and efficiency are reduced
Solution Approach 1:
The patent implements automated machine learning models that perform classification independently without human intervention. The models self-learn from training data and automatically classify new data, achieving both high accuracy and high productivity by eliminating manual classification processes
Data Source
AI summary
Aspects of the present disclosure provide techniques for automated data classification through machine learning. Embodiments include providing first inputs to a first machine learning model based on a column header of a column from a table and receiving a first output from the first machine learning model in response to the first inputs, wherein the first output indicates a first likelihood that the column relates to a given classification. Embodiments include providing second inputs to a second machine learning model based on a value from the column and receiving a second output from the second machine learning model in response to the second inputs, wherein the second output indicates a second likelihood that the value relates to the given classification. Embodiments include determining whether to associate the value with the given classification based on the first output and the second output.