Pseudo-Sentence Classification for Faster Tabular Data Labeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for separating and labeling tabular data in big data systems face challenges such as manual separation being time-consuming, the heterogeneous nature of data requiring domain expertise, and the inefficiency of AI/NLP-based systems in retraining with new data types, especially due to erroneous labeling and the time-consuming nature of model retraining.
Innovation Solution
A machine learning framework that includes a classifier model trained with augmented training data through sampling and shuffling to generate pseudo sentences, allowing for efficient labeling of tabular data types, with features extracted from columns and updated using an update function to enhance accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual separation of tabular data is performed, then data can be accurately separated and labeled, but the process becomes time-consuming and inefficient
Solution Approach 1:
The patent replaces manual mechanical data separation and labeling processes with an automated machine learning system. The system uses feature extraction models to convert tabular data into pseudo-sentences, which are then processed by classifier models to automatically generate labels, eliminating the need for manual human effort while maintaining or improving accuracy through consistent algorithmic application.
Solution Approach 2:
The machine learning system performs self-service by automatically extracting features from tabular data, generating pseudo-sentences, classifying the data types, and producing labels without requiring continuous human intervention. The system trains on existing labeled data and independently processes new data, making the data separation process autonomous and efficient.
2Adaptability or versatility
If AI and NLP based systems retrain models with new data types, then the system can handle new data types, but the retraining process is time-consuming and may produce erroneous labels
Solution Approach 1:
The system performs preliminary actions by pre-processing tabular data into feature sets and then into pseudo-sentences before classification. This structured preprocessing pipeline is established in advance and can be quickly applied to new data types without requiring complete model retraining, as the feature extraction and pseudo-sentence generation steps remain consistent across different data types.
Solution Approach 2:
The patent segments the machine learning process into distinct modular components: feature extraction, pseudo-sentence generation, and classification. This segmentation allows the system to adapt to new data types by updating or retraining only the classification component while reusing the established feature extraction and pseudo-sentence generation pipelines, reducing overall retraining time and improving adaptability.
3Adaptability or versatility
If heterogeneous data from different sources is processed, then comprehensive data coverage is achieved, but domain expertise is required and complexity increases
Solution Approach 1:
The patent implements a universal feature extraction model that processes diverse tabular data from different sources through a common pipeline. The model extracts features and generates pseudo-sentences in a standardized format that can be handled by the classifier, allowing the system to accommodate heterogeneous data types without requiring separate processing logic for each data source, thus managing complexity while maintaining versatility.
Data Source
AI summary
A method implements extraction using a machine learning framework. The method includes extracting a set of features from a table. The method further includes applying a feature extraction model to the set of features to generate a pseudo sentence. The method further includes applying a classifier model to the pseudo sentence to generate a label prediction for the table. The classifier model is trained with an augmented training pseudo sentence created by one or more of sampling and shuffling. The method further includes updating a label of the table with the label prediction.