Textual Data Classification with POS-Aware Structural Preprocessing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for classifying textual data blocks, such as automatic signature block extraction in emails, are unreliable due to the lack of exact standards and features in signature block representations, leading to poor performance especially when sender's information is absent.
Innovation Solution
A machine learning-based system and method that incorporates an improved textual data preprocessing stage, involving the replacement of textual data elements characterized by specific part-of-speech categories, character sequences, and contextual definitions with respective tokens, followed by embedding into a vector space using TF-IDF measures, to enhance classification reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If predefined features (e.g., sender's name, URL links) are used for classification, then the classification process is simple, but the reliability is poor when sender information is absent
Solution Approach 1:
The patent transforms the classification approach by changing from using predefined semantic features (sender name, URL) to using structural and syntactic parameters (part-of-speech tags, text patterns, formatting characteristics). This parameter transformation enables reliable classification even when traditional sender information is absent, as the structural patterns remain consistent regardless of content variability.
Solution Approach 2:
The patent applies preliminary text processing steps (tokenization, part-of-speech tagging, structural analysis) before classification to extract meaningful features. This preliminary action prepares the data in a way that reveals structural patterns essential for reliable classification, allowing the system to identify signature blocks based on their inherent structure rather than relying on the presence of specific content elements.
2Reliability
If general data preprocessing is applied, then data diversity is reduced and prediction reliability increases, but the preprocessing approach significantly affects model performance
Solution Approach 1:
The patent segments the preprocessing task into distinct, modular components: tokenization, part-of-speech tagging, structural pattern extraction, and feature transformation. Each segment handles a specific aspect of text analysis, making the overall complex preprocessing pipeline manageable and tunable. This segmentation allows optimization of individual steps without overwhelming complexity.
Solution Approach 2:
The patent develops a universal preprocessing framework that handles multiple text types and classification tasks through the same core mechanisms (POS tagging, structural analysis). This multi-functional approach reduces preprocessing complexity by avoiding task-specific customizations while maintaining high prediction reliability across different classification scenarios.
3Measurement precision
If exact standards and features are required for signature block identification, then classification accuracy improves, but such standards do not exist in textual data
Solution Approach 1:
The patent resolves this contradiction by changing from content-based parameters (specific words, sender information) to structural parameters (POS tag sequences, formatting patterns, text organization). This parameter transformation creates de facto standards that capture the essential characteristics of signature blocks without requiring predefined exact standards, thereby maintaining both accuracy and adaptability.
Solution Approach 2:
The patent uses disposable, easily generated structural features (POS tags, syntactic patterns) that can be rapidly computed from any text input. These features serve as temporary but sufficient markers for classification purposes, eliminating the need for persistent, rigid standards while maintaining high classification accuracy through their consistent structural properties.
Data Source
AI summary
A method and a system of classifying textual data blocks are claimed. The method includes receiving at least one textual data block in an original version, including a plurality of textual data elements; performing a preprocessing procedure on the at least one textual data block in the original version, wherein the preprocessing procedure includes replacing the textual data elements characterized by pertinence to at least one specific part-of-speech (POS) category with a respective POS token, thereby obtaining the at least one textual data block in a preprocessed version; inferring a pretrained ML-based model on the at least one textual data block in the preprocessed version, to classify the at least one textual data block by pertinence to the at least one class.