Text Classifier Training Data Generation via Semantic Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generating suitable labeled training data for custom-built text classifiers is challenging due to the need for representative samples and the high cost of time from subject matter experts, making it difficult for non-experts to properly label data samples.
Innovation Solution
A system that assists users in identifying semantically similar and dissimilar text samples by determining term and context embeddings, presenting candidate word groups for labeling, and generating labeled training data based on user input to simplify the training process for text classifiers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If subject matter experts manually label text samples to generate training data, then the quality and reliability of labeled training data is improved, but the time and cost required for labeling increases significantly
Solution Approach 1:
The patent introduces an automated text processing system that acts as an intermediary between raw text samples and expert reviewers. The system performs preliminary processing including identifying candidate word groups, generating semantic representations, and filtering samples before presenting them to experts for final labeling. This intermediary processing layer reduces the volume of work for experts while maintaining data quality.
Solution Approach 2:
The system performs preliminary actions by automatically identifying candidate word groups, generating their semantic representations, and pre-filtering the text corpus before expert involvement. This preliminary processing prepares the data in advance, so that when experts do review samples, they are reviewing pre-processed, organized candidate groups rather than raw unprocessed text, significantly reducing their time investment.
2Reliability
If subject matter experts manually identify and label representative text samples, then the reliability of the text classifier is improved, but the complexity and difficulty of the labeling process increases
Solution Approach 1:
The system enables semi-automated labeling where the automated processing performs most of the work of identifying and preparing candidate word groups, and experts only need to review and confirm the labeling decisions. The system serves itself by automatically generating the structured candidate groups that experts need to review, reducing their operational burden while maintaining reliability.
Solution Approach 2:
The automated processing system acts as an intermediary that transforms raw text into structured candidate word groups with semantic representations, making the labeling task easier for experts. This intermediary layer handles the complex tasks of text parsing, word group identification, and semantic analysis, leaving experts to perform only the simpler task of reviewing and confirming labels.
3Productivity
If non-experts attempt to label text samples without specialized knowledge, then the availability of labeling resources increases, but the accuracy and appropriateness of labeling decreases
Solution Approach 1:
The system performs the complex analytical work itself through automated text processing, including identifying candidate word groups and generating semantic representations. Non-expert reviewers only need to perform the simpler task of reviewing pre-processed candidate groups and making labeling decisions based on the organized presentation, which does not require specialized domain knowledge.
Solution Approach 2:
The automated processing system acts as an intermediary that prepares and structures the text data before presentation to non-expert reviewers. It performs the difficult tasks of text analysis and candidate identification, transforming raw text into organized candidate word groups with contextual information, making the labeling task accessible to non-experts while maintaining accuracy.
4Reliability
If a comprehensive set of labeled samples is generated to cover all categories, then the text classifier becomes more reliable, but the quantity of data processing and labeling work increases
Solution Approach 1:
The system extracts and identifies specific candidate word groups from the larger text corpus that are most relevant for training. Rather than requiring comprehensive manual review of all possible text samples, the automated processing extracts representative candidate groups based on semantic analysis, focusing efforts on the most valuable samples for classifier training.
Solution Approach 2:
The system performs preliminary filtering and identification of candidate word groups before expert review. This preliminary action identifies and prepares only the most relevant samples that need expert attention, reducing the overall volume of work required while ensuring comprehensive coverage of important categories through systematic semantic analysis.
Data Source
AI summary
A method includes receiving input designating a term of interest in a document of a document corpus and determining a target context embedding representing a target word group that includes the term of interest and context words located in the document proximate to the term of interest. The method also includes identifying, from among the document corpus, a first candidate word group that is semantically similar to the target word group and a second candidate word group that is semantically dissimilar to the target word group. The method further includes receiving user input identifying at least a portion of the first candidate word group as associated with a first label and identifying at least a portion of the second candidate word group as not associated with the first label. The method also includes generating labeled training data based on the user input to train a text classifier.