Document Classification With Attribute-Based Anchor Word Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing document classification models using machine learning struggle to effectively utilize supplementary short texts, such as definition sentences, due to differences in word feature quantities between the short texts and input documents, leading to reduced classification accuracy.
Innovation Solution
A document classification apparatus that divides sentences into word units, calculates appearance statistics, provides attributes for specialty, specificity, and category independence, extracts anchor words, and performs target learning to generate word embedding vectors for improved classification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If supplementary short texts are used for training the document classification model, then the model can utilize domain-specific knowledge, but the classification accuracy deteriorates due to excessive weighting of short text words or burying of important keywords
Solution Approach 1:
The patent applies local quality by differentiating the treatment of words based on their attributes. Words are classified into positive examples (anchor words with high specialty/specificity) and negative examples (other words) based on their attribute scores. This selective differentiation allows the model to focus on high-value words from supplementary texts while avoiding the harmful effects of treating all words equally, thus resolving the contradiction between utilizing domain knowledge and maintaining classification accuracy.
Solution Approach 2:
The patent segments the training process into distinct phases: first training with only input documents to establish a baseline word embedding space, then selectively incorporating supplementary text words as positive examples during fine-tuning. This segmentation allows the model to first learn general patterns and then selectively integrate domain-specific knowledge, preventing the supplementary text from overwhelming the model with its distinctive word features.
2Ease of manufacture
If all words in supplementary text are treated equally in the training process, then the training process is simple, but important keywords are buried in general words reducing classification performance
Solution Approach 1:
The patent implements local quality by assigning different roles to different words based on their attributes. Words with high specialty and specificity scores become anchor words (positive examples), while other words become negative examples. This attribute-based differentiation automatically identifies important keywords without complex manual curation, maintaining training simplicity while significantly improving classification performance by preventing keyword burying.
Data Source
AI summary
A document classification apparatus includes a processing circuit. The processing circuit configured to: divide a sentence included in a target document into word units; divide a sentence included in a supplementary text into word units; calculate an appearance statistic of a word obtained from the target document and the supplementary-text; provide, to the word, an attribute representing specialty, specificity, or category independence based on the appearance statistic; extract an anchor word from words included in the supplementary text based on the attribute; perform target learning in accordance with an attribute of the anchor word and generate a word embedding vector; and estimate a classification category of a document based on the word embedding vector.