Token-Level Argument Mining with Adaptive Transformer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing argument mining techniques struggle to effectively identify argument components and their relationships in essays written by middle school students, due to differences in writing conventions, such as run-on sentences, unsupported claims, and multiple claims in a single sentence.
Innovation Solution
A token-level classification approach using a BERT-based multi-task learning architecture, which combines adaptive pretraining on relevant unlabeled data with fine-tuning on labeled datasets, to identify claim and premise tokens directly, rather than segmenting text into argumentative units first.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional argument mining techniques are used to identify argument components in student essays, then the process follows conventional segmentation approaches, but the accuracy is insufficient due to differences in student writing conventions such as run-on sentences and multiple claims per sentence
Solution Approach 1:
The patent changes the fundamental parameter of the argument mining approach by switching from sentence-level segmentation to token-level classification. This parameter change allows the model to handle run-on sentences and multiple claims per sentence effectively, as it can identify argument components at the word level rather than being constrained by sentence boundaries. The BERT-based model processes tokens individually, enabling accurate identification of claim and premise tokens even in complex student writing structures.
2Ease of manufacture
If text is segmented into argumentative units first before identifying components, then the process follows a pipeline approach, but the accuracy in identifying claim and premise tokens is reduced
Solution Approach 1:
The patent inverts the traditional pipeline approach by performing token-level classification directly without prior segmentation into argumentative units. Instead of following the conventional sequence of segmentation then component identification, the model classifies each token as claim, premise, or other directly from the input text. This inversion eliminates the intermediate segmentation step that was causing accuracy losses, particularly in handling student essays with non-standard structures.
Solution Approach 2:
The patent merges the segmentation and component identification tasks into a single token-level classification process. By combining these previously separate steps into one unified model operation, the system achieves both segmentation and component identification simultaneously, improving efficiency and accuracy. The BERT model with classification heads performs both functions in one pass through the data.
3Measurement precision
If adaptive pretraining on relevant unlabeled data is performed before fine-tuning, then the model achieves better performance on student essays, but the training process becomes more complex
Solution Approach 1:
The patent applies preliminary action by performing adaptive pretraining on unlabeled student essay data before fine-tuning on labeled datasets. This pretraining step allows the model to learn domain-specific language patterns, writing conventions, and argumentation styles specific to student essays. By preparing the model in advance with domain-relevant unlabeled data, the subsequent fine-tuning on smaller labeled datasets becomes more effective, achieving better overall performance despite the increased training complexity.
Data Source
AI summary
Data is received that includes a passage of text generated in response to a prompt which comprises a plurality of sentences. Thereafter, the passage of text is tokenized into a plurality of tokens each corresponding to a different word in the passage of text. A first classification head of an adaptive fine-tuned transforms classifies each of the tokens into one of a plurality of classes. A second classification head of the adaptive fine-tuned transformer model classifies each of the sentences as either including or not including an argument. Data can then be provided which characterizes the first and second classifications. Related apparatus, systems, techniques and articles are also described.