Argument Scoring via Discourse Unit Segmentation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current automated scoring systems for argumentative writing lack a universal, generic model that can effectively identify good arguments across various topics, leading to performance gaps between topic-specific and generic models, and struggle to handle unseen essay prompts, raising fairness concerns and inefficiencies in large-scale scoring contexts.

Innovation Solution

A robust argument analysis system that identifies discourse segments, classifies sentences into discourse units, evaluates argumentative content, and assigns scores using a conditional mutual information criterion, trained on both same-topic and cross-topic sets of essays, incorporating lexical and discourse structure features to generalize across different prompts.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If a universal generic model is used for scoring argumentative writing across all topics, then the system can handle unseen prompts and maintain fairness, but the performance gap between topic-specific and generic models reduces scoring accuracy

Engineering Contradiction:

Improveability to handle unseen promptsVSAvoidscoring accuracy

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent segments the essay into discourse units (claims, premises, counterclaims, rebuttals) and evaluates argumentative features at the sentence level within each unit. This segmentation allows the generic model to capture topic-independent argumentation patterns while maintaining accuracy by focusing on structural and rhetorical features rather than topic-specific content.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent shifts from evaluating topic-specific content to analyzing the dimensional structure of arguments, including discourse organization, rhetorical strategies, and logical relationships between claims and premises. This dimensional approach enables the system to assess argument quality across diverse topics by focusing on the architecture of reasoning rather than subject matter.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Measurement precision

If topic-specific models are used for each essay prompt, then scoring accuracy is improved, but the system cannot handle new prompts without rebuilding and fairness concerns arise due to performance disparity across prompts

Engineering Contradiction:

Improvescoring accuracyVSAvoidability to handle unseen prompts

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent develops a universal scoring model that performs multiple functions: it evaluates argumentative features across different discourse units, handles various essay prompts, and maintains consistent scoring standards. The model uses topic-independent features such as claim-premise relationships, rhetorical strategies, and discourse organization that apply universally across all argumentative writing tasks regardless of topic.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent changes the parameters evaluated by the scoring system from topic-specific content features to topic-independent argumentative features. By focusing on parameters such as discourse structure, logical relationships, and rhetorical effectiveness rather than subject matter content, the model achieves consistent performance across different prompts without requiring topic-specific customization.

Inventive Principle:

Principle #35Parameter changes

3Productivity

If automated scoring focuses on argument structure (claims and premises), then the system can efficiently evaluate essays, but it fails to identify whether the arguments are actually good in the context of the topic

Engineering Contradiction:

Improvescoring efficiencyVSAvoidargument quality assessment

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent introduces discourse units as intermediary structures that connect the automated scoring system to the quality of arguments. By organizing sentences into claims, premises, counterclaims, and rebuttals, and evaluating the logical relationships and rhetorical strategies within these units, the system efficiently assesses argument quality without requiring deep topic-specific knowledge, thus maintaining both productivity and measurement precision.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS11151894B1Systems and methods for scoring argument critique written essays

Publication Date: 2021.10.19 EDUCATIONAL TESTING SERVICE

AI summary

Methods and systems for scoring an argument critique written essay, including identifying a discourse segment of the argument critique written essay, determining a position of each sentence in the discourse segment, classifying sentences into discourse units, evaluating an argumentative content of each sentence, and assigning an argumentative score to the essay based on the argumentative content of each sentence in the discourse segment of that essay. Methods and systems for training the scoring method are also disclosed. Corresponding apparatuses, systems, and methods are also disclosed.