Text Classifier Domain Term Replacement

Solution Overview

Problem

Existing text classification systems face challenges in mitigating bias, particularly due to domain term referencing biases, which can lead to inaccurate classification of input texts, especially when trained on ground truth data that includes frequent domain terms.

Innovation Solution

The proposed solution involves preprocessing input texts by replacing identified domain terms using a domain terms selector and replacer module, aligning the input text with pre-processed ground truth data to train the machine learning model, thereby reducing domain term referencing bias and improving classification accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If domain terms are frequently present in ground truth training data, then the text classifier can learn domain-specific patterns, but domain term referencing bias is introduced leading to inaccurate classification

Engineering Contradiction:

Improveclassification accuracyVSAvoidbias in classification

Core Design Contradiction:

Measurement precisionVSReliability

Solution Approach 1:

The patent applies preliminary action by preprocessing the input text before classification to replace domain terms with generic terms. This pre-processing step ensures that the text classifier receives bias-mitigated input, allowing the model to focus on content rather than being influenced by frequent domain term patterns in the training data.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces an intermediary component (the domain term replacement mechanism) that sits between the raw input text and the text classifier. This intermediary replaces domain-specific terms with generic counterparts, mediating the interaction between input text and classifier to prevent bias transmission while preserving classification accuracy.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If the text classifier is trained on original ground truth data with domain terms, then it learns from authentic data patterns, but it becomes biased toward texts containing those domain terms

Engineering Contradiction:

Improvetraining data authenticityVSAvoidclassification accuracy for bias-free text

Core Design Contradiction:

ReliabilityVSMeasurement precision

Solution Approach 1:

The patent applies preliminary action by preprocessing the input text before classification to replace domain terms with generic terms. This pre-processing step ensures that the text classifier receives bias-mitigated input, allowing the model to focus on content rather than being influenced by frequent domain term patterns in the training data.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent changes the parameter of domain term frequency by replacing domain-specific terms with generic terms. This parameter transformation reduces the weighting influence of domain terms in the input text, allowing the classifier to make decisions based on content rather than term frequency patterns.

Inventive Principle:

Principle #35Parameter changes

3Measurement precision

If domain terms are replaced in input text before classification, then bias is reduced and classification accuracy improves, but additional preprocessing steps are required

Engineering Contradiction:

Improveclassification accuracyVSAvoidtext processing pipeline complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent applies segmentation by dividing the text processing task into distinct components: domain term identification, domain term replacement, and classification. This modular approach allows each component to be optimized independently while maintaining overall system accuracy and reducing bias.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies preliminary action by preprocessing the input text before classification to replace domain terms with generic terms. This pre-processing step ensures that the text classifier receives bias-mitigated input, allowing the model to focus on content rather than being influenced by frequent domain term patterns in the training data.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS9940323B2Text classifier operation

Publication Date: 2018.04.10 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

A mechanism is provided in a data processing system for text classification. A domain terms selector component, executing on a processor of the data processing system, receives an input text. A domain terms selector component executing on a processor of the data processing system identifies at least one domain term in the input text. A domain terms replacer component executing on a processor of the data processing system replaces the at least one identified domain term in the input text with a corresponding replacement term to form a revised input text. A text classifier component configured with a trained text classifier machine learning model classifies the revised input text to form a class determination.