Synthetic Text Generation for Sentence Classification Data Augmentation

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing deep learning models for sentence classification require large amounts of manually labeled training data, which is costly and time-consuming to create, especially for domain-specific tasks, as they tend to overfit with small datasets and lack generalizability.

Innovation Solution

A method and system for generating labeled textual training data using a human-in-the-loop learning-to-augment framework, which includes a sentence generator and classifier, utilizing reinforcement learning and weak supervision to synthesize new sentences and labels, allowing for iterative improvement with minimal expert supervision.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manually labeled training data is used for sentence classification, then model accuracy is improved, but data creation cost and time increase

Engineering Contradiction:
Improvemodel accuracyVSAvoiddata creation time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent uses a text generation model to automatically copy and transform existing labeled text data into new synthetic training data samples. This copying approach with transformations maintains the quality and labeling while eliminating manual annotation time, directly resolving the contradiction between accuracy and time cost.

Inventive Principle:
Principle #26Copying

Solution Approach 2:

The system employs a self-service mechanism where the text generation model automatically creates and labels training data without human intervention. The model serves itself by generating synthetic data that it can then use for training, eliminating the need for external human annotators and significantly reducing data creation time while maintaining accuracy.

Inventive Principle:
Principle #25Self-service

2Measurement precision

If manually labeled training data is used for sentence classification, then model accuracy is improved, but production cost increases

Engineering Contradiction:
Improvemodel accuracyVSAvoiddata production cost
Core Design Contradiction:
Measurement precisionVSEase of manufacture

Solution Approach 1:

By copying and transforming existing labeled data through automated text generation, the patent eliminates the need for expensive human annotation services. This approach maintains data quality and model accuracy while dramatically reducing production costs associated with manual labeling.

Inventive Principle:
Principle #26Copying

Solution Approach 2:

The self-service text generation model automatically produces training data without requiring external human expertise or payment for annotation services, thereby maintaining high model accuracy while reducing production costs to minimal computational expenses.

Inventive Principle:
Principle #25Self-service

3Productivity

If small amount of training data is used, then data creation effort is reduced, but model generalization deteriorates

Engineering Contradiction:
Improvedata creation efficiencyVSAvoidmodel generalization
Core Design Contradiction:
ProductivityVSReliability

Solution Approach 1:

The patent uses copying with transformation to synthesize multiple diverse training samples from limited seed data. This approach efficiently creates large datasets from small inputs, maintaining high data creation efficiency while improving model generalization through increased data volume and diversity.

Inventive Principle:
Principle #26Copying

Solution Approach 2:

The text generation model introduces new dimensions of variation in the training data by transforming existing samples into diverse synthetic versions. This dimensional expansion of the training dataset maintains efficiency while significantly improving model generalization capabilities.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

4Measurement precision

If domain-specific training data is created manually, then task performance is improved, but expert annotation cost increases

Engineering Contradiction:
Improvetask performanceVSAvoidannotation cost
Core Design Contradiction:
Measurement precisionVSEase of manufacture

Solution Approach 1:

The system copies and transforms domain-specific labeled examples to generate additional training data, maintaining task-specific performance while eliminating the need for expensive domain expert annotation services. The automated generation preserves domain knowledge without requiring ongoing expert involvement.

Inventive Principle:
Principle #26Copying

Data Source

PatentUS11875120B2Augmenting textual data for sentence classification using weakly-supervised multi-reward reinforcement learning
Publication Date: 2024.01.16 ROBERT BOSCH GMBH
  • US11875120B2 patent drawing
  • US11875120B2 patent drawing
  • US11875120B2 patent drawing

AI summary

A system and method are disclosed that enable rapid and cost-effective human-in-the-loop synthesis of domain-specific textual training data for a deep learning model. The data augmentation process incorporates a sentence generator, a sentence classifier, and weak-supervision by a domain expert that is ‘in the loop.’ Generally, both the sentence generator and the sentence classifier are implemented as machine learning models. The sentence generator generates new sentences based on manually labeled sentences and the sentence classifier generates labels for the newly generated sentences. The new sentences are corrected or verified by a domain expert and then used to retrain one or both of the sentence generator and the sentence classifier.