Data Augmentation System for NLP Named Entity Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies require significant time and labor to prepare search queries including named entities, especially for training data in natural language processing tasks.
Innovation Solution
A data augmentation system that uses a processor to acquire past search queries, train a model to generate virtual search queries with different named entities, and execute data augmentation based on these generated queries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual preparation of training data including named entities is performed, then the quality and accuracy of named entity recognition can be maintained, but the time and labor consumption increases extremely
Solution Approach 1:
The system performs preliminary actions by automatically generating synthetic training data with named entities before the actual named entity recognition task. A language model generates search queries containing named entities, which are then used as training data, eliminating the need for manual data preparation while ensuring sufficient training samples are available.
Solution Approach 2:
The system enables self-service by using an automated pipeline where a language model generates training data with named entities without human intervention. The process includes automatic query generation, named entity insertion, and dataset creation, allowing the system to prepare its own training data independently.
2Productivity
If a large amount of training data including named entities is manually prepared, then the performance of named entity recognition model can be improved, but the complexity and cost of data preparation increases
Solution Approach 1:
The system replaces the mechanical process of manual data preparation with an automated computational process. A language model automatically generates search queries and inserts named entities, substituting human manual work with algorithmic processes that reduce complexity and enable scalable data generation.
Solution Approach 2:
The system changes parameters by adjusting the language model's generation parameters to produce diverse training data. By modifying temperature, top-k sampling, and other generation parameters, the system can control the diversity and quality of generated named entity examples, optimizing training data characteristics without manual intervention.
Data Source
AI summary
Provided is a data augmentation system including at least one processor configured to: acquire a first search query including a first named entity, which was actually input in a search executed in a past; execute training of a first model that outputs a second search query including a virtual second named entity, which is different from the first named entity, based on the first search query; and execute data augmentation based on the second search query output by the trained first model.