Semantic Token Tagging via Synthetic Canonical Sequences
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional natural language processing (NLP) applications face challenges in domain-specific semantic token tagging, as they often rely on purely syntactic labels that do not capture domain-dependent semantic information, leading to performance drops and requiring labor-intensive annotation efforts for training data.
Innovation Solution
The system employs semantically rich labels by automatically generating training data using knowledge graphs and language models to create canonical sentences and their variations, filtering high-quality sequence variations for training neural network models, thereby overcoming the need for extensive human annotation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If domain adaption approaches are used to compensate for performance drop in POS tagging systems, then performance across different domains is improved, but the complexity of the system increases
Solution Approach 1:
The patent applies preliminary action by pre-training POS tagging systems on domain-specific data before deployment. The system performs domain adaption in advance by training on domain-specific corpora, creating domain-specific models that can be directly applied without requiring complex runtime adjustments. This resolves the contradiction by improving reliability through advance domain-specific training while avoiding the need for complex adaptive mechanisms during operation.
2Measurement precision
If semantically rich labels are used to capture domain-dependent semantic information, then NLP application accuracy is improved, but the need for labor-intensive annotation increases
Solution Approach 1:
The patent applies copying by using syntactic labels as templates or patterns that are then adapted to create semantically rich labels. Instead of manually annotating semantic information from scratch, the system copies the structured format of syntactic labels and fills them with domain-specific semantic content automatically or with minimal human input. This resolves the contradiction by maintaining high measurement precision through semantically rich labels while dramatically reducing annotation time by reusing the copying approach.
Solution Approach 2:
The system performs preliminary syntactic labeling automatically, then uses these pre-generated labels as a foundation for creating semantically rich annotations. By having the syntactic structure prepared in advance, the system reduces the manual effort needed for semantic annotation, as annotators only need to enhance the pre-existing syntactic labels with domain-specific semantic information rather than creating annotations from scratch.
3Reliability
If a large quantity of annotated resources are used to train machine learning models, then model performance is improved, but the resource intensity of data preparation increases
Solution Approach 1:
The patent applies copying by generating synthetic training data that replicates the structure and characteristics of annotated resources. The system creates artificial domain-specific corpora by copying patterns from existing data and transforming them into training examples with semantically rich labels. This resolves the contradiction by providing sufficient training data to maintain model performance while avoiding the resource-intensive process of manually annotating large quantities of real-world data.
Solution Approach 2:
The system performs preliminary data generation and synthetic annotation creation before model training. By preparing training data in advance through automated synthesis and transformation processes, the system ensures sufficient training resources are available while dramatically improving data preparation efficiency. The preliminary action of generating synthetic data eliminates the need for time-consuming manual annotation of large datasets.
Data Source
AI summary
A computing system can receive a request to apply semantic token tagging on a specified domain, and can retrieve a set of data associated with the specified domain from a data storage facility. Canonical sequences can be formed from strings included in the data set. Each canonical sequence can be permutated to form sequence variations and each sequence variation can be verified against a generalized domain. Semantic token tagging can be applied to the specified domain using a subset of the sequence variations that are successfully verified as training data.