Document Tagging via Mapping Table for Domain Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic tagging systems for documents are limited in their ability to accurately annotate text from domains different from their training data, as they rely on scarce high-quality human-generated annotations, which are labor-intensive and often specific to narrow domains like news articles.
Innovation Solution
A system that uses a metadata-based tagger to generate annotations for documents from a second tag set, establishing a mapping table between tags from different sets based on correlation, and then uses this mapping to further train a statistical tagger, allowing it to improve its accuracy in annotating text from diverse domains.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a statistical tagger is trained on hand-generated training data from a specific domain (e.g., news articles), then tagging accuracy is improved for that domain, but the tagger cannot effectively annotate text from different domains
Solution Approach 1:
The patent introduces a mapping table as an intermediary between the training domain tag set and the target domain tag set. This mapping table enables the statistical tagger to translate tags from the training domain to the target domain, allowing accurate tagging across different domains without retraining the tagger on target domain data.
Solution Approach 2:
The patent changes the parameter space by establishing a mapping between different tag sets from different domains. By transforming the tag parameters through the mapping table, the system adapts the tagger's output to be relevant to the target domain while maintaining the benefits of training on high-quality hand-generated data.
2Reliability
If hand-generated training data is used to train the tagger, then training data quality is improved, but the process becomes labor-intensive and time-consuming
Solution Approach 1:
The patent performs preliminary action by creating a mapping table between tag sets in advance. This mapping table can be generated once and reused for tagging multiple documents in the target domain, eliminating the need for continuous hand-generated training data collection and processing for each new document.
Solution Approach 2:
The patent uses copying by leveraging the structure and relationships from the training domain tag set and replicating them in the target domain through the mapping table. This allows the system to copy the quality and reliability characteristics of hand-generated training data to the target domain without manually creating training data for each target domain.
3Measurement precision
If a metadata-based tagger is used to generate tags from user-defined metadata, then domain-specific relevance is improved, but the tags may not align with the statistical tagger's tag set
Solution Approach 1:
The patent merges the strengths of both approaches by combining the statistical tagger's ability to identify patterns in text with the metadata-based tagger's domain-specific knowledge. The mapping table serves as a bridge that integrates these two different tagging approaches into a unified system.
Solution Approach 2:
The mapping table acts as an intermediary that reconciles the different tag sets from the statistical tagger and metadata-based tagger. It translates between the two tag sets, enabling them to work together harmoniously and produce tags that are both statistically accurate and domain-relevant.
Data Source
AI summary
An automated technique for tagging documents includes using a semantic tagger to generate an annotation that associates a standard tag with a first text fragment of the user-defined document, wherein the tagger is trained on a standard document annotated with a standard tag, associating the first user-defined tag with a second text fragment of the user-defined document in response to the second text fragment matching a value associated with the first user-defined tag, and establishing a mapping between the standard tag and the first user-defined tag in response to existence of a requisite correlation between the standard tag and the user-defined tag. The technique may further include selecting from the user-defined document a tagged text fragment that is associated with a second user-defined tag, and providing the tagged text fragment and a standard tag associated by the mapping with the second user-defined tag to the tagger as additional training input.