Document-Specific Gazetteers for NER with Limited Annotation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current named entity recognition (NER) models require large amounts of manually annotated training data and struggle with performance when data varies in language and domain, necessitating a method to improve NER performance without relying on extensive additional training data.
Innovation Solution
The approach involves training a named entity recognition model using document-level entity tags, which are used to generate a document-specific gazetteer and extract features for tokens, allowing for prediction of entity labels without the need for extensive gazetteers, and incorporating these features into a Conditional Random Field (CRF) model for improved recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large collections of manually annotated training data are used to train NER models, then NER performance is improved, but the time and cost for manual annotation increases significantly
Solution Approach 1:
The system performs preliminary action by using document-level entity tags to pre-generate a document-specific gazetteer before the actual NER process. This gazetteer contains entity names and types extracted from the document itself, preparing the model with domain-specific knowledge in advance, thereby reducing the need for extensive manual annotation while maintaining high NER performance.
2Reliability
If standard NER models are used for English newswire services, then performance is good on standard datasets, but performance declines when data varies in language and domain
Solution Approach 1:
The system applies local quality by creating a document-specific gazetteer tailored to each document's domain and language characteristics. Instead of using a generic gazetteer for all documents, the system extracts and customizes entity names and types specific to each document, thereby maintaining high reliability on standard datasets while improving adaptability to varied languages and domains.
Solution Approach 2:
The system implements dynamics by making the gazetteer adaptive to each document rather than static. The document-specific gazetteer is dynamically generated based on the input document's content, allowing the NER model to adapt to different languages and domains automatically, thus improving versatility while maintaining reliability.
3Measurement precision
If large gazetteers are used for NER, then recall is improved by including known names, but the complexity and resource requirements of the system increases
Solution Approach 1:
The system extracts only the necessary entity names and types from the document to create a document-specific gazetteer, rather than using a large comprehensive gazetteer. This extraction approach maintains high recall by including relevant known names while reducing system complexity and resource requirements by eliminating unnecessary gazetteer entries.
Data Source
AI summary
A method for entity recognition employs document-level entity tags which correspond to mentions appearing in the document, without specifying their locations. A named entity recognition model is trained on features extracted from text samples tagged with document-level entity tags. A text document to be labeled is received, the text document being tagged with at least one document-level entity tag. A document-specific gazetteer is generated, based on the at least one document-level entity tag. The gazetteer includes a set of entries, one entry for each of a set of entity names. For a text sequence of the document, features for tokens of the text sequence are extracted. The features include document-specific features for tokens matching at least a part of the entity name of one of the gazetteer entries. Entity labels are predicted for the tokens in the text sequence with the named entity recognition model, based on the extracted features.