Named Entity Recognition Model Using Pattern Embeddings for Low-Resource Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing named entity recognition (NER) systems face challenges in accurately identifying seldom encountered entities due to the lack of labeled training data, particularly for sensitive personal information, leading to overfitting and poor generalization to unseen data.
Innovation Solution
The implementation of an improved NER computer model that utilizes entity pattern embeddings and semantic category embeddings in addition to word and character embeddings, allowing for the automatic learning of intrinsic patterns and context information without relying on extensive annotated data or external knowledge resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning computer models are trained using large amounts of manually labeled training data, then the ability to recognize standard lexical entities is improved, but the ability to recognize seldom encountered entities deteriorates due to lack of training data and overfitting
Solution Approach 1:
The patent segments the entity recognition task into multiple components: pattern matching layer (for structural patterns), semantic analysis layer (for meaning), and machine learning layer (for trained entities). This segmentation allows each layer to specialize in different aspects, enabling the system to handle both standard and seldom encountered entities effectively without requiring extensive training data for all entity types
Solution Approach 2:
The patent introduces pattern embeddings and semantic representations as intermediary layers between the input text and the entity recognition output. These intermediaries capture general linguistic patterns and meanings that transfer across different entity types, allowing the model to recognize seldom encountered entities by leveraging patterns from standard entities without overfitting to specific training examples
2Measurement precision
If machine learning models are trained extensively on specific training data, then identification accuracy for training entities is improved, but generalization to unseen textual content deteriorates
Solution Approach 1:
The patent performs preliminary pattern extraction and semantic analysis before applying machine learning classification. By pre-processing the text to identify structural patterns and semantic meanings, the system prepares features that are more generalizable and less dependent on specific training examples, thereby improving performance on unseen data while maintaining accuracy on training entities
Solution Approach 2:
The patent transforms the input data by converting text into multiple representations including pattern embeddings, semantic vectors, and token features. This parameter transformation allows the model to learn from diverse linguistic patterns rather than memorizing specific entities, improving generalization while preserving the ability to accurately identify trained entities through the combined feature representations
Data Source
AI summary
Natural language processing (NLP) methodologies and mechanisms are provided that include a named entity recognition (NER) computer model augmented to operate on an entity pattern embedding input feature in addition to other embedding input features. The mechanisms tokenize natural language content (NLC) to generate tokens and process a selected token in accordance with a predetermined entity pattern embedding technique to generate an entity pattern embedding input feature for the selected token. The entity pattern embedding input feature specifies a pattern of characters present in the selected token. The mechanisms process the NLC to generate the other embedding input features in accordance with other embedding techniques, and process, by the NER computer model, the other embedding input features and the entity pattern embedding input feature for the selected token to generate a predicted tag for the selected token. The predicted tag specifies a named entity type classification for the selected token.