Deep Hybrid Neural Network for NER Accuracy and Adaptability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Named entity recognition (NER) in Natural Language Processing is challenging due to the scarcity of supervised training data and the difficulty in adapting to new languages and domains, as existing methods rely on language-specific resources and hand-crafted features that do not scale well.
Innovation Solution
A deep hybrid neural network architecture that uses a deep compositional character-to-word model to learn from limited supervised training data without language-specific resources or hand-crafted features, combining character-level information with pre-trained word embeddings and bidirectional Long Short-Term Memories, and sequential conditional random fields for end-to-end learning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If hand-crafted features and language-specific resources are used for NER, then NER accuracy can be improved in specific domains, but the system becomes difficult to adapt to new languages and domains and does not scale well
Solution Approach 1:
The patent applies universality by using language-agnostic orthographic features (such as character n-grams, suffixes, prefixes) that can be applied across any language without modification. The feature engineering framework is designed to be domain-independent, allowing the same NER system to process multiple languages and domains using the same universal feature set, thereby resolving the contradiction between achieving good NER accuracy and maintaining adaptability to new languages and domains
2Measurement precision
If language-specific resources and hand-crafted features are used, then NER performance can be improved for specific tasks, but the development complexity and resource requirements increase significantly
Solution Approach 1:
The patent extracts and eliminates the need for language-specific resources and domain-specific feature engineering by focusing on universal orthographic features that can be automatically generated from any text. This extraction of the essential language-independent characteristics of text allows the system to achieve good NER performance without the complexity of manually crafting features for each language and domain
Solution Approach 2:
The system employs automatic feature generation from orthographic patterns that can be derived self-service style from the input text itself, without requiring external language-specific resources or manual feature engineering. The orthographic features are automatically computed from the character sequences in the text, reducing development complexity while maintaining NER performance
3Productivity
If supervised training data is limited, then training time and computational resources are reduced, but NER model accuracy deteriorates
Solution Approach 1:
The patent changes the parameter space by using orthographic features at the character level rather than word-level features, which creates a more granular and informative representation that can be effectively learned from limited supervised data. This parameter change in feature granularity allows the model to capture subtle orthographic patterns that are language-independent and can be learned with fewer training examples, thereby maintaining NER model accuracy while improving training efficiency
Data Source
AI summary
In an example, a text sentence comprising a plurality of words is obtained. Each of the plurality of words is passed through a deep compositional character-to-word model to encode character-level information of each of the plurality of words into a character-to-word expression. The character-to-word expressions are combined with pre-trained word embeddings. The combined character-to-word expressions and pre-trained word embeddings are fed into one or more bidirectional long short-term memories to learn contextual information for each of the plurality of words. Then, sequential conditional random fields are applied to the contextual information for each of the plurality of words.