Hierarchical Named Entity Recognition with Parallel Sub-Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hierarchical named entity recognition (HNER) systems face challenges with increased complexity, longer training times, and reduced accuracy due to the complexity of models and the flattening of hierarchical structures, which hinders the leverage of correlations between category labels and results in less accurate predictions.
Innovation Solution
A novel HNER system utilizing a parallel architecture with an encoder model and multiple hierarchical level models, each trained independently to predict categories at specific hierarchical levels, leveraging the Bidirectional Encoder Representations from Transformers (BERT) for context learning and employing a cross-entropy loss function for training, allowing for parallel processing and improved accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a single complex model is used for hierarchical named entity recognition, then the system can process multiple hierarchical levels, but the model complexity increases and training time becomes longer
Solution Approach 1:
The patent divides the hierarchical NER task into multiple independent sub-tasks, with each sub-model responsible for predicting categories at a specific hierarchical level. This segmentation allows each model to focus on a single level's category set, reducing individual model complexity while maintaining overall hierarchical prediction capability through parallel execution of multiple sub-models.
2Adaptability or versatility
If a single complex model is used for hierarchical named entity recognition, then the system can process multiple hierarchical levels, but the training time increases
Solution Approach 1:
By segmenting the training process into independent sub-model training tasks, each sub-model can be trained separately and in parallel on its specific hierarchical level data. This parallel training approach significantly reduces total training time compared to training a single complex model sequentially, while still achieving comprehensive hierarchical prediction capability.
3Device complexity
If hierarchical structures are flattened, then the system complexity is reduced, but the correlation between category labels is lost and accuracy decreases
Solution Approach 1:
The patent preserves hierarchical correlations by organizing category sets into multiple hierarchical levels rather than flattening them. Each sub-model operates on its specific level's category set, maintaining the dimensional structure of the hierarchy. This approach retains the correlation information between parent and child categories, enabling more accurate predictions while keeping system complexity manageable through modular sub-models.
Data Source
AI summary
A novel system is described for performing hierarchical named entity recognition (“HNER”) processing that includes identifying categories at different hierarchical levels for a named entity. The HNER system uses a novel architecture comprising an encoder model and a system of trained machine learning (ML) models to perform the HNER processing, where each trained model in the system of ML models corresponds to a particular hierarchical level, and each model is trained to extract one or more named entities and predict a category for each extracted named entity for the corresponding hierarchical level. Novel techniques are also described for training the various models in HNER system including an encoder model and models in the system of models.