Vector Hash Tables for Accurate Named-Entity Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing named-entity resolution (NER) technologies face challenges in accurately and efficiently classifying entities due to variations in nomenclature, typographical errors, and lack of standardized taxonomies, particularly in unstructured text data from multiple sources, which requires large volumes of manually labeled training data and is time-consuming.
Innovation Solution
A data structure comprising a vector hash table with buckets storing alias strings and corresponding vector embeddings, and optionally a tag hash table with standard entity names, using artificial neural networks for deterministic vector embedding, to automatically tag and label entities, generate zero-shot NER models, and configure networks for improved entity recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual labeling of training data is used to achieve accurate entity classification, then classification accuracy is improved, but time consumption and labor cost increase significantly
Solution Approach 1:
The system performs preliminary actions by pre-processing unstructured text data to extract potential entity mentions and their contextual information before the main classification task. This includes tokenization, part-of-speech tagging, and initial entity candidate identification, which prepares the data in advance for more efficient automated classification while maintaining accuracy.
Solution Approach 2:
The patent introduces an intermediary component - a trained machine learning model that acts as a mediator between raw unstructured text and final entity classifications. This intermediary model has been pre-trained on manually labeled data to learn entity patterns, and then automatically applies this knowledge to new data, eliminating the need for manual labeling in production while preserving classification accuracy.
2Measurement precision
If standardized nomenclature and taxonomy are implemented for entity classification, then entity recognition consistency is improved, but system complexity increases due to data structure requirements
Solution Approach 1:
The patent segments the complex entity classification task into multiple manageable components: (1) entity mention extraction, (2) entity type classification, (3) entity normalization to standardized nomenclature, and (4) relationship extraction. This segmentation allows each component to handle a specific aspect, reducing overall system complexity while improving consistency through standardized processing pipelines.
Solution Approach 2:
The system transforms unstructured text data into structured representations by changing parameters such as converting free-text entity mentions into standardized entity types with controlled vocabularies. This parameter transformation enables consistent entity recognition while the underlying data structures remain flexible and adaptable to different domains.
3Measurement precision
If large volumes of training data are used to train NER models, then model accuracy is improved, but data processing time and computational resources increase
Solution Approach 1:
The system performs preliminary data processing and filtering to identify and extract only the most relevant training examples before model training. This includes pre-processing text to extract potential entity mentions, filtering out low-quality or duplicate examples, and organizing data into efficient formats. This preliminary action reduces the volume of data that requires intensive processing while maintaining model accuracy.
Solution Approach 2:
The patent extracts and utilizes only the essential features and patterns from large volumes of training data that are most relevant for entity recognition. Instead of processing all raw data equally, the system identifies and extracts key entity mentions, contextual features, and labeling patterns, discarding redundant information. This extraction approach maintains model accuracy while significantly reducing computational requirements.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure relates to named-entity resolution and recognition, such as may be used to pre-process data for input to automated cybersecurity and telecommunications network management systems. Methods described herein relate to a data structure comprising a hash table consisting of one or more buckets, referred to herein as a 'vector hash table', wherein: each bucket of the vector hash table stores a string comprising one or more aliases for a respective entity, referred to herein as an 'alias string'; and for each bucket, a corresponding key is a vector embedding of that alias string.