Topic Segmentation Model for Entity Extraction Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional entity extraction models, such as Named-Entity Recognition (NER) and Generative Pre-trained Transformer 3 (GPT3), struggle with accurately identifying and separating similar data entities in large amounts of unstructured text, particularly when dealing with multiple instances of the same class or categories, leading to reduced accuracy and increased processing power requirements.
Innovation Solution
The integration of a topic segmentation model that divides unstructured text into smaller, topically related segments, allowing entity extraction models to process each segment individually, thereby improving accuracy and efficiency by reducing the overall processing burden and enabling parallel processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional entity extraction models process large amounts of unstructured text directly, then they can extract entities from the text, but accuracy decreases and processing power requirements increase when dealing with multiple instances of the same class
Solution Approach 1:
The patent divides unstructured text into multiple structured segments based on different topics or categories before entity extraction. Each segment is then processed separately by entity extraction models, which improves accuracy by reducing confusion between similar entities of the same class while enabling parallel processing to reduce overall computational burden.
2Measurement precision
If entity extraction models process all unstructured text together, then they can extract entities from diverse topics, but they struggle to accurately separate similar data entities belonging to different topics
Solution Approach 1:
The patent segments unstructured text into topic-specific structured segments using topic modeling or classification before entity extraction. This allows the system to maintain high entity separation accuracy within each topic while collectively covering diverse topics across all segments, resolving the contradiction between precision and versatility.
Solution Approach 2:
The patent applies topic-specific processing characteristics to each segmented group of text. Different segmentation strategies, extraction models, or parameters can be applied to different topics based on their specific characteristics, optimizing entity extraction accuracy for each topic while maintaining overall system versatility.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
This disclosure relates to extracting entities from unstructured text. The unstructured text is segmented into structured segments with one or more instances, that belong to different topics, with a topic segmentation model. Each instances of the structured segment is operated on by an entity extraction model to extract entities, and the extracted entities associated with each topic is produced in a computer-readable format. The relations between extracted entities associated with each topic may be identified.