Entity Recognition via Dynamic Window Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for processing raw and unstructured medical text data require significant human intervention for entity recognition and annotation, limiting efficiency and scalability.
Innovation Solution
A computing system that employs word embeddings and character embeddings to automatically identify entities in medical text data by comparing embeddings with a knowledge base, using a dynamically expanding window and confidence scoring to determine entity representations without human intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual entity recognition and annotation methods are used, then accuracy can be maintained through human judgment, but productivity is significantly reduced due to time-consuming manual processing
Solution Approach 1:
The patent replaces the mechanical human annotation process with an automated computational system that uses word embeddings, character embeddings, and confidence scoring algorithms to identify and annotate entities in medical text data, achieving both high speed and high accuracy simultaneously
Solution Approach 2:
The system transforms unstructured medical text data into structured format by changing parameters such as converting text to embeddings, calculating confidence scores, and filtering results based on threshold values, enabling automated high-speed processing while maintaining accuracy
2Adaptability or versatility
If extensive human intervention is provided for entity recognition, then annotation quality can be ensured, but the system becomes less scalable and more complex
Solution Approach 1:
The system enables self-service automated entity recognition by training models on medical text data to automatically identify entities, calculate confidence scores, and filter results without requiring human intervention during the processing phase, thereby improving scalability while managing complexity through automation
Solution Approach 2:
The patent performs preliminary actions by pre-training word embeddings and character embeddings on medical corpus data before actual entity recognition tasks, allowing the system to scale efficiently by reusing pre-computed embeddings across multiple processing tasks without reprocessing the same data
3Loss of time
If manual annotation processes are used, then detailed human expertise can be applied, but loss of time occurs due to the sequential nature of manual review
Solution Approach 1:
The system implements continuous automated processing that operates without interruption, processing medical text data through embedding generation, confidence scoring, and filtering in an uninterrupted pipeline, thereby minimizing time loss while ensuring complete data processing through systematic coverage of all text elements
Data Source
AI summary
Some embodiments provide a non-transitory machine-readable medium that stores a program. The program receives a set of words. The program further retrieves an entry from a knowledge base comprising a plurality of entries. Each entry includes a text description of a concept. The program also determines an embedding for the entry based on the text description of the concept. The program further iteratively determines an embedding for a word in the set of words, increasing a size of a window of words in the set of words, and calculating a confidence score for the entry with respect to the word based on the embedding for the entry and the embedding for the word until a successive calculated confidence score decreases below a previous calculated confidence score. The program also determines that a window of words in the set of words having a previous size represents an entity.