Trie-Based Annotation Extraction from Grammar-Poor Text
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional natural language processing techniques perform poorly in extracting biomedical annotations from grammar-poor and unstructured text, such as clinical notes, due to issues like lack of punctuation, grammatical errors, and abbreviations, leading to faulty annotations and poor linkage to medical ontologies.
Innovation Solution
The use of a trie-based approach that allows for a limited number of 'skips' during text processing to prevent falling off the trie, combined with graph theory to determine a clique of non-overlapping phrases providing maximal coverage, which are then linked to relevant ontology codes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional NLP techniques are used to extract annotations from grammar-poor text, then the processing speed is maintained, but the annotation accuracy and reliability deteriorate due to lack of punctuation, grammatical errors, and abbreviations
Solution Approach 1:
The patent segments the text processing task into distinct phases: tokenization of grammar-poor text, trie-based pattern matching for phrase identification, graph construction from identified phrases, and clique detection for optimal phrase selection. This segmentation allows each component to handle specific aspects of the complexity, improving overall reliability without requiring a monolithic complex system
Solution Approach 2:
The patent introduces intermediate data structures as mediators: tokens as intermediaries between raw text and phrases, phrases as intermediaries between tokens and graph nodes, and the graph itself as an intermediary between multiple phrases and the final clique selection. These intermediaries transform the unstructured grammar-poor text into progressively more structured representations, improving annotation accuracy at each stage
2Loss of information
If a trie-based approach with skips is used to process grammar-poor text, then the coverage of identified phrases is maximized, but the computational time and processing resources increase
Solution Approach 1:
The patent applies partial action by allowing a limited number of skips (e.g., 1-3 skips) during trie traversal rather than requiring exact matches. This partial deviation from strict matching enables the system to capture phrases in grammar-poor text without exhaustively checking every possible variation, thereby maximizing phrase coverage while controlling processing time through a bounded skip parameter
3Measurement precision
If graph theory is applied to determine cliques of non-overlapping phrases, then the precision of annotation selection is improved, but the computational complexity increases
Solution Approach 1:
The patent performs preliminary actions by constructing the graph and identifying all potential cliques before selecting the optimal set. The graph is built in advance with nodes representing phrases and edges representing non-overlapping relationships, allowing efficient clique detection algorithms to operate on a pre-processed structure rather than evaluating all phrase combinations from scratch, thus improving selection precision while managing computational complexity
Data Source
AI summary
Techniques are described for processing a text document or passage to derive a suitable set of phrases from the document or passage. These phrases may in turn be related to codes or other labels useful to a reviewer, such as insurance, diagnostic, or clinical codes, genes related to identified phenotypes, and so forth. In certain embodiments, one or more tries generated based on respective ontologies may be used to process and parse the input text passage or document to derive candidate phrases. To improve performance, a limited number of skips may be allowed. The candidate phrases and corresponding intervals may, in one implementation, be used to populate a graph having nodes and edges and from which a set of phrases may be determined that provides maximal coverage of the text passage or document and having limited (or no) overlaps.