Character-Level NER via 1D Xception Layer for Poor OCR Text
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional named-entity recognition (NER) techniques require full text annotation for start and end positions of entities, which is challenging in cases of poor OCR quality and data format variations, leading to annotation failures and multiple predictions for the same entity.
Innovation Solution
A deep learning neural network-based system that uses a one-dimensional Xception layer for sequence classification, predicting named entity values without the need for prior annotation, capable of handling text in column format and varying data formats.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full text annotation is performed to identify start and end positions of entities, then named entity recognition can be achieved, but annotation failures occur in cases of poor OCR quality and data format variations
Solution Approach 1:
The patent segments the text processing task into character-level predictions rather than requiring full entity annotation. Each character is classified independently as belonging to an entity or not, which eliminates the need for manual annotation of start and end positions while maintaining accuracy even in challenging OCR conditions
Solution Approach 2:
The system performs self-service by automatically identifying named entities through deep learning without requiring external annotation. The neural network learns to identify entities directly from the text data, eliminating the manual annotation process that is prone to failures in poor quality scenarios
2Productivity
If classical NLP NER techniques are used to extract field information, then entity information can be obtained, but multiple predictions for the same entity are generated making it difficult to utilize
Solution Approach 1:
By segmenting the prediction task into character-level classifications, the system generates a sequence of predictions where each character is independently classified. This segmented approach naturally resolves the multiple predictions problem by providing a clear character-by-character breakdown that can be easily aggregated into single entity predictions
Solution Approach 2:
The patent changes the dimension of prediction from entity-level to character-level. Instead of predicting entire entities at once (which causes multiple predictions), the system predicts each character's entity status, providing a new dimensional perspective that eliminates prediction conflicts and enables consistent entity extraction
3Measurement precision
If text annotation is performed to find start and end positions, then named entities can be identified, but this process is time-consuming and requires manual intervention
Solution Approach 1:
The system eliminates manual annotation by performing self-service through automated deep learning. The neural network directly processes the text and generates character-level entity predictions without human intervention, completely removing the time-consuming annotation step while maintaining accurate entity position identification
Solution Approach 2:
The system performs preliminary action by pre-training the deep learning model on labeled data, so that during actual use, no manual annotation is needed. The model has already learned the patterns of named entities and can automatically identify them in new text, saving significant time compared to on-demand annotation
Data Source
AI summary
An illustrative embodiment provides a computer implemented method for extracting named entity information from full text includes obtaining text data from a document. The method also includes tokenizing the text data as numbers to produce a tokenized layer. The method also includes creating an embedded layer from the tokenized layer by encoding the text data by indices. The method also includes creating a one-dimensional Xception layer from the embedded layer according to a one-dimensional convolution. The method also includes determining a sequence of classification for the text data according to the one-dimensional Xception layer. The method also includes determining a named entity value from the text data according to the sequence of classification, wherein each sequence of classification predicts one character of the named entity value.