Prompt-Learned Information Extraction and Image Reordering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing optical character recognition (OCR) methods for extracting information from textual images face challenges such as the difficulty in generating information templates, labor-intensive modifications due to text style changes, and diminished accuracy in identifying key information, especially with noise and lack of semantic relationships.
Innovation Solution
A method using a pre-trained natural language processing (NLP) model to generate semantic vectors, determine semantic similarity based on text positions, and re-train the model to create enhanced NLP models for key-value pair generation and logical relationship prediction through prompt learning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If a pre-structured information template is used to extract text from textual images, then the extraction process becomes automated, but the template generation becomes labor intensive and requires manual modifications when text style changes
Solution Approach 1:
The system uses a pre-trained NLP model that automatically generates semantic vectors and identifies key-value pairs without requiring manual template creation or modification. The model self-adjusts to different text styles and document formats through its pre-trained knowledge, eliminating the need for human intervention in template manufacturing while maintaining automated extraction functionality.
2Device complexity
If traditional OCR methods are used to extract text, then the process is simple, but accuracy in identifying key information is diminished due to noise and lack of semantic relationships
Solution Approach 1:
The system transforms text extraction from a simple optical recognition task to a semantic understanding task by generating semantic vectors that capture the meaning and contextual relationships of text. This parameter transformation enables the system to distinguish key information from noise based on semantic similarity rather than just text presence, significantly improving identification accuracy while maintaining processing simplicity through automated vector generation and matching.
3Measurement precision
If information templates are modified to adapt to text style changes, then extraction accuracy improves, but the process becomes more labor intensive
Solution Approach 1:
Instead of modifying templates manually, the system changes the approach by using pre-trained NLP models that automatically adapt to different text styles through their pre-trained knowledge bases. The model generates semantic vectors that naturally accommodate various text formats without requiring template updates, eliminating the time-consuming manual modification process while maintaining high extraction accuracy across different document types.
Data Source
AI summary
Information extraction and image restructuring includes generating semantic vectors to encode portions of text extracted from a document. For each semantic vector a semantic similarity between a schema key and other text encoded therein is determined based on their respective positions within the document. An enhanced NLP model is created using the semantic vectors, each labeled according to the semantic similarity. The text, including schema key, are re-encoded as a key and candidate vectors. Key-value pairs are generated by matching the key vector with a predetermined number of candidate vectors. The enhanced NLP model, using prompt learning, is repurposed to perform a next-sentence prediction that predicts which of the candidate vectors is logically related to the schema key. Based on the next-sentence prediction, the discrete portion of text identified as the schema key and portion of text determined to be logically related thereto are output.