Entity Recognition With Image-Text Fusion for Complete Entities
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing entity recognition methods in business cards face issues with entity discontinuity due to the fragmentation of structured information, leading to inaccuracies in identifying complete entities.
Innovation Solution
An entity recognition method that combines image and textual features to determine the continuity of entities by using a pre-trained feature extracting model and a complete-entity determining model, such as VL-BERT, to splice entities and improve recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If OCR technology is used to obtain text information from business cards, then text extraction efficiency is improved, but entity continuity is lost due to fragmentation
Solution Approach 1:
The patent segments the entity recognition process into two stages: first using OCR to extract text efficiently, then using a NER model to identify entities. This segmentation allows each stage to optimize for its specific function while the overall system maintains entity continuity through the NER stage's ability to understand context and relationships between extracted text elements.
Solution Approach 2:
The patent introduces an intermediary NER (Named Entity Recognition) model that acts as a bridge between OCR text extraction and the final structured information output. This intermediary processes the raw text from OCR, identifies complete entities by understanding context and relationships, and outputs structured information that maintains entity continuity, thus resolving the fragmentation issue while preserving OCR's efficiency.
2Measurement precision
If NER technology is used to obtain structured information, then entity recognition capability is improved, but entity discontinuity problem persists
Solution Approach 1:
The patent applies preliminary action by first extracting text information using OCR before performing entity recognition. This preliminary text extraction creates a foundation that the NER model can process more effectively, allowing the NER stage to focus on identifying complete entities and maintaining continuity rather than also performing text extraction.
Solution Approach 2:
The NER model serves as an intermediary between the fragmented text output from OCR and the requirement for continuous, complete entities. It processes the fragmented text by understanding context, relationships, and semantic meaning to reconstruct complete entities, thus resolving the discontinuity problem while preserving the high entity recognition capability provided by NER technology.
3Productivity
If general-purpose OCR and NER technologies are used separately, then processing speed is improved, but recognition accuracy deteriorates due to entity fragmentation
Solution Approach 1:
The patent merges OCR technology and NER technology into an integrated entity recognition system. Instead of using them as separate, independent processes, the system combines them so that OCR extracts text and the NER model immediately processes this text to identify complete entities. This merging allows the system to maintain the processing speed benefits of both technologies while achieving higher recognition accuracy by ensuring entity continuity through the combined approach.
Data Source
AI summary
An entity recognition method and apparatus, an electronic device, a storage medium, and a computer program product are provided. The method includes: recognizing a to-be-recognized image to determine a preliminary recognition result for entities in the to-be-recognized image; determining, in response to determining that the preliminary recognition result includes a plurality of entities of a same category, image features of the to-be-recognized image and textual features of the plurality of entities; determining whether the plurality of entities is a consecutive complete entity based on the image features and the textual features, to obtain a complete-entity determining result; and obtaining a final recognition result based on the preliminary recognition result and the complete-entity determining result.