Image-Text Entity Recognition for Split-Line Entity Continuity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing entity recognition methods suffer from entity discontinuity issues, particularly in recognizing structured information from business cards, leading to inaccuracies in extracting complete entities due to text being split across multiple lines.
Innovation Solution
An entity recognition method that combines textual features with image features to determine the continuity of entities by using a pre-trained global pointer model, feature extraction networks, and a complete-entity determining model like VL-BERT to splice split entities into complete entities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If OCR technology is used to obtain text information from business cards, then text extraction is achieved, but entity discontinuity occurs when text is split across multiple lines
Solution Approach 1:
The patent merges multiple features (image features, textual features, layout features, and semantic features) into a unified representation to determine entity continuity. This combination allows the system to reconstruct complete entities even when text is fragmented across multiple lines in the OCR output.
Solution Approach 2:
The patent introduces an entity continuity determination module as an intermediary between OCR text extraction and final entity recognition. This module uses multiple feature types to bridge the gap between fragmented text and complete entities, acting as a mediator that restores continuity.
2Productivity
If NER technology is applied after OCR to extract structured information, then key field information is obtained, but entity discontinuity problems persist
Solution Approach 1:
The patent performs preliminary entity continuity determination before final entity recognition. By pre-processing the OCR output with multiple feature analyses, the system prepares corrected entity representations that maintain continuity, making the subsequent NER process more accurate and efficient.
Solution Approach 2:
The patent uses a composite approach combining multiple feature types (image, text, layout, semantics) similar to composite materials. This multi-feature composition creates a more robust entity recognition system that overcomes the limitations of traditional NER when applied to fragmented OCR text.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure provides an entity recognition method and apparatus, an electronic device, a storage medium, and a computer program product, relates to the field of artificial intelligence, specifically relates to the technical field of deep learning and image recognition, and may be used in a scenario of named entity recognition. A specific implementation solution includes: recognizing a to-be-recognized image to determine a preliminary recognition result for entities in the to-be-recognized image; determining, in response to determining that the preliminary recognition result includes a plurality of entities of a same category, image features of the to-be-recognized image and textual features of the plurality of entities; determining whether the plurality of entities is a consecutive complete entity based on the image features and the textual features, to obtain a complete-entity determining result; and obtaining a final recognition result based on the preliminary recognition result and the complete-entity determining result. The present disclosure solves the problem of the entity discontinuity, and improves the accuracy of entity recognition.