Keypoint-Based Text Localization for Skewed Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image analysis systems struggle to accurately detect and recognize text in images due to text being rotated, inclined, or curved, and variations in character height, leading to ineffective identification of text locations and content.
Innovation Solution
A keypoint-based multi-label word segmentation and localization method using machine learning techniques, including a regional proposal network and convolutional neural networks, to identify text regions and keypoints, enabling accurate detection of skewed or curved text and improving optical character recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional image analysis techniques are used to detect text in images, then the system is simple and easy to implement, but the text detection accuracy deteriorates when text is rotated, inclined, curved, or has varying character heights
Solution Approach 1:
The patent segments text detection into multiple stages: first detecting text lines, then detecting individual words within lines, and finally detecting characters. This hierarchical segmentation allows the system to handle complex text variations (rotation, inclination, curvature) by breaking down the problem into manageable sub-tasks, improving detection accuracy without requiring a single overly complex model
Solution Approach 2:
The patent introduces keypoint coordinates as an additional dimensional representation alongside traditional bounding boxes. By representing text regions with multiple keypoints (corners, midpoints) rather than simple rectangular boundaries, the system can accurately describe rotated, inclined, and curved text, significantly improving measurement precision for non-standard text orientations
2Reliability
If traditional text detection methods are used, then the processing speed is fast, but the ability to handle skewed or curved text deteriorates
Solution Approach 1:
The patent performs preliminary text line detection before word and character detection. By first identifying text lines and their orientations, the system prepares the data structure and keypoint coordinates in advance, which accelerates subsequent word and character detection steps. This preliminary organization of data improves processing speed while maintaining reliability in handling skewed and curved text
3Measurement precision
If simple bounding box detection is used, then the computational requirements are low, but the precision in locating text regions with varying orientations deteriorates
Solution Approach 1:
The patent applies different levels of detail to different parts of the text detection process. Text lines use bounding boxes with orientation angles, while individual words and characters use keypoint coordinates. This local differentiation of representation quality allows precise localization of text regions with varying orientations without uniformly applying high computational resources to all detection tasks, optimizing the balance between precision and computational efficiency
Data Source
AI summary
Techniques for keypoint-based multi-label word segmentation and localization are described. A machine learning model identifies bounding regions of text within an image, and then generates multiple channel matrices representing predicted keypoints of the text within the bounding regions. The keypoints can be used to rectify the corresponding graphical content from the image including the text to improve the ability to perform optical character recognition and identify the text. Line and word segmentation and localization can be performed together.