Handwritten Form Field Extraction via Machine Learning Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
State-of-the-art Intelligent Character Recognition (ICR) engines are not sufficiently capable for real-life applications, often resulting in high error rates when converting images of handwritten text into textual content, and require significant human labeling and computational inefficiencies in processing large numbers of forms.
Innovation Solution
A machine learning system that identifies and extracts handwritten form fields by preprocessing training data to determine the average form structure, using incremental principal component analysis to reduce noise and variance, and generating connected components to filter out non-essential pixels, thereby focusing comparisons on specific form fields for efficient processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional ICR engines are used to convert handwritten text images into textual content, then the conversion process can be performed, but high error rates occur and significant human labeling is required
Solution Approach 1:
The patent segments the form image into multiple regions of interest (ROIs) based on form field locations. Instead of processing the entire form image, the system divides it into discrete fields such as name, address, and phone number areas. This segmentation allows the ICR engine to focus on specific regions, improving recognition accuracy while reducing the computational burden and human labeling requirements.
Solution Approach 2:
The system performs preliminary processing by pre-identifying form field locations and extracting ROIs before applying the ICR engine. Training data is used to establish the spatial locations of form fields in advance, so that when processing actual forms, the system can directly extract and process only the relevant regions. This preliminary action eliminates the need for human labeling of each character while maintaining high accuracy.
2Productivity
If traditional ICR engines process handwritten forms, then text conversion is achieved, but computational inefficiencies occur when processing large numbers of forms
Solution Approach 1:
The patent extracts only the essential regions of interest from complete form images. By using training data to identify form field locations, the system extracts only the relevant text regions for processing, discarding the rest of the form image. This extraction approach significantly reduces computational workload and energy consumption while maintaining processing throughput for large volumes of forms.
Solution Approach 2:
The system applies different processing quality levels to different regions of the form. High-quality processing is applied only to identified form field regions, while other areas are either discarded or processed with lower quality settings. This local quality approach optimizes computational efficiency by concentrating resources on essential areas, enabling high productivity when processing large numbers of forms.
3Loss of information
If the entire form image is processed for handwritten text recognition, then all text can be captured, but processing time and computational resources increase significantly
Solution Approach 1:
The form image is segmented into multiple form field regions based on training data that identifies the location and boundaries of each field. Instead of processing the entire image, the system processes only these segmented regions individually. This segmentation ensures that all relevant text information is captured while significantly reducing processing time compared to analyzing the complete form image.
Solution Approach 2:
The system performs preliminary identification of form field locations using training data before the actual text recognition process. This preliminary action creates a map of where text should be located, allowing the system to directly extract and process only those regions. As a result, no text information is lost while processing time is dramatically reduced.
Data Source
AI summary
Disclosed herein is a machine learning application for automatically reading filled-in forms. There are multiple steps involved in using a computer to accurately read a handwritten form. First, the system identifies the form. Second, the system identifies what parts of the form are important. Third, the important parts are extracted as image data (known as shreds). Finally, fourth, the system interprets the shreds. This application is focused on steps two and three of that overall process. The disclosed techniques relate to training a machine learning system on a given series of forms such that when provided future filled-in forms within that series, the system is able to extract the portions of the filled-in form that are important/relevant.