Document Image Analysis with OCR and Transformer Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
There is a significant need for improved accessibility and comprehension of information in documents for individuals with vision impairment, illiteracy, and other disabilities that affect reading skills, as over 2 billion people suffer from near or distance vision impairment, and 763 million adults are illiterate, hindering their ability to read and write.
Innovation Solution
A computer-implemented method using transformer-based machine learning and neural networks for image analysis and processing, which extracts sub-images from documents, applies optical character recognition, classifies sections, and generates textual summaries, enhancing accessibility through label output and user interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional text extraction and reading methods are used, then reading accuracy is maintained for able readers, but accessibility is severely limited for individuals with vision impairment or illiteracy
Solution Approach 1:
The patent introduces an intermediary system comprising OCR technology, NLP processing, and text-to-speech conversion that mediates between the physical document and users with disabilities. This intermediary transforms inaccessible physical text into accessible digital formats with audio output, enabling users with vision impairment or illiteracy to access document content without directly reading the original text
Solution Approach 2:
The patent replaces the mechanical process of visual reading with an automated digital processing system. Instead of requiring users to visually process text, the system uses OCR to extract text, NLP to analyze and structure it, and text-to-speech to convert it into audio, substituting the mechanical act of reading with automated technological processes
2Loss of information
If comprehensive text extraction is performed on entire documents, then complete information is captured, but processing time and computational resources increase significantly
Solution Approach 1:
The patent applies segmentation by dividing the document processing into distinct stages: OCR text extraction, NLP text analysis and structuring, and text-to-speech conversion. This segmentation allows each component to process only its specific portion of the data pipeline, reducing the computational burden on any single processing step while maintaining complete information capture
Solution Approach 2:
The patent performs preliminary OCR text extraction and NLP structuring before the text-to-speech conversion. By preparing the text data in advance with proper structuring and formatting, the system reduces the processing time required during the final audio generation phase, as the conversion process receives pre-processed, organized text rather than raw unstructured data
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-implemented method comprising performing an image analysis process based on an image of a physical document, the image analysis process comprising: extracting from the image sub-images of sections of the physical document; and for each sub-image, extracting text in the sub-image using optical character recognition, OCR, classifying, based on the sub-image and the text in the sub-image and using a transformer-based machine learning, ML, model, the sub-image as corresponding to a label among a set of labels; generating, based on the sub-image and based on the text in the sub-image and using a neural network, a textual summary of the text in the sub-image; and outputting the labels of the sub-images and, in response to selection of at least one label by a user, outputting the corresponding at least one textual summary.