Language Detection in Document Images Using Shallow Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional OCR systems face inefficiencies in multilingual document recognition due to the need for specific language models and inability to differentiate between similar font languages, leading to garbled characters and increased processing delays.
Innovation Solution
A computer-implemented method using a text recognition algorithm trained with a simple shallow neural network and a generated mixed language corpus for rapid language detection, which determines the language of characters in images and computes confidence levels to ensure accurate identification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple language models are used for multilingual document recognition, then language coverage is improved, but processing time and system complexity increase
Solution Approach 1:
The system performs preliminary language detection on the input image to identify the language type before selecting the appropriate OCR model. This preliminary action avoids the need to try multiple models sequentially, significantly reducing processing time while maintaining support for multiple languages.
Solution Approach 2:
A language detection model serves as an intermediary component between the input image and the OCR recognition models. This mediator identifies the language type and routes the image to the appropriate specialized model, eliminating the need for trial-and-error processing with multiple models.
2Adaptability or versatility
If multiple language models are used for multilingual document recognition, then language coverage is improved, but device complexity increases
Solution Approach 1:
The system segments the OCR processing task into two distinct stages: language detection and language-specific recognition. This segmentation allows the use of a simple language detection model followed by a specialized recognition model, reducing overall system complexity compared to using multiple full-featured OCR models.
Solution Approach 2:
The language detection model acts as an intermediary that simplifies the system architecture by providing a clear decision path for model selection, avoiding the complexity of managing and coordinating multiple OCR models simultaneously.
3Extent of automation
If image classification is used for language detection, then automation is improved, but measurement precision decreases due to inability to differentiate similar fonts
Solution Approach 1:
The language detection model analyzes multiple parameters of the text images including font characteristics, character structures, and linguistic patterns rather than relying on a single classification parameter. This multi-parameter approach maintains high automation while improving precision for distinguishing similar fonts.
Solution Approach 2:
The system uses confidence levels as feedback to determine whether the language detection result is reliable. When the confidence level is low, the system can adjust its processing strategy, such as trying alternative models or methods, thereby maintaining high precision while preserving automation.
Data Source
AI summary
A computer-implemented method, according to one embodiment, includes: receiving an image having characters that correspond to a language, and using a text recognition algorithm to determine a first language believed to correspond to the characters. A first confidence level associated with the first language is also computed, and a determination is made as to whether the first confidence level associated with the first language is outside a predetermined range. In response to determining that the first confidence level associated with the first language is not outside the predetermined range, the first language is output as the given language. The text recognition algorithm is trained using a simple shallow neural network and a generated mixed language corpus. The generated mixed language corpus is formed by: randomly sampling libraries having vocabulary and/or characters therein, and combining the randomly sampled vocabulary and/or characters to form the generated mixed language corpus.