Formula Recognition Using Global and Local Image Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current formula recognition methods in images, particularly those using convolutional neural networks, struggle to accurately recognize formulas with low character resolution, such as handwritten formulas, due to the reduction in feature resolution caused by pooling layers.
Innovation Solution
A formula recognition method utilizing a convolutional neural network with specific configurations, including DenseNet, to extract both global and local image features, combined with a decoder using GRU layers to iteratively decode these features, enhancing the recognition of small characters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If pooling layers are used in the convolutional neural network to process the target image, then the processing speed and abstraction capability are improved, but the feature resolution is reduced, leading to poor recognition of small characters
Solution Approach 1:
The patent segments the feature extraction process into two distinct paths: a global feature extraction path that uses pooling layers for speed, and a local feature extraction path that uses only convolutional layers for high-resolution detail preservation. This segmentation allows the system to simultaneously achieve both processing efficiency and high measurement precision for small characters.
Solution Approach 2:
The patent applies local quality by using different processing strategies for different parts of the image: global pooling operations are applied to overall structure recognition, while local convolutional operations without pooling are applied to small character regions to maintain high feature resolution and recognition accuracy.
2Productivity
If only global image features are extracted using pooling layers, then the processing efficiency is improved, but the recognition accuracy of formulas with low character resolution deteriorates
Solution Approach 1:
The patent merges two feature extraction approaches: global feature extraction using pooling layers and local feature extraction using convolutional layers without pooling. The local image features with high resolution are combined with global features to improve recognition accuracy of formulas with low character resolution while maintaining processing efficiency.
Solution Approach 2:
The patent introduces an intermediary local feature extraction path that processes images through convolutional layers only (without pooling) to generate high-resolution local features. These local features act as an intermediary that bridges the gap between global feature processing and accurate small character recognition, thereby improving overall reliability.
3Device complexity
If the convolutional neural network uses standard pooling operations, then the model complexity is reduced, but the ability to retain high feature resolution for small characters is lost
Solution Approach 1:
The patent segments the neural network into two distinct processing streams: a standard CNN path with pooling layers for global features, and a modified CNN path without pooling layers for local features. This segmentation allows the system to maintain simple model architecture overall while preserving high feature resolution in the local processing path for small character recognition.
Data Source
AI summary
A formula recognition method and apparatus, a computer-readable medium, and an electronic device. The formula recognition method includes acquiring a target image including a formula, processing the target image to obtain a global image feature and a local image feature, and processing the global image feature and the local image feature to obtain the formula included in the target image.