Coding and decoding-based mathematical formula identification method and device, and readable storage medium

A technology of mathematical formulas and recognition methods, applied in the field of image recognition, to achieve the effect of enhancing feature extraction, simplifying network structure, and enhancing the ability to distinguish

Pending Publication Date: 2022-03-29
NANJING UNIV OF POSTS & TELECOMM
View PDF0 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

These methods are usually based on recurrent neural network structures, which have problems of timing dependence and computational complexity

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Coding and decoding-based mathematical formula identification method and device, and readable storage medium
  • Coding and decoding-based mathematical formula identification method and device, and readable storage medium
  • Coding and decoding-based mathematical formula identification method and device, and readable storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0039] A kind of mathematical formula recognition method based on encoding and decoding described in the present invention, such as figure 1 As shown, it specifically includes the following steps:

[0040] Step 1. Image preprocessing: crop and adjust the size of the picture containing only handwritten mathematical formulas. The picture only contains all the formula parts and as few blank areas as possible, and the size is cut to 256*256 pixels;

[0041] Step 2. Image feature encoding: pass the processed image through the encoding network fused by the improved ResNet convolutional network and the position encoding module to obtain the input of the decoding network.

[0042] The traditional convolutional neural network will face the problem of gradient disappearance / gradient explosion after the depth of the network is deepened. Therefore, ResNet introduces a residual network structure, that is, a feed-forward shortcut connection is introduced between the input and output, so tha...

Embodiment 2

[0070] A device applied to the codec-based mathematical formula recognition method, the device comprising:

[0071] The image processing module is used to crop and grayscale the pictures containing only formulas;

[0072] The feature encoding module connected with the module is used to complete the extraction of image feature information, and by position encoding, calculate and add position information;

[0073] The feature decoding module connected with the module is used for image feature sequence decoding and character prediction. The encoding network is connected by sub-networks, each sub-network contains a multi-head self-attention network and a forward network; the decoding network is used to calculate the positional relationship of the feature sequence and output a predictive sequence, using the L-softmax function for prediction The relationship between sequences imposes stronger constraints, and through these predicted sequences, the best character path is selected, a...

Embodiment 3

[0076] Based on the off-line handwritten mathematical formula recognition method based on the encoding and decoding model in the first embodiment, the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor , to realize the steps of the above method.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

According to the mathematical formula recognition method and device based on coding and decoding and the readable storage medium, feature coding is carried out on an input picture through a ResNet network and a position coding module, then a multi-head attention model and a forward network are combined to carry out decoding calculation on a feature sequence, prediction is achieved, and the steps of single-character cutting and recognition are avoided; the spatial relationship between characters can be learned from the overall information of the handwritten mathematical formula, and finally the recognition of the whole handwritten mathematical formula is completed. The method has the beneficial effects that the position information is added in the output of the ResNet network in the coding module, so that the coding module can more accurately learn the feature information of the formula picture; in the decoding module, different from a method using a recurrent neural network, the method performs parallel calculation by using a multi-head attention model, so that the running speed is obviously improved.

Description

technical field [0001] The invention relates to the technical field of image recognition, in particular to an end-to-end off-line handwritten mathematical formula recognition method and device based on an encoding and decoding model. Background technique [0002] Mathematical formulas are often used in daily life, especially in the field of education, scientific and technological work, etc. Therefore, the effective identification of mathematical formulas has become a very important task. Among them, because of its convenience, handwritten mathematical formulas also make their correct recognition more practical. But unlike ordinary text, mathematical formulas often contain complex two-dimensional structures, and in offline handwritten mathematical formulas, traditional optical character recognition technology often cannot be used because of the irregularity of handwritten characters and the inability to obtain stroke information. Therefore, handwritten mathematical formula r...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06V10/774G06V10/82G06K9/62G06F17/16G06N3/04G06N3/08
CPCG06F17/16G06N3/08G06N3/045G06F18/214
Inventor 周名杰程艳云
Owner NANJING UNIV OF POSTS & TELECOMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap