Offline handwritten mathematical formula recognition and reconstruction method

Abstract

The invention discloses an offline handwritten mathematical formula recognition and reconstruction method, which is characterized in that a character segmentation problem is converted into a target detection problem in a formula segmentation stage, so that the segmentation problem of broken characters and adhesion characters is effectively solved, and the accuracy of a character segmentation result is greatly improved; at the character recognition phase, two convolutional neural network models based on batch normalization and global average pooling methods and only using convolution kernels with the size of 3 * 3 are designed for the problem of offline handwritten mathematical character recognition, the voting results of the two models are used as recognition results in the recognition stage, so that the recognition of mathematical characters can be accurately completed, and the character recognition accuracy is improved; at the formula analysis and reconstruction stage, the geometricconstraint is carried out according to the position relation among the characters, the semantic constraint is carried out by using a two-dimensional random context-independent grammar, and finally, the characters are combined from bottom to top, so that the formula reconstruction is completed.

Description

technical field [0001] The invention relates to the technical field of image processing and pattern recognition, in particular to an off-line handwritten mathematical formula recognition method. Background technique [0002] Mathematical formulas are very common in scientific and technological documents such as textbooks and scientific papers. Unlike text lines with a one-dimensional structure, the spatial relationship between characters in mathematical formulas is complex, and there are two-dimensional structures such as up-down, left-right, upper-right, lower-right, and surround. Due to the complex structure of mathematical formulas, the process of inputting formulas into electronic documents is cumbersome and complicated, and the recognition technology of mathematical formulas can easily convert formula images or stroke sequences of handwritten formulas into editable formula formats in electronic documents. [0003] At present, mathematical formula recognition systems are...

AI Technical Summary

Problems solved by technology

Due to the arbitrariness of handwritten characters, compared with printed math characters, handwritten math characters have serious adhesion, different characters have different sizes, and the same character has different shapes, which has caused many difficulties for the segmentation and recognition of off-line handwritten math formulas.

Compared with online data, offline data lacks time series information and is more challenging in terms of formula segmentation and recognition

Images