Handwritten mathematical formula primitive feature enhancement method based on adaptive symbol strategy

Abstract

The invention belongs to the technical field of computer vision and artificial intelligence, and discloses a handwritten mathematical formula primitive feature enhancement method based on a self-adaptive symbol strategy, which comprises the following steps of: 1, dynamically scaling primitive scale matching: realizing undistorted scaling by keeping the aspect ratio of an original image, only fixing the height and dynamically calculating the width to obtain a primitive scale matching result; adaptive interpolation is carried out; step 2, pixel disturbance of the morphological filtering chain: isolated noise points are gradually eliminated through the multi-stage morphological filtering chain, broken strokes are repaired, and large-area background interference is removed; and the optimal structure elements are adaptively selected for different symbol types. According to the method, dynamic scaling of geometric constraints is combined with intelligent morphological filtering, so that the core problems of symbol deformation, noise interference, multi-source data difference and the like in handwritten mathematical formula recognition are solved, and the accuracy of a recognition system is remarkably improved. And an efficient and reliable preprocessing scheme is provided for education AI, intelligent correction and other scenes.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and artificial intelligence technology, and in particular relates to a method for enhancing the features of handwritten mathematical formula primitives based on an adaptive symbol strategy. Background Art

[0002] Handwritten mathematical formula recognition is an important research area in computer vision and artificial intelligence, particularly in scenarios such as intelligent education and digitized scientific research documents. However, due to the complexity and diversity of handwritten mathematical formulas, existing recognition technologies still face many challenges in practical applications, mainly in the following aspects:

[0003] Symbol diversity interference: Similar symbols in handwritten mathematical formulas (such as "0" and "o", "1" and "l") are easily confused. Traditional methods rely on fixed threshold segmentation and have poor robustness. Multi-source data differences: Different acquisition devices (scanners / flatbeds) lead to inconsistent image resolution and noise distribution, affecting the model's generalization ability. Structural analysis difficulties: Complex nested structures (such as fractions containing superscripts and subscripts) require manual rules and cannot be modeled end-to-end. Long-tail distribution problem: Insufficient samples of rare symbols (such as C and ⌒) lead to low recognition accuracy. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for enhancing the features of handwritten mathematical formula primitives based on an adaptive symbol strategy to solve the above-mentioned technical problems.

[0005] To address the technical issues of symbol ambiguity and layout parsing difficulties in handwritten formula recognition, the present invention's method for enhancing handwritten mathematical formula primitive features based on an adaptive symbol strategy solves these challenges through an innovative combination of feature enhancement and structural modeling, providing a high-precision technical foundation for educational AI. The specific technical solution is as follows:

[0006] A method for enhancing the primitive features of handwritten mathematical formulas based on an adaptive symbol strategy comprises the following steps:

[0007] Step 1: Dynamically scaled primitive scale matching: By maintaining the aspect ratio of the original image, only fixing the height and dynamically calculating the width, lossless scaling is achieved, and adaptive interpolation is performed;

[0008] Step 2: Pixel perturbation in the morphological filter chain: A multi-stage morphological filter chain is used to gradually eliminate isolated noise points, repair broken strokes, and remove large areas of background interference; the optimal structuring element is adaptively selected for different symbol types.

[0009] Furthermore, the step 1 includes the following steps:

[0010] Step 1.1: Input the handwritten mathematical formula image and obtain its original height H and width W;

[0011] Step 1.2: Use a dynamic scaling algorithm based on geometric similarity to dynamically calculate the width by fixing the height to ensure that the geometric features of the symbol are not destroyed;

[0012] Step 1.3: Use bicubic interpolation algorithm to perform image scaling.

[0013] Furthermore, the step 1.2 includes the following steps:

[0014] Input the original image height H, width W, the target height is a fixed value new_H, and the scaled width new_W is calculated by the following formula:

[0015]

[0016] Aspect ratio conservation: The aspect ratio of the image before and after scaling α = W / H should remain unchanged, that is, α = new_W / new_H;

[0017] Width derivation: From the conservation of α, we can get new_W = new_W × α, and substitute α = W / H.

[0018] Furthermore, the step 1.3 includes the following steps:

[0019] The interpolation kernel function calculation formula is as follows:

[0020]

[0021] Where ω(i,j) is the weight based on the pixel overlap area;

[0022] Use the bicubic interpolation algorithm for image scaling, and its kernel function is:

[0023] ω(x)={(a+2)|x| 3 -(a+3)|x| 2 +1, when |x|≤1

[0024] {a|x| 3 -5a|x| 2 +8a|x|-4a, when 1<|x|<2

[0025] Where a = -0.5, which avoids the jagged effect while preserving the continuity of the strokes.

[0026] Furthermore, the step 2 includes the following steps:

[0027] Step 2.1: Propose a pixel perturbation method based on morphological filter chain;

[0028] Step 2.2: Design a multi-level morphological processing pipeline;

[0029] Step 2.3: Develop an adaptive selection strategy for structural elements.

[0030] Furthermore, the step 2.1 includes the following steps:

[0031] Based on set theory and lattice theory, this algorithm performs local transformations on images through structuring elements. Its core operations include erosion and dilation, which combine to achieve advanced effects of opening and closing operations. The erosion operation is defined as the minimum value filtering of the structuring element on the image:

[0032] I eroded (x,y)=min I(x+u,y+v),(u,v)∈S (1-16)

[0033] Where I(x,y) is the input binary image, S is the structure element matrix, which defines the domain shape and represents the (u,v) offset coordinates within the structure element. If and only if the structure element S completely covers the foreground area, the center pixel is retained as the foreground. This feature is used to eliminate isolated noise points.

[0034] The dilation operation is the dual operation of corrosion and is defined as maximum filtering:

[0035] I dilated (x,y)=max I(x+u,y+v),(u,v)∈S (1-17)

[0036] As long as the structuring element S intersects with the foreground, the center pixel is set to the foreground. This operation is used to repair broken strokes. The mathematical expression of the structuring element S is formalized as a binary matrix. The shape of the matrix directly affects the processing effect. The corresponding formula is as follows:

[0037]

[0038] Where k controls the kernel size, k = 1 corresponds to a 3×3 kernel, and the cross kernel S cross Used to protect stroke intersections.

[0039] Furthermore, the step 2.2 includes the following steps:

[0040] 1) First level: Use 3×3 cross-shaped structural elements for corrosion to eliminate isolated noise points;

[0041] 2) Second level: dilation using 2×2 rectangular structural elements to repair broken strokes;

[0042] Level 3: Adaptive open operation to remove large-area background interference.

[0043] Furthermore, the step 2.3 includes the following steps:

[0044] For slender symbols: use 1×3 horizontal structural elements;

[0045] For cross symbols: use 3×3 cross structure element;

[0046] For complex structures: use 5×5 circular structural elements.

[0047] The method for enhancing the features of handwritten mathematical formula primitives based on the adaptive symbol strategy of the present invention has the following advantages:

[0048] 1. Effectively eliminate symbol deformation and maintain geometric feature consistency

[0049] The height is fixed and the width is adaptively adjusted through a dynamic scaling algorithm, strictly maintaining the original aspect ratio of the symbol and avoiding geometric distortion caused by forced scaling (such as a circular symbol being deformed into an ellipse).

[0050] A bicubic interpolation algorithm is used for high-quality image scaling, which suppresses the aliasing effect while preserving the continuity of strokes, ensuring that the features extracted by the convolutional neural network have physical consistency (for example, the two lines of the "+" sign are always orthogonal).

[0051] 2. Intelligently remove noise and repair strokes to enhance symbol recognizability

[0052] Through a multi-stage morphological filtering chain (erosion → dilation → adaptive opening operation), isolated noise points are gradually eliminated, broken strokes are repaired, and large-area background interference is removed, significantly improving image quality.

[0053] The optimal structural elements are adaptively selected for different symbol types (slender, cross, complex structure), avoiding excessive corrosion or expansion (such as "1" breaking or "=" sticking) caused by traditional fixed core methods.

[0054] 3. Improve multi-source data compatibility and model generalization capabilities

[0055] Dynamic scaling and adaptive morphological processing can adapt to input images from different acquisition devices (scanners, flatbeds, cameras), effectively alleviating problems such as resolution differences and uneven noise distribution.

[0056] Standardized output formats (uniform height, centered layout, and binarized tensors) reduce data distribution differences and enhance the robustness of the model in different application scenarios (such as K12 education and scientific research documents).

[0057] 4. Optimize the structural analysis ability of complex formulas

[0058] Through stroke repair and noise suppression, connected symbols (such as the numerator and fraction line in a fraction) are clearly separated, providing high-precision input for subsequent two-dimensional layout analysis (such as nested superscripts and subscripts, and square roots).

[0059] The adaptive structural element strategy protects key topological features (such as the intersection of "×" and the curve continuity of "∫") and reduces the error rate of structural analysis.

[0060] 5. Efficient computing, suitable for real-time applications

[0061] Dynamic width calculation only requires one division operation (time complexity O(1)), which is much lower than the traditional projection-based adaptive segmentation method.

[0062] The number of iterations of the morphological filter chain is controllable (default is 1), which ensures the effect while meeting the real-time requirements of educational applications (such as handwriting calculators).

[0063] In summary, this invention combines dynamic scaling of geometric constraints with intelligent morphological filtering to address core issues in handwritten mathematical formula recognition, such as symbol deformation, noise interference, and multi-source data discrepancies. This significantly improves the accuracy of the recognition system (experiments show that symbol recognition accuracy on public datasets has increased by over 12%). Furthermore, its modular design allows for seamless integration into existing recognition processes, providing an efficient and reliable preprocessing solution for scenarios such as educational AI and intelligent grading. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a schematic diagram of the effect of the present invention. DETAILED DESCRIPTION

[0065] In order to better understand the purpose, structure and function of the present invention, the handwritten mathematical formula primitive feature enhancement method based on the adaptive symbol strategy of the present invention is further described in detail below with reference to the accompanying drawings.

[0066] In the task of handwritten mathematical formula recognition, unifying multi-source datasets is crucial. This can eliminate data distribution differences (such as resolution, noise, and writing style), standardize input formats (unified image size, binarization, and annotation specifications), enhance data quality (denoising and edge enhancement), and balance category distribution (oversampling rare symbols). This significantly improves the model's generalization, training efficiency, and recognition accuracy, ensuring the algorithm's robustness in various scenarios. The core of multi-source handwritten mathematical formula image processing lies in enhancing symbol features and eliminating noise interference, improving the stroke-level feature representation of formula symbols. To ensure the uniformity of input images and improve the accuracy of subsequent processing, a multi-source image normalization algorithm based on adaptive symbol feature enhancement is proposed. The algorithm consists of primitive scale matching based on dynamic scaling and a pixel perturbation method based on a morphological filter chain. The processing flow and pseudocode are shown below, along with the corresponding mathematical formulas and explanations.

[0067]

[0068]

[0069] The method for enhancing the features of handwritten mathematical formula primitives based on an adaptive symbol strategy of the present invention comprises the following steps:

[0070] Step 1: Dynamically scaled primitive scale matching: By maintaining the aspect ratio of the original image, only fixing the height and dynamically calculating the width, lossless scaling is achieved, and adaptive interpolation is performed.

[0071] Due to the different data collection devices and sources, the background symbols used in handwritten mathematical formula recognition vary significantly, and the original input images often have different scale ratios. This size difference directly affects the performance of subsequent feature extraction and recognition models. Overly large images waste computing resources, while undersized images may lose key details (such as the continuity of fraction lines or the relationship between superscripts and subscripts). To ensure the stability of the subsequent multi-scale symbol feature extraction and recognition algorithms, we need to reduce the long-tail problem of the model in multi-scale symbol recognition.

[0072] Traditional image scaling typically uses fixed-size input or simple cropping strategies, but these methods have significant drawbacks: Fixed-size forced scaling: Directly stretching all images to a uniform size introduces distortion, disrupting the geometric proportions of symbols (e.g., a circular symbol "○" becomes an ellipse), making it difficult for convolutional neural networks to extract effective features. Crop-and-fill methods: Padding the edges to achieve uniform size, but this introduces redundant noise, and the padded areas can interfere with the attention mechanism's ability to locate the main structure of the formula.

[0073] To address these issues, we propose a primitive scale matching method based on dynamic scaling. The core idea is to maintain the aspect ratio of the original image, fix only the height, and dynamically calculate the width, achieving lossless scaling and performing adaptive interpolation. This method significantly enhances robustness when dealing with complex structures (such as multi-level fractions and matrices).

[0074] Step 1.1: Input the handwritten mathematical formula image and obtain its original height H and width W;

[0075] Step 1.2: Use a dynamic scaling algorithm based on geometric similarity to dynamically calculate the width by fixing the height (new_H = 80px) to ensure that the geometric characteristics of the symbol are not destroyed;

[0076] The goal of the dynamic scaling algorithm is to convert an input image of any size into a standard format with uniform height but adaptive width. Its mathematical foundation is the similarity principle in geometric transformations. Given the original image height H and width W, and a fixed target height new_H, the scaled width new_W can be calculated using the following formula:

[0077]

[0078] Aspect ratio conservation: The aspect ratio of the image before and after scaling, α = W / H, should remain unchanged, that is, α = new_W / new_H.

[0079] Width derivation: From the conservation of α, we can get new_W = new_W × α, and substitute α = W / H.

[0080] Step 1.3: Image scaling using bicubic interpolation

[0081] The interpolation kernel function calculation formula is as follows:

[0082]

[0083] where ω(i, j) is a weight based on the pixel overlap area. The proposed dynamic scaling algorithm maintains consistency in the primitive geometry through geometric constraints and optimized interpolation. Aspect ratio constraints prevent sign distortion and ensure that the features extracted by the convolutional neural network are physically consistent (e.g., the two lines of the "+" sign are always orthogonal). Computational efficiency is optimized: dynamic width calculation requires only a single division operation, with a time complexity of O(1), far lower than the adaptive segmentation step in traditional methods, such as projection-based width estimation.

[0084] Use the bicubic interpolation algorithm for image scaling, and its kernel function is:

[0085] ω(x)={(a+2)|x| 3 -(a+3)|x| 2+1, when |x|≤1

[0086] {a|x| 3 -5a|x| 2 +8a|x|-4a, when 1<|x|<2

[0087] Where a = -0.5, which avoids the jagged effect while preserving the continuity of the strokes.

[0088] Step 2: Pixel perturbation in the morphological filter chain: A multi-stage morphological filter chain is used to gradually eliminate isolated noise points, repair broken strokes, and remove large areas of background interference; the optimal structuring element is adaptively selected for different symbol types.

[0089] Step 2.1: Propose a pixel perturbation method based on morphological filter chain;

[0090] In handwritten mathematical formula image recognition, the input image is contaminated by various noise sources. These include noise introduced by writing tools, graphite particle diffusion during pencil writing, sampling jitter from electronic tablets, and abnormal sensing points when writing with a passive pen. There is also interference from the paper background during image scanning, such as yellowed paper texture or printed background lines. There is also digitization noise introduced by the hardware during formula acquisition, and salt and pepper noise from scanners or cameras.

[0091] While traditional denoising methods such as Gaussian filtering or median filtering can smooth out noise, they can blur symbol edges (for example, the two short lines of the equal sign "=" may become intertwined), severely impacting subsequent structural analysis. Linear filtering methods are particularly difficult to distinguish between noise and valid strokes in binarized formula images.

[0092] This paper proposes a pixel perturbation method based on a morphological filter chain. Based on set theory and lattice theory, it uses structuring elements (SEs) to perform local transformations on images. Its core operations include erosion and dilation, which can be combined to achieve advanced effects such as opening and closing. The erosion operation is defined as a minimum filter on the structuring element over the image:

[0093] I eroded (x,y)=min I(x+u,y+v),(u,v)∈S (1-16)

[0094] Where I(x,y) is the input binary image, and S is the structuring element matrix, which defines the domain shape and represents the (u,v) offset coordinates within the structuring element. If and only if the structuring element S completely covers the foreground area, the center pixel is retained as the foreground. This feature effectively eliminates isolated noise points, such as those with an area smaller than S, but can result in thinner strokes (for example, the digit "1" may be broken). The dilation operation is the dual operation of erosion and is defined as a maximum filter:

[0095] I dilated (x,y)=max I(x+u,y+v),(u,v)∈S (1-17)

[0096] As long as the structuring element S intersects the foreground, the center pixel is set to the foreground. This operation can repair broken strokes. The mathematical expression of the structuring element S can be formalized as a binary matrix. The shape of the matrix directly affects the processing effect. The corresponding formula is as follows:

[0097]

[0098] Where k controls the kernel size, k = 1 corresponds to a 3×3 kernel. cross ) is used to protect the intersection of strokes (such as the "×" sign). The present invention proposes a pixel perturbation algorithm based on the morphological filter chain. In the selection of dynamic structural elements, for thin line symbols (such as "-", "'"), a 1×3 horizontal kernel S is used. cross = [1 1 1] to erode and avoid excessive erosion in the vertical direction; for isolated noise, a 2×2 rectangular kernel S is used. cross Carry out corrosion.

[0099] Step 2.2: Design a multi-level morphological processing pipeline:

[0100] 1) First level: Use 3×3 cross-shaped structural elements for corrosion to eliminate isolated noise points;

[0101] 2) Second level: dilation using 2×2 rectangular structuring elements to repair broken strokes;

[0102] 3) Level 3: Adaptive open operation to remove large-area background interference;

[0103] Step 2.3: Develop an adaptive selection strategy for structural elements:

[0104] For thin and long symbols (such as "-", "|"): use 1×3 horizontal structure elements;

[0105] For cross symbols (such as "+", "×"): use 3×3 cross structure element;

[0106] For complex structures (such as the integral symbol "∫"): a 5×5 circular structure element is used.

[0107] Figure 1The image is then converted to a binary image after processing. This process emphasizes the strokes of symbols in the formula, reduces background noise, and enhances the contrast between symbols and background to facilitate feature extraction. This ensures that the formula is centered in the image, preventing symbols from being truncated near the edges. Erosion and filling operations separate contiguous symbols, making it easier to parse complex mathematical formula structures (such as fractions, subscripts, and radicals). These steps effectively enhance the representation of primitive features, improve the consistency of multi-source images, and enhance the robustness and accuracy of the model.

[0108] It will be understood that the present invention is described by way of some embodiments, and it will be appreciated by those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are intended to be protected by the present invention.

Claims

1. A method for enhancing the primitive features of handwritten mathematical formulas based on an adaptive symbol strategy, characterized in that: The following steps are involved: Step 1: Dynamically scaled primitive scale matching: By maintaining the aspect ratio of the original image, only fixing the height and dynamically calculating the width, lossless scaling is achieved, and adaptive interpolation is performed; Step 2: Pixel perturbation in the morphological filter chain: A multi-stage morphological filter chain is used to gradually eliminate isolated noise points, repair broken strokes, and remove large areas of background interference; the optimal structuring element is adaptively selected for different symbol types.

2. The method for enhancing the features of handwritten mathematical formula primitives based on an adaptive symbol strategy according to claim 1, characterized in that: The step 1 comprises the following steps: Step 1.1: Input the handwritten mathematical formula image and obtain its original height H and width W; Step 1.2: Use a dynamic scaling algorithm based on geometric similarity to dynamically calculate the width by fixing the height to ensure that the geometric features of the symbol are not destroyed; Step 1.3: Use bicubic interpolation algorithm to perform image scaling.

3. The method for enhancing the features of handwritten mathematical formula primitives based on an adaptive symbol strategy according to claim 2, characterized in that: The step 1.2 includes the following steps: Input the original image height H, width W, the target height is a fixed value new_H, and the scaled width new_W is calculated by the following formula: Aspect ratio conservation: The aspect ratio of the image before and after scaling, α = W / H, should remain unchanged, that is, α = new_W / new_H; Width derivation: From the conservation of α, we can get new_W = new_W × α, which can be substituted into α = W / H.

4. The method for enhancing the features of handwritten mathematical formula primitives based on an adaptive symbol strategy according to claim 2, characterized in that: The step 1.3 includes the following steps: The interpolation kernel function calculation formula is as follows: Where ω(i,j) is the weight based on the pixel overlap area; Use the bicubic interpolation algorithm for image scaling, and its kernel function is: ω(x) = {(a + 2)|x| 3 -(a + 3)|x| 2 + 1, when |x| ≤ 1 {a|x| 3 -5a|x| 2 +8a|x|-4a, when 1 < |x| < 2 Where a = -0.5, which avoids the jagged effect while preserving the continuity of the strokes.

5. The method for enhancing the features of handwritten mathematical formula primitives based on an adaptive symbol strategy according to claim 1, characterized in that: The step 2 comprises the following steps: Step 2.1: Propose a pixel perturbation method based on morphological filter chain; Step 2.2: Design a multi-level morphological processing pipeline; Step 2.3: Develop an adaptive selection strategy for structural elements.

6. The method for enhancing the features of handwritten mathematical formula primitives based on an adaptive symbol strategy according to claim 5, characterized in that: The step 2.1 includes the following steps: Based on set theory and lattice theory, this algorithm performs local transformations on images through structuring elements. Its core operations include erosion and dilation, which combine to achieve advanced effects of opening and closing operations. The erosion operation is defined as the minimum value filtering of the structuring element on the image: I eroded (x,y)=minI(x+u,y+v),(u,v)∈S (1-16) Where I(x,y) is the input binary image, S is the structure element matrix, which defines the domain shape and represents the (u,v) offset coordinates within the structure element. If and only if the structure element S completely covers the foreground area, the center pixel is retained as the foreground. This feature is used to eliminate isolated noise points. The dilation operation is the dual operation of corrosion and is defined as maximum filtering: Yo dilated (x,y)=maxI(x+u,y+v),(u,v)∈S (1-17) As long as the structuring element S intersects with the foreground, the center pixel is set to the foreground. This operation is used to repair broken strokes. The mathematical expression of the structuring element S is formalized as a binary matrix. The shape of the matrix directly affects the processing effect. The corresponding formula is as follows: Where k controls the kernel size, k = 1 corresponds to a 3×3 kernel, and the cross kernel S cross Used to protect stroke intersections.

7. The method for enhancing the features of handwritten mathematical formula primitives based on an adaptive symbol strategy according to claim 5, characterized in that: The step 2.2 includes the following steps: 1) First level: Use 3×3 cross-shaped structural elements for corrosion to eliminate isolated noise points; 2) Second level: dilation using 2×2 rectangular structuring elements to repair broken strokes; 3) Level 3: Adaptive open operation to remove large-area background interference.

8. The method for enhancing the features of handwritten mathematical formula primitives based on an adaptive symbol strategy according to claim 5, characterized in that: The step 2.3 includes the following steps: For slender symbols: use 1×3 horizontal structural elements; For cross symbols: use 3×3 cross structure element; For complex structures: use 5×5 circular structural elements.

Images