Intelligent handwritten mathematical formula recognition correction method and system, and medium

Abstract

This invention provides an intelligent handwritten mathematical formula recognition and correction method, system, and medium, relating to the field of text recognition technology. The method includes: acquiring raw image data of handwritten mathematical formulas; preprocessing to obtain standardized image data; determining a structural feature matrix; identifying formula error types and patterns based on the changing trends of formula grammatical topological relationships and grammatical logic deviation coefficients over multiple recognition cycles; configuring correction confidence factors; determining the formula error level; performing graded correction feedback; and outputting a standard formula. It also drives a voice broadcast unit to provide synchronous voice correction prompts. This invention solves the technical problems of instability in handwritten mathematical formula recognition, difficulty in guaranteeing recognition and correction accuracy, and resulting in erroneous or inconsistent recognition results. It achieves a complete closed loop from image acquisition to intelligent correction of handwritten mathematical formulas, thereby improving the accuracy, stability, and correction precision of handwritten mathematical formula recognition.

Description

Technical Field

[0001] This invention relates to the field of text recognition technology, specifically to a method, system, and medium for intelligent handwritten mathematical formula recognition and correction. Background Technology

[0002] Handwritten mathematical formulas possess unique advantages due to their natural interaction method that aligns with human cognition. They allow users to quickly record complex mathematical structures with intuitive and free strokes, making them particularly suitable for classroom notes, creative derivations, and personalized problem-solving processes. Handwritten formulas not only help deepen the memory and understanding of mathematical concepts, but their flexible spatial layout and symbol connection methods also more accurately reflect the user's thought process. In education, scientific research, and daily communication, handwritten formulas lower the barrier to formal input, becoming an important way to express and convey mathematical thinking.

[0003] However, while existing handwritten formula recognition methods can handle some standardly written formulas, their accuracy remains unstable for personalized, illegible, or complex nested structures, particularly prone to errors in symbol ambiguity, spatial relationship resolution, and contextual understanding. Especially when dealing with complex formulas or irregular handwriting, current technologies cannot consistently guarantee high-precision recognition and correction, leading to errors or inconsistencies in handwritten formula recognition results.

[0004] Existing technologies suffer from unstable handwritten mathematical formula recognition and difficulty in guaranteeing recognition and correction accuracy, leading to errors or inconsistencies in recognition results. Summary of the Invention

[0005] The purpose of this application is to provide an intelligent handwritten mathematical formula recognition and correction method, system and medium to solve the technical problems of unstable handwritten mathematical formula recognition and difficulty in guaranteeing recognition and correction accuracy in the existing technology, which leads to errors or inconsistencies in the recognition results.

[0006] In view of the above problems, this application provides an intelligent handwritten mathematical formula recognition and correction method, system and medium.

[0007] The first aspect of this application provides an intelligent handwritten mathematical formula recognition and correction method, which includes: acquiring raw image data of handwritten mathematical formulas, optimizing image quality through denoising and binarization preprocessing to obtain standardized image data; performing coupling analysis based on the contour matching threshold corresponding to symbol strokes, the structural specification threshold corresponding to operators, and the grammatical logic deviation coefficient corresponding to expressions, combined with the formula grammatical topology, to formulate a structural feature matrix; simultaneously, based on the formula grammatical topology and the changing trend of the grammatical logic deviation coefficients over multiple recognition periods, identifying formula error type patterns; configuring correction confidence factors based on the structural feature matrix, combined with the formula error type patterns and standardized image data, wherein the correction confidence factors are strongly correlated with the input layer weights of the formula recognition and correction model; using the correction confidence factors, determining the formula error level for graded correction feedback, wherein the graded correction feedback is used to output a standard formula conforming to mathematical expression specifications and drive a voice broadcast unit to provide synchronous voice correction prompts.

[0008] Optionally, the formula error type patterns are classified into symbolic morphological errors, operator contradiction errors, grammatical logic errors, and expression structure errors; in the standardized image data, symbolic morphological errors and expression structure errors in the formula error type patterns are distinguished by contour similarity; at the same time, operator contradiction errors and grammatical logic errors in the formula error type patterns are identified based on the rule matching degree and semantic association matrix in the grammatical analysis results.

[0009] Optionally, based on the formula syntax topology, the standardized image data is decomposed into multiple image pyramid layers, and edge feature components related to symbol stroke contours are extracted; based on the expression structure error in the formula error type pattern, a comparative analysis is performed with the edge feature components; if the edge feature components show a symbol connection break length exceeding the break threshold and a sharp increase in structure fitting error within a preset recognition area, a dynamic correction mechanism for the syntax logic deviation coefficient corresponding to the expression is triggered.

[0010] Optionally, the symbol contour matching degree, operator structure compliance rate, and expression syntax adaptation value are used as input variables; based on the input variables, the structural feature matrix is ​​fused to update the support function for formula recognition correction; and the support function is used to perform correlation coefficient feedback verification on the correction confidence factor.

[0011] Optionally, a mapping relationship matrix between the support function and the input layer weights of the formula recognition correction model is set, and the influence weight vector of the correction confidence factor is decomposed and determined; when the correlation coefficient of the correction confidence factor is lower than the correlation coefficient threshold for Q consecutive recognition periods, a weight adaptive update mechanism is triggered, wherein the Q recognition periods are the empirically calibrated sliding window length.

[0012] Optionally, the learning rate can be dynamically adjusted based on changes in the handwriting input environment, employing a cosine annealing strategy: ,in, Let be the learning rate for the t-th iteration. and Here, denoted as minimum learning rate and maximum learning rate, respectively, and T represents the total number of iterations.

[0013] Optionally, a formula recognition and correction model under a bidirectional channel attention mechanism is set up; wherein, the first unidirectional channel uses a convolutional neural network to extract symbol morphological features of standardized image data, and constructs a morphological correction factor by combining contour matching threshold and symbol similarity, and uses the channel attention mechanism to adaptively adjust the weights of different symbol types to highlight easily confused character features; the second unidirectional channel converts the symbol sequence into a semantic vector through a Transformer encoder, and uses a spatial attention mechanism to identify the dependency structure rules between operators and expressions.

[0014] Optionally, a multi-scale feature fusion unit is set to perform tensor product operation on shallow symbol texture features and deep grammatical and semantic features; at the same time, the classification results of formula error level and error correction probability value are output through fully connected layers respectively. Combined with the user interaction environment change response mechanism, when the input device switching or handwriting style change is detected, the influence weight vector of the correction confidence factor and the correlation coefficient of the correction confidence factor exceed the normal range, and the incremental learning mechanism is automatically triggered to fine-tune the local recognition parameters.

[0015] A second aspect of this application provides an intelligent handwritten mathematical formula recognition and correction system, comprising: an image data acquisition module for acquiring raw image data of handwritten mathematical formulas, optimizing image quality through denoising and binarization preprocessing to obtain standardized image data; a coupling analysis module for performing coupling analysis based on the contour matching threshold corresponding to symbol strokes, the structural specification threshold corresponding to operators, and the grammatical logic deviation coefficient corresponding to expressions, combined with the formula grammatical topology, to formulate a structural feature matrix; an error type recognition module for identifying formula error type patterns based on the formula grammatical topology and the changing trend of the grammatical logic deviation coefficients over multiple recognition periods; a correction confidence factor configuration module for configuring a correction confidence factor based on the structural feature matrix, combined with the formula error type pattern and the standardized image data, wherein the correction confidence factor is strongly correlated with the input layer weights of the formula recognition and correction model; and a standard formula output module for using the correction confidence factor to determine the formula error level for graded correction feedback, wherein the graded correction feedback is used to output a standard formula conforming to mathematical expression specifications and drive a voice broadcast unit to provide synchronous voice correction prompts.

[0016] A third aspect of this application provides a computer-readable storage medium storing a computer program that, when executed, implements the steps of the above-described intelligent handwritten mathematical formula recognition and correction method.

[0017] One or more technical solutions provided in this application have at least the following technical effects or advantages: By collecting raw image data of handwritten mathematical formulas, and optimizing image quality through denoising and binarization preprocessing, standardized image data is obtained. Based on the contour matching threshold corresponding to symbol strokes, the structural specification threshold corresponding to operators, and the grammatical logic deviation coefficient corresponding to expressions, coupled analysis is performed using the formula grammatical topology to determine a structural feature matrix. Simultaneously, based on the formula grammatical topology and the changing trend of the grammatical logic deviation coefficient over multiple recognition cycles, formula error type patterns are identified. Based on the structural feature matrix, combined with the formula error type patterns and standardized image data, a correction confidence factor is configured. This correction confidence factor is strongly correlated with the input layer weights of the formula recognition correction model. Using the correction confidence factor, the formula error level is determined for graded correction feedback. This graded correction feedback outputs a standard formula conforming to mathematical expression specifications and drives a voice broadcast unit to provide synchronous voice correction prompts. This achieves a complete closed loop from image acquisition to intelligent correction of handwritten mathematical formulas, thereby improving the technical effect of handwritten mathematical formula recognition accuracy, stability, and correction precision.

[0018] The above description is merely an overview of the technical solution of this application. To better understand the technical means of this application and to facilitate its implementation according to the description, and to make the above and other objects, features, and advantages of this application more apparent, specific embodiments of this application are described below. It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent through the following description. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0020] Figure 1 A flowchart illustrating the intelligent handwritten mathematical formula recognition and correction method provided in this application.

[0021] Figure 2A schematic diagram of the structure of the intelligent handwritten mathematical formula recognition and correction system provided in this application.

[0022] Figure labeling: Image data acquisition module 11, coupling analysis module 12, error type identification module 13, correction confidence factor configuration module 14, standard formula output module 15. Detailed Implementation

[0023] This application provides an intelligent handwritten mathematical formula recognition and correction method, system, and medium to address the technical problems of unstable handwritten mathematical formula recognition and difficulty in guaranteeing recognition and correction accuracy in existing technologies, leading to errors or inconsistencies in the recognition results. It achieves a complete closed loop from image acquisition to intelligent correction of handwritten mathematical formulas, thereby improving the technical effect of increasing the accuracy, stability, and correction precision of handwritten mathematical formula recognition.

[0024] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. It should be understood that the present invention is not limited to the exemplary embodiments described herein. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention. It should also be noted that, for ease of description, only the parts related to the present invention are shown in the accompanying drawings, not all of them.

[0025] Example 1, as Figure 1 As shown, this application provides an intelligent handwritten mathematical formula recognition and correction method, which includes: The original image data of handwritten mathematical formulas were collected, and the image quality was optimized by denoising and binarization preprocessing to obtain standardized image data.

[0026] Specifically, the process involves directly acquiring raw image information containing handwritten mathematical formulas through various input devices, such as handwriting tablets and cameras, without any processing. The raw image data is then preprocessed, including denoising and binarization to optimize image quality. Denoising techniques such as mean filtering and median filtering are used to denoise the raw image data. For example, mean filtering calculates the average value of the pixels surrounding each pixel in the raw image data and replaces that pixel's value, thus smoothing the image and reducing the impact of noise. If the acquired image of the handwritten mathematical formula contains isolated noise points, after mean filtering, the pixel values ​​around these noise points are averaged, reducing the impact of the noise and making the raw image data clearer.

[0027] After denoising, binarization is performed. The pixel values ​​in the denoised image are converted to only two values, typically 0 and 255, representing black and white respectively, according to a set threshold, enhancing the contrast between the symbols and the background. Binarization clearly separates important elements such as symbols, operators, and numbers from the background, reducing background interference and highlighting the essential characteristics of the mathematical formula. For example, with a threshold of 128, in the denoised image, pixel values ​​greater than 128 are set to 255 (white), representing the background, while pixel values ​​less than or equal to 128 are set to 0 (black), representing the handwritten mathematical formula. After denoising and binarization preprocessing, standardized image data is obtained. This standardized image data has good quality and can well reflect the structural features of the handwritten formula, containing clear symbol strokes, operators, and formula structure information.

[0028] By optimizing image quality, irrelevant noise is removed and important features are enhanced, thereby improving the accuracy of subsequent recognition and analysis. The obtained standardized image data provides a clear and stable foundation for the extraction of structural feature matrices, making subsequent formula error type pattern recognition more efficient and accurate.

[0029] Based on the contour matching threshold corresponding to the symbol stroke, the structural specification threshold corresponding to the operator, and the grammatical logic deviation coefficient corresponding to the expression, coupled analysis is performed in conjunction with the formula grammatical topology to formulate the structural feature matrix.

[0030] Specifically, symbol strokes are the basic units in formula recognition, representing combinations of lines and strokes in handwritten mathematical symbols. The contour matching threshold measures the similarity between the contour of each symbol's stroke in a handwritten mathematical formula and a preset standard contour. Different symbols have different standard contours; for example, the standard contour of the number 8 consists of two nested elliptical shapes. When the similarity between the handwritten 8 contour and the standard contour reaches or exceeds the contour matching threshold, the symbol's stroke contour is considered to meet the requirements through this matching process. If the matching degree between the symbol's stroke contour and the standard symbol is lower than the contour matching threshold, the symbol is considered to potentially contain errors and requires further correction. By using the contour matching threshold corresponding to the symbol's strokes, it is possible to determine whether the symbol is correct or whether there are problems such as broken strokes or distorted shapes. For operators in mathematical formulas, such as the plus sign "+" and the minus sign "-", a structural standardization threshold is specified. This threshold defines a compliant range for combinations of operators and symbols, including the positional relationship of the operator relative to adjacent symbols (e.g., horizontal or vertical spacing), the alignment of symbols and operators, and the relative order of operators in the expression (e.g., the plus sign should be between two terms). This is used to assess whether the arrangement of symbols and operators conforms to the general rules of mathematical expressions and whether there are problems such as operator misalignment, repetition, or missing operators. For the overall expression structure of the formula, the overall structure is analyzed based on the syntactic-logical deviation coefficient. The syntactic-logical deviation coefficient reflects the logical connections and the standardization of symbol arrangement in the formula. For example, in a simple linear equation with two variables, if the order of terms and the use of symbols do not conform to mathematical grammar rules, the corresponding syntactic-logical deviation coefficient can be calculated. The formula's syntactic topology describes the logical structure and spatial arrangement relationships between various symbols, operators, and expressions in a mathematical formula. For example, in a multiplication expression, the order of the multiplicand, the multiplication sign, and the multiplier, as well as their relative positions on the plane, constitute the syntactic topology of the multiplication expression.

[0031] This paper couples the contour matching threshold, structural specification threshold, and grammatical logic deviation coefficient with the formula grammatical topology. First, for each symbol in the standardized image data, the stroke contours of the symbol are extracted using an edge detection algorithm, such as the Canny operator. The extracted stroke contours are then compared with preset standard contours. Common similarity calculation methods include Hausdorff distance and shape context matching. For example, the Hausdorff distance formula is used to calculate the maximum and shortest distance d between the symbol contour point set A and the standard contour point set B. H For coordinates (A, B), the smaller the distance, the more similar the symbol contour is to the standard contour. The symbol contour matching degree can be defined as: S = 1 - d H (A,B) / d max , where dmax The maximum contour distance of the symbol is denoted by 0, and the matching degree is between 0 and 1. The calculated symbol contour matching degree is compared with the contour matching threshold. When the symbol contour matching degree is greater than or equal to the contour matching threshold, the symbol contour is judged to be qualified.

[0032] For each operator, its structure is checked for compliance with the specification, and the deviation from the specification threshold is calculated to obtain the operator structure compliance rate. For example, the operator position (xi, yi) and the positions of adjacent symbols (xi-1, yi-1) and (xi+1, yi+1) are extracted, and the actual distances Δx and Δy between the operator and adjacent symbols are calculated. These are compared with the specification range to obtain the deviation D. The operator structure compliance rate is defined as R = 1 - D. For the entire expression, its syntax-logic deviation coefficient is calculated according to mathematical syntax rules. The syntax-logic deviation coefficient is used to measure whether the entire expression conforms to mathematical syntax rules. For example, for a linear equation in two variables ax + b = 0, the grammatical logic includes: whether there are equal signs on both sides, whether the order of coefficients, variables, and operators is correct, and whether the use of parentheses is reasonable. The specific calculation method is as follows: map the expression to a sequence of symbols, define a matrix of legal adjacency relationships of symbols and record the set of symbols allowed to appear after each symbol, check the symbol sequence pair by pair, and calculate the number of violations for each symbol. The number of violations is calculated as follows: if the arrangement between two adjacent symbols conforms to the grammatical rules, it is not counted as a violation; if they do not conform to the prescribed rules, such as incorrect symbol positions or disordered relationships between symbols, it is counted as one violation. The grammatical logic deviation coefficient is obtained by calculating the ratio between the number of violations of all symbol pairs and the total number of symbol pairs. The more violations there are, the higher the grammatical logic deviation coefficient is, and the more serious the grammatical problem of the formula is.

[0033] By combining formula syntax topology, the mutual influence and correlation between symbol contour matching degree, operator structure compliance rate, and syntax logic deviation coefficient are analyzed, and a structural feature matrix is ​​constructed. This structural feature matrix reflects the structural and logical relationships between symbols, operators, and expressions. For example, one row of the structural feature matrix can correspond to one symbol, and different columns of that row represent information such as the contour matching degree of that symbol and its topological relationship with adjacent operators. One column of the matrix can correspond to a feature type; for example, the contour matching degree information of all symbols can form one column.

[0034] By integrating feature information from multiple levels—symbols, operators, and expressions—a comprehensive structural feature matrix is ​​constructed, enabling more accurate identification of error types in formulas, such as symbolic errors, operator contradictions, or syntactic / logical errors. Simultaneously, the structural feature matrix provides crucial information for configuring correction confidence factors, contributing to improved accuracy and reliability in formula identification and correction.

[0035] Simultaneously, based on the formula syntax topology relationship and the changing trend of the syntax logic deviation coefficient over multiple recognition periods, formula error type patterns are identified.

[0036] Furthermore, the formula error type patterns are classified into symbolic morphological errors, operator contradiction errors, grammatical logic errors, and expression structure errors. In the standardized image data, symbolic morphological errors and expression structure errors in the formula error type patterns are distinguished by contour similarity. At the same time, operator contradiction errors and grammatical logic errors in the formula error type patterns are identified based on the rule matching degree and semantic association matrix in the grammatical analysis results.

[0037] Specifically, within each recognition cycle, based on pre-set mathematical grammar rules, the grammatical topology is analyzed to determine whether the logical and spatial arrangement of elements such as symbols, operators, and parentheses in the formula conforms to the specifications. At the same time, the grammatical logic deviation coefficient of the formula within that cycle is accurately calculated. The grammatical logic deviation coefficients calculated within multiple recognition cycles are dynamically tracked, and their trend curves are plotted to identify formula error type patterns. The formula error type patterns include: symbol form errors, operator contradiction errors, grammatical logic errors, and expression structure errors.

[0038] Symbol morphology errors refer to problems with the strokes and shape of handwritten symbols, such as writing the number 3 as 8, or an incorrect plus sign "+". Errors in symbol morphology are detected through contour similarity comparison; a low contour match between a symbol and a standard template indicates a symbol morphology error. Operator contradiction errors typically occur when operators are used improperly or in the wrong position within a formula, such as a plus and minus sign appearing in the same position, or a formula lacking a necessary operator. The correctness of operators can be accurately determined through rule matching and semantic association matrices in the syntax analysis results, and logical comparisons can be made with other parts of the formula. For example, by analyzing the logical relationships between symbols, operator contradiction errors are identified if there are no matching terms before or after an operator, or if the operator usage does not conform to conventional rules. Syntactic logic errors are related to the overall syntactic structure of the formula and the compliance of symbol arrangement. For example, an equation may lack symbols, or parentheses may not be correctly matched. Identifying syntactic logic errors relies on formula structural analysis and the syntactic dependencies between symbols; this type of error can be effectively identified through syntax analysis and semantic association matrices. Expression structure errors refer to anomalies in the overall structure of the formula, such as mismatched parentheses or misplaced superscripts and subscripts. By analyzing the formula structure and comparing the geometric features of the symbols, such as their relative positions and the positions of superscripts and subscripts, errors in the expression structure can be effectively identified.

[0039] By performing hierarchical analysis of the symbolic form, operator structure, syntactic logic, and expression structure of handwritten data formulas, this system can not only identify simple errors but also handle various potential error types in complex mathematical expressions. This ensures high accuracy and reliability in the entire mathematical formula recognition process and provides a clear direction and basis for subsequent error correction. For example, in cases of symbolic form errors and expression structure errors, it can accurately pinpoint which part of the symbolic form or formula structure needs correction. In cases of syntactic logic errors and operator contradictions, it can effectively deduce structural problems in the formula, thereby providing users with a more intelligent mathematical formula recognition and error correction experience.

[0040] Furthermore, the method also includes: decomposing standardized image data into multiple image pyramid layers based on the formula syntax topology relationship, and extracting edge feature components related to symbol stroke contours; comparing and analyzing the expression structure errors in the formula error type pattern with the edge feature components; if the edge feature components show a symbol connection break length exceeding the break threshold and a sharp increase in structure fitting error within a preset recognition area, then a dynamic correction mechanism for the syntax logic deviation coefficient corresponding to the expression is triggered.

[0041] Specifically, the image pyramid layer is a multi-scale image representation method. By downsampling a standardized image multiple times, a series of image pyramid layers with gradually decreasing resolution are obtained. Specifically, the range of key regions is determined according to the topological relationship of the formula syntax. For example, for a formula containing fractions, the approximate regions where the numerator, denominator, and fraction bar are located are identified. Taking a standardized handwritten mathematical formula image with a resolution of 800×600 pixels as an example, assuming that there is a fraction in the formula, the analysis determines that the numerator region is approximately in the range of 200-400 rows and 300-500 columns above the image, the denominator region is in the range of 400-600 rows and 300-500 columns below the image, and the fraction bar spans 300-500 columns and is approximately at the 400th row. Next, a Gaussian pyramid downsampling method is used to generate the next layer image by starting with the normalized image (layer 0, resolution 800×600) through Gaussian blur and downsampling operations. Each time the image is downsampled, the length and width of the image are halved. For example, the resolution of the first layer image becomes 400×300, the second layer becomes 200×150, and so on, generating 4-5 layer image pyramid images. Each layer image contains information at different scales, from the high-resolution detail information at the bottom layer to the large-scale contour information at the top layer.

[0042] Edge feature components related to symbol stroke contours are extracted from multiple image pyramid layers. These edge feature components refer to feature information in the image that reflects the contour direction and connection of symbol strokes, and can be obtained through edge detection algorithms, such as the Canny edge detection algorithm. Based on pre-identified formula error type patterns, expression structure errors are compared and analyzed with the extracted edge feature components. During the comparison and analysis, a preset identification region is set. This preset identification region is a key area where errors may exist, determined based on the formula syntax topology. If, within the preset identification region, the edge feature component shows a symbol connection break length exceeding a break threshold and a sharp increase in structure fitting error, a dynamic correction mechanism for the syntax logic deviation coefficient corresponding to the expression is triggered. The symbol connection break length refers to the length of the broken portion when a break occurs between symbol strokes. The structure fitting error is based on the geometric features and positional relationships between symbols. When an expression structure error exists, a fitting algorithm, such as least squares or RANSAC, is used to fit the edges of the symbols, calculating the error between the actual symbol contour and the expected structure. The structure fitting error refers to the distance or difference value between the symbol contour and the ideal structure calculated during the symbol connection process. If there is a significant deviation between the edge of a symbol and the expected structure—that is, a mismatch in the contour or a broken connection—a large fitting error will occur. For example, if the strokes of the originally continuous number 8 break in a certain area, with the break length exceeding a pre-set threshold of 2 pixels, the fitting error between the edge feature of this number and the structure of the standard 8 increases from 0.1 to 0.5. In this case, the dynamic correction mechanism for the syntax-logic deviation coefficient of the expression is triggered. Once triggered, the syntax-logic deviation coefficient is automatically adjusted. This coefficient quantifies structural errors and logical inconsistencies in the formula, and it changes dynamically as errors are corrected. The dynamic correction mechanism adjusts the syntax-logic deviation coefficient in real time based on anomalies in the edge feature components. For example, if the original syntax-logic deviation coefficient was 0.2, it might be corrected to 0.4 due to the detection of the aforementioned anomalies in the edge feature components, to more accurately reflect the current degree of error in the formula.

[0043] Through multi-level analysis of image pyramids and edge feature components, a more comprehensive understanding of the symbols and expression structures in formulas can be achieved. This is particularly effective for complex mathematical formulas, enabling the detection of subtle errors. When the breaks in the connections between symbols exceed a threshold, anomalies in the formula structure can be quickly identified, and errors can be corrected by adjusting the grammatical logic deviation coefficient. This not only improves the accuracy of error identification but also ensures that the mathematical formula adheres to the correct structure and logic throughout the entire identification process, making the error correction process more intelligent and efficient.

[0044] Based on the structural feature matrix, combined with the formula error type pattern and standardized image data, a correction confidence factor is configured, which is strongly correlated with the input layer weights of the formula recognition and correction model.

[0045] Specifically, the structural feature matrix is ​​quantified by calculating the similarity between the geometric shape of the symbol and the standard shape, obtaining a similarity value of 0-1. The operator arrangement is quantified as the deviation from the standard arrangement, calculated by normalizing the distance between the actual position and the standard position to the 0-1 range. For the formula syntax structure, its completeness and rationality are checked according to grammatical rules, and a score of 0-1 is given. Different weights are assigned based on the formula error type patterns, such as symbol morphological errors, operator contradiction errors, grammatical logic errors, and expression structure errors. The values ​​of each item in the quantified structural feature matrix are multiplied by their corresponding weights and then summed. A second weighted adjustment is then performed, considering the impact of each type of error on the overall error. A weighted average fusion algorithm is used to derive a comprehensive correction confidence factor. For example, the structural features of the currently identified handwritten formula fragment are quantified into three core indicators: symbol morphological similarity (Sc), operator arrangement regularity (So), and grammatical structure completeness (Sg), which, after normalization, are 0.82, 0.75, and 0.90, respectively. Based on the impact of error patterns, the weights for symbolic morphology errors (0.3), operator contradiction errors (0.4), and syntax / logic errors (0.3) are set. The quantified structural feature matrix values ​​are multiplied by their corresponding weights and then summed to calculate the basic structure score: 0.82 × 0.3 + 0.75 × 0.4 + 0.90 × 0.3 = 0.816. Then, a secondary weighting adjustment is performed, considering the impact of each error type on the overall error. Using normalized feature data, the formula is determined to have the lowest operator arrangement regularity, indicating an operator contradiction error with a weight of 0.9. A secondary weighting adjustment is then performed, resulting in a correction confidence factor of 0.9 × 0.816 = 0.7344, thus quantifying the correction confidence factor. Furthermore, the correction confidence factor is strongly correlated with the input layer weights of the formula recognition and correction model.

[0046] By configuring a correction confidence factor, it can be ensured that when correcting formula errors, appropriate measures are taken according to the reliability of the error, avoiding over-correction or under-correction.

[0047] Furthermore, based on the structural feature matrix, combined with the formula error type pattern and standardized image data, a correction confidence factor is configured. The method further includes: using symbol contour matching degree, operator structure compliance rate, and expression syntax adaptation value as input variables; based on the input variables, fusing the structural feature matrix, and updating the support function for formula recognition correction; and using the support function to perform correlation coefficient feedback verification on the correction confidence factor.

[0048] Specifically, the symbol contour matching degree, operator structure compliance rate, and expression syntax fit value are used as input variables. The symbol contour matching degree is calculated by contour comparison method to measure the similarity between the symbol and the standard template, and measures the accuracy of the symbol form. The operator structure compliance rate is calculated based on the relative position and arrangement rules between the symbol and the operator to determine whether the operator conforms to the mathematical expression standard. The expression syntax fit value is used to evaluate the degree of fit of the entire mathematical expression in terms of syntax structure with the standard syntax. According to the syntax rules, the order of symbols, bracket matching, subscript and superscript relationships in the expression are checked and scored. The higher the fit value, the more correct the expression syntax.

[0049] These three input variables are fused with the structural feature matrix, which contains comprehensive information such as the geometric shape of symbols, the arrangement of operators, and the grammatical structure of formulas. During fusion, different weights are assigned based on the degree of correlation between each input variable and related elements in the structural feature matrix. For example, the symbol contour matching degree is closely related to the symbol geometric shape part of the structural feature matrix and can be given a higher weight; the operator structure compliance rate corresponds to the operator arrangement specification part of the structural feature matrix and is given a corresponding weight; and the expression syntax fit value is related to the syntax structure part of the entire structural feature matrix. By using a weighted summation method, the information of the input variables is integrated into the structural feature matrix, thereby updating the support function for formula recognition and correction. The support function reflects the degree of support for formula recognition and correction under different combinations of input variables and structural features. Essentially, it maps symbol matching, operation structure, and syntax consistency to a comprehensive evaluation value within the interval [0,1], that is, the support function Fs(S)=σ(ws) ⊤ S+bs), where ws=[wc,wo,wg] is the feature weight vector, and S=[Sc,So,Sg] ⊤ bs is the bias term, which is the minimum value, such as 0.01. σ(⋅) is the Sigmoid function. The support function Fs(S)∈[0,1] represents the overall recognition reliability of the current formula.

[0050] Then, the updated support function is used to perform correlation coefficient feedback verification on the corrected confidence factor. The correlation coefficient measures the degree of linear correlation between the support function and the corrected confidence factor. The correlation coefficient between the support function and the corrected confidence factor is calculated using the Pearson correlation coefficient to determine whether the influence of the support function on the corrected confidence factor is reasonable. If the correlation coefficient is within the expected range, such as greater than 0.8, it indicates that the support function can reflect the changes in the corrected confidence factor well. If the correlation coefficient deviates significantly, the support function needs to be adjusted and optimized to ensure the accuracy and reliability of the corrected confidence factor.

[0051] By combining multiple input variables and structural feature matrices, the corrective confidence factor is accurately calculated and dynamically adjusted, which not only improves the accuracy of the recognition process but also enhances its robustness in complex formula error correction. This enables it to provide accurate error correction functions under different handwriting styles and complex formula scenarios, ultimately achieving more efficient and intelligent mathematical formula recognition and correction.

[0052] Furthermore, the corrected confidence factor is strongly correlated with the input layer weights of the formula recognition correction model. The method includes: setting a mapping relationship matrix between the support function and the input layer weights of the formula recognition correction model, and decomposing and determining the influence weight vector of the corrected confidence factor; when the correlation coefficient of the corrected confidence factor is lower than the correlation coefficient threshold for Q consecutive recognition periods, a weight adaptive update mechanism is triggered, wherein the Q recognition periods are the empirically calibrated sliding window length.

[0053] Specifically, the strong correlation between the correction confidence factor and the input layer weights of the formula recognition and correction model is reflected by setting a mapping matrix between the support function and the input layer weights. The goal is to transform the calculation result of the support function into the weights of the input layer of the formula recognition and correction model, optimize the weight allocation of the model, and thus improve its ability to identify and correct error types. First, a mapping matrix is ​​constructed, defining the mapping relationship between the support function and the input layer weights of the formula recognition and correction model. The support function quantifies the recognition reliability of each part of the formula, such as the matching degree of symbols and the compliance of operators, while the input layer weights determine the importance of each feature in the recognition and correction process.

[0054] In constructing the mapping matrix, the input features of the support function are first determined. These features include symbol matching degree, operator structure compliance rate, and expression syntax fit value, which quantify the correctness of symbol form, operator position, and the fit of the formula's syntax structure, respectively. Based on the structure of the formula recognition model, the weights of the input layer are defined. These weights reflect the degree of attention the formula recognition model pays to different features when correcting formulas. The construction process of the mapping matrix maps each feature in the support function to each weight in the input layer, constructing a matrix where the number of rows equals the number of features in the support function, the number of columns equals the number of weights in the input layer, and each element in the matrix represents the degree of influence of a certain feature on a certain weight. Thus, the mapping matrix Mij∈RT m×n , where m is the number of support function features, n is the number of weights in the input layer, and Mij represents the influence coefficient of support feature i on the j-th input weight.

[0055] To determine the mapping relationship, a combination of empirical calibration and training is used. For empirical calibration, domain experts initially set the values ​​of the mapping relationship matrix elements based on their understanding of the support function features and input layer weights. For example, if experts believe that symbol matching degree has a significant impact on input layer weights related to symbol shape, then the corresponding matrix elements can be assigned larger initial values. During training, feedback from backpropagation is used to further adjust the values ​​of the mapping relationship matrix elements. In each training iteration, the formula recognition correction model makes predictions based on the input data and current weights, calculates the error between the prediction result and the true label, and propagates the error back to the input layer using the backpropagation algorithm to obtain the contribution of each input layer weight to the error. Simultaneously, the performance of each feature of the support function in this training is analyzed, and the impact of each feature on the formula recognition model's decision is quantified based on the correlation between the feature and the error. For example, if it is found that symbol matching degree plays a significant role in reducing the error in this training, then the values ​​of the mapping relationship matrix elements related to symbol matching degree can be appropriately increased. By continuously updating the values ​​of these elements during training and optimizing the mapping matrix, we can ensure that the influence of each feature in the support function on the input layer weights is reasonable and effective, thereby ultimately improving the accuracy of formula recognition and correction.

[0056] By mapping the relationship matrix, each feature in the support function is mapped to the weights of the input layer of the formula recognition and correction model. This allows the model to adjust its weight distribution based on specific recognition confidence levels, such as symbol matching and operator compliance, during subsequent training and inference, thereby improving the accuracy of the correction results. An influence weight vector is generated based on the calculated correction confidence factor. This influence weight vector, calculated using the mapping relationship matrix, quantifies the degree of influence of each recognition feature, such as symbol matching and operator compliance, on the final correction decision. The influence weight vector is: V = M ⊤ S, where each component represents the confidence enhancement strength of different input feature channels. The weight vector will be input into the formula recognition and correction model to adjust the input layer weights of the model, enabling it to be optimized for specific recognition tasks and accurately identify errors in the formula.

[0057] To ensure the stability of the correction model across different recognition periods, an adaptive weight update mechanism is introduced. This mechanism dynamically adjusts the weights based on the correlation coefficient of the correction confidence factor. When the correlation coefficient of the correction confidence factor falls below a set correlation coefficient threshold for Q consecutive recognition periods, it indicates that the correction factor has little impact on formula recognition, triggering the adaptive update mechanism. Q recognition periods are an empirically calibrated sliding window length, representing a fixed time or iteration range. By observing the changes in the correlation coefficient within this window, it's possible to more stably and accurately determine whether weight updates are needed. Q is an integer greater than 1, and the correlation coefficient threshold can be dynamically set according to actual needs. For example, if Q is set to 5 recognition periods, and the correlation coefficient is below the threshold for 5 consecutive periods, it indicates an anomaly in the relationship between the current support function and the correction confidence factor, potentially failing to accurately reflect the formula recognition correction. Updating the input layer weights based on Q periods allows the formula recognition correction model to adapt to changes in the current input, avoiding erroneous correction results under low confidence conditions.

[0058] By setting a mapping matrix and determining the influencing weight vector, a precise relationship is established between the correction confidence factor and the input layer weights. This allows the formula recognition and correction model to dynamically adjust the input layer weights based on changes in the correction confidence factor, thereby improving the accuracy and adaptability of the model in formula recognition and correction. The adaptive weight update mechanism enables timely adjustments when the model's performance becomes abnormal, preventing a decline in recognition effectiveness due to the long-term use of unreasonable weights and ensuring the model's stability and reliability.

[0059] Furthermore, the learning rate is dynamically adjusted based on changes in the handwriting input environment, employing a cosine annealing strategy: ,in, Let be the learning rate for the t-th iteration. and Here, denoted as minimum learning rate and maximum learning rate, respectively, and T represents the total number of iterations.

[0060] Specifically, to address variations in handwriting input environments, the training process of the formula recognition and correction model is optimized by dynamically adjusting the learning rate, employing a cosine annealing strategy to balance the learning rate. The learning rate determines the pace at which the formula recognition and correction model updates its parameters in each iteration; appropriately adjusting the learning rate accelerates the training process and avoids overfitting or undertraining. The core idea of ​​the cosine annealing strategy is to gradually decrease the learning rate as training progresses, enabling the formula recognition and correction model to converge quickly in the early stages and allowing for more precise weight adjustments.

[0061] The learning rate is adjusted according to the following formula: ,in, Let be the learning rate for the t-th iteration. and Here, denoted by the minimum and maximum learning rates, and T represents the total number of iterations. The annealing process makes the learning rate decrease as training progresses, which helps the model identify and correct itself to find the optimal solution more stably as it approaches convergence, and reduces over-adjustment in the later stages of training.

[0062] By dynamically adjusting the learning rate, the formula recognition and correction model can quickly explore the parameter space in the initial stage and finely adjust it in the later stages of training, avoiding skipping the optimal solution. Furthermore, the cosine annealing strategy avoids the abrupt changes that traditional methods, such as linear decay, may cause when adjusting the learning rate, providing a smooth and effective learning process. Through this dynamic adjustment strategy, it can better adapt to changes in training data under different handwriting input environments, improving recognition accuracy and maintaining good generalization ability across diverse handwriting styles, thereby enhancing the performance of formula recognition and correction.

[0063] Furthermore, the method also includes: setting up a formula recognition and correction model under a bidirectional channel attention mechanism; wherein, the first unidirectional channel uses a convolutional neural network to extract symbol morphological features of standardized image data, combines contour matching threshold and symbol similarity to construct a morphological correction factor, and uses the channel attention mechanism to adaptively adjust the weights of different symbol types to highlight easily confused character features; the second unidirectional channel converts symbol sequences into semantic vectors through a Transformer encoder, and uses a spatial attention mechanism to identify the dependency structure rules between operators and expressions.

[0064] Specifically, to improve the accuracy of recognition and correction, the formula recognition and correction model employs a bidirectional channel attention mechanism. This mechanism uses two independent unidirectional channels working in parallel to analyze and correct the symbol morphology and grammatical structure of formulas at different levels. The first unidirectional channel uses a convolutional neural network (CNN) to extract symbol morphological features from standardized image data. The CNN automatically learns the basic shape, strokes, and contours of symbols from images through a series of convolutional and pooling layers. Each convolutional layer contains several convolutional kernels, which perform convolution operations with local regions of the input image, outputting the convolution result as a feature map. The core of the convolution operation is to use the convolutional kernel to perform a sliding window operation on local regions of the input image, thereby extracting features of that region, such as shape information like edges, corners, and curves. As the network deepens, the convolutional layers can gradually extract more abstract and high-level features, such as details like the stroke direction and thickness variations of symbols.

[0065] Each convolutional layer extracts a feature map, which is then processed by pooling layers. Pooling layers perform operations such as max pooling or average pooling to downsample the feature maps output by the convolutional layers, reducing the size of the feature maps, lowering computational complexity, and preserving important spatial information. For example, max pooling selects the maximum value in the feature map to represent the features of a local region, effectively compressing the data while retaining key information. The purpose of pooling layers is to reduce the number of parameters and computational cost, avoid overfitting, and enhance the model's translation invariance, meaning that the symbol's shape can still be recognized even when its position changes in the image. After several convolutional and pooling operations, the resulting symbol feature map is further transformed into a high-dimensional feature vector through fully connected layers. This high-dimensional feature vector accurately represents the symbol's morphological features, including the direction of strokes, the curvature of curves, and the symbol's structure and contour. Based on this, a morphological correction factor is generated by combining a contour matching threshold and symbol similarity. This morphological correction factor quantifies the degree of deviation in symbol morphology. For example, when recognizing the number "1" and the letter "I", the formula recognition correction model determines whether confusion exists by comparing contour similarity and assigns higher attention weights to such symbols. The attention mechanism further adaptively adjusts the weights of different symbol types, highlighting easily confused symbols and ensuring accurate differentiation and processing of these symbols during correction.

[0066] The second unidirectional channel combines a Transformer encoder with a spatial self-attention mechanism to process symbol sequences and extract syntactic and logical relationships between symbols, thereby achieving accurate analysis and error correction of formula structures. First, it acquires the symbol sequences from handwritten mathematical formulas, including numbers, letters, operators, and special symbols. These handwritten symbols are then converted into machine-recognizable discrete symbols using OCR (Optical Character Recognition) or other technologies. For example, it performs connected component detection on the feature map, scanning foreground pixels in the image using an eight-neighbor connectivity labeling algorithm. Spatially continuous stroke regions are labeled as independent connected components, and the area, bounding rectangle, aspect ratio, centroid coordinates, and stroke density of each connected component are calculated. False connected regions with areas below the noise threshold are filtered out. For regions with overlapping strokes or multiple symbols in contact, further refinement is achieved by combining horizontal and vertical projection segmentation methods. Specifically, by statistically analyzing the pixel projection distribution in the X and Y directions, local troughs in the projection curves are detected as candidate segmentation boundaries. Overlapping regions are then segmented based on a stroke spacing threshold and a minimum character width constraint to obtain corresponding candidate symbol regions. The stroke spacing threshold is set to 0.15–0.30 times the average character width. If the average character width in the current formula is 40 pixels, the stroke spacing threshold is set to 6–12 pixels. The minimum character width constraint is set to 0.35–0.50 times the average character width. For example, if the average character width is 40 pixels, the minimum character width constraint can be set to 14–20 pixels. If the bounding box width of any sub-region after segmentation is less than this minimum character width constraint, the segmentation operation is canceled, and the original connected region is preserved.

[0067] Then, a corresponding bounding box is generated for each candidate symbol region, and the center coordinates, width and height dimensions, aspect ratio, and baseline offset relative to the main writing line of the formula are extracted. The main writing line is obtained by statistically analyzing the vertical distribution density of the centroids of all symbols in the line, serving as a reference for determining the hierarchical relationship of the formula. Further, using the bounding boxes of each candidate symbol as nodes, spatial adjacency edges are established based on the horizontal distance, vertical distance, overlap ratio, and relative size relationship between nodes, constructing a two-dimensional layout relationship diagram. The edge weights between adjacent nodes are calculated using the Euclidean distance between symbols, used to characterize the spatial association strength between symbols. Then, the candidate symbols are structurally analyzed according to the spatial arrangement rules of mathematical expression, including: when a connected region with a length exceeding a preset ratio threshold and a width-to-height ratio greater than the horizontal line determination threshold is detected, it is identified as a candidate fraction line. The preset ratio threshold is dynamically determined based on the average character width in the current formula line. When the length of the candidate connected region is greater than 1.5 to 2.5 times the average character width, it is determined to meet the fraction line length condition. The horizontal line determination threshold is determined based on the width-to-height ratio of the connected region. For example, when the width-to-height ratio of the connected region is greater than 8:1 to 15:1, it is identified as a horizontal line structure.

[0068] The symbol sets in the adjacent regions above and below each fraction are searched separately, and the numerator and denominator regions are determined by the region clustering results. When the center ordinate of a candidate symbol is higher than the center of the reference character and its size ratio is smaller than the preset ratio of the reference character, it is marked as a superscript candidate region. Conversely, when the center ordinate is lower than the baseline of the reference character and its size is smaller, it is marked as a subscript candidate region. The adjacent regions above and below can be defined as search ranges extending upward and downward by 0.5 to 1.5 times the average character height, respectively, with the center of the candidate fraction line as the reference. The numerator and denominator symbol sets are determined by a clustering rule that the distance between the centers of connected components is less than 1.2 times the average character spacing.

[0069] For square root structures, the corresponding region of the radix expression is determined by detecting connected contours with broken line corner features and combining their horizontal coverage. For large operators such as integral and summation symbols, they are identified based on the feature that the size of the operator's bounding box is significantly larger than the average size of its neighborhood, and the upper and lower limit parameter regions are determined based on the spatial distribution relationship of the symbols in its upper and lower neighboring regions. For nested bracket structures, hierarchical nesting relationships are established by detecting the regional inclusion relationship between the left and right brackets and the bracket height coverage. For matrix structures, based on the row and column projection distribution results of candidate symbols, clustering algorithms are used to perform row and column alignment analysis on the symbols to determine the number of rows, columns, and element correspondences of the matrix.

[0070] After completing the above two-dimensional structure analysis, each symbol node and its spatial hierarchy are mapped to an expression tree structure, where operators are parent nodes and operands are child nodes. A tree-like grammatical topology is recursively generated based on mathematical operation precedence and spatial hierarchy. Then, following the traversal rule of prioritizing the main expression and recursively expanding local substructures, the expression tree is linearized to generate a symbol sequence. Simultaneously, structural positional encoding information is added to each symbol, including baseline hierarchical encoding, subscript / superscript hierarchical encoding, fraction hierarchical index, and matrix row and column coordinate encoding, thus obtaining serialized input data that balances two-dimensional spatial structural information and symbolic semantic information. Multiple symbols are arranged according to their order in the formula and input to the Transformer encoder for further processing. The Transformer encoder, based on a self-attention mechanism, converts each symbol into a high-dimensional dense vector through embedding operations. The embedding vector of each symbol is obtained by looking up the embedding matrix. During the training of the embedding matrix, the semantic information of the symbols can be effectively captured. At the same time, positional encoding is added to the embedding vector of each symbol, ensuring that the formula recognition and correction model can perceive the position of each symbol in the sequence and avoid the loss of positional order information.

[0071] After obtaining the embedded representations of the symbols, the self-attention mechanism processes the input feature sequence. Through three sets of linear transformations, query vectors, key vectors, and value vectors are generated. Each symbol interacts through these vectors, calculating the similarity between the query vector and the key vector to obtain the dependencies and attention weights between symbols. By calculating the similarity and applying the softmax function, the resulting attention weights influence the weighted convergence of symbols, thus forming an enhanced representation of each symbol in its context. For example, in the formula "3+5", the self-attention mechanism not only makes the "+" symbol pay attention to the information of the operands "3" and "5", but also helps the formula recognition and correction model understand its syntactic role as an addition operator. In this way, the Transformer encoder can convert a sequence of symbols into a sequence of semantic vectors containing rich semantic information. This sequence of semantic vectors not only reflects the basic semantics of each symbol but also considers the combinatorial and operational relationships between symbols. Building upon the Transformer encoder, the spatial attention mechanism further analyzes the spatial layout between symbols. By calculating the spatial distance and positional relationships between symbols, an attention weight is assigned to each symbol to highlight the structural association features between symbols, helping the formula recognition and correction model capture the spatial dependencies between symbols. For example, in mathematical formulas, the positional relationships of symbols such as parentheses and operators are crucial. Spatial attention mechanisms can automatically adjust the level of attention given to these relationships to ensure that the structure of the formula is accurately captured. The output of the spatial attention mechanism can be fused with the original semantic vector sequence, for example, through concatenation or addition, to obtain a more comprehensive and accurate semantic representation.

[0072] In the training phase of the formula recognition and correction model, a training sample set is first constructed. The input of the training sample is serialized symbol data after two-dimensional structure parsing, including symbol embedding vector, position encoding vector and structural level label. At the same time, the corresponding supervision label is the canonical expression of the standard mathematical formula and its expression tree structure. The symbol level label is used to indicate the standard category and position of each input symbol, and the structural level label is used to indicate the subscript and superscript dependency relationship between symbols, the scope of operator, and the nesting hierarchy of brackets.

[0073] During training, the output of the formula recognition and correction model is divided into symbol classification results, structural relationship prediction results, and sequence reconstruction results, and corresponding loss functions are calculated for each. The symbol classification loss is calculated by the difference between the probability distribution of the symbol categories output by the model and the labeled symbol categories, and is used to constrain symbol recognition accuracy. It is determined using the standard cross-entropy function. The structural relationship loss is used to constrain the topological consistency of two-dimensional structures such as subscripts, superscripts, fractions, and brackets. It is calculated by aligning the predicted expression tree and the standard expression tree at the node level, calculating the subscript dependency error rate, the fraction upper and lower region mismatch rate, and the bracket nesting structure deviation rate, and quantified using structural edit distance or tree structure difference. The sequence reconstruction loss is used to constrain the consistency between the model output and the standard expression, and is obtained by calculating the bitwise cross-entropy or sequence-level negative log-likelihood between the predicted sequence and the target sequence. Then, the three loss functions are weighted to form a joint loss function for overall optimization. The weight coefficients are initially set empirically based on the performance of the validation set and are dynamically adjusted during training according to the contribution of each loss gradient to the reduction of the total loss. That is, when the reduction rate of a certain loss term is lower than a preset threshold, its weight ratio is increased to strengthen the corresponding ability learning, thereby achieving adaptive balance optimization between symbol recognition, structural modeling and sequence reconstruction capabilities.

[0074] For parameter updates, a gradient descent optimization method based on backpropagation is adopted to jointly update the parameters of the convolutional neural network, the Transformer encoder, and the attention mechanism end-to-end. At the same time, a cosine annealing learning rate strategy is introduced to dynamically adjust the update step size during training, enabling the formula recognition and correction model to converge quickly in the early stage of training and stabilize in the later stage. In terms of the error correction-driven mechanism, when the model prediction result is structurally inconsistent with the labeled expression tree, the error node is located, and the error is backpropagated to the corresponding symbol representation and structural hierarchical encoding. The attention weights of the relevant channels are strengthened and corrected, thereby realizing incremental parameter optimization based on structural error feedback. Ultimately, the model gradually learns the mapping relationship between symbol morphological features and two-dimensional grammatical structure.

[0075] The formula recognition and correction model employs a bidirectional channel attention mechanism, a convolutional neural network to extract symbol morphological features, and a Transformer encoder to capture the grammatical relationships between symbols. This provides a comprehensive and detailed data foundation for calculating the correction confidence factor. The correction confidence factor is then calculated using input variables such as symbol contour matching degree, operator structure compliance rate, and grammatical logic fit value. The mapping matrix between the support function and the input layer weights of the formula recognition and correction model is used to adjust the weights of the model, thereby optimizing the output of the correction confidence factor and ensuring dynamic weight updates during formula recognition, further improving the accuracy and reliability of formula correction.

[0076] Furthermore, the method also includes: setting up a multi-scale feature fusion unit to perform tensor product operation on shallow symbol texture features and deep grammatical semantic features; simultaneously, outputting the classification results of formula error level and error correction probability value through fully connected layers respectively; combined with the user interaction environment change response mechanism, when the input device switching or handwriting style change is detected, the influence weight vector of the correction confidence factor and the correlation coefficient of the correction confidence factor exceed the normal range, automatically triggering the incremental learning mechanism to fine-tune the local recognition parameters.

[0077] Specifically, a multi-scale feature fusion unit is set between the convolutional neural network and the Transformer encoder to fuse symbolic texture features and syntactic semantic features extracted from different levels, further improving the accuracy of formula recognition and error correction. The convolutional neural network extracts shallow symbolic texture features, such as the shape of strokes, curve variations, and symbol thickness, reflecting the visual information of the symbols. The Transformer encoder and other structures extract deep syntactic semantic features, capturing the logical and syntactic relationships between symbols, such as the dependency between operators and operands, and bracket matching. The multi-scale feature fusion unit receives two channel outputs and uses tensor product operations to fuse these two types of features, generating a comprehensive feature vector that integrates morphological and semantic information.

[0078] The fused feature vector is passed from the multi-scale feature fusion unit to the fully connected layer. Each node in the fully connected layer is connected to all nodes in the previous layer, comprehensively considering all information from the input features. During this process, the formula recognition and correction model learns the weight relationship between each feature and the final output through training. By applying an activation function to the output of the fully connected layer, using ReLU or Sigmoid, it is converted into a non-linear feature representation, enabling the formula recognition and correction model to better handle complex patterns and relationships. Finally, the output of the fully connected layer is divided into two parts: one part is the formula error level classification result normalized by the softmax activation function, representing the possible error types in the formula, such as symbol errors, structural errors, and syntax errors; the other part is the error correction probability value processed by the sigmoid function, representing the confidence level of the formula recognition and correction model in correcting each error type.

[0079] Meanwhile, a user interaction environment change response mechanism is introduced. In real-world application scenarios, the input device used by the user may change, such as switching from a tablet to a mobile phone. Differences in screen size and touch precision between different devices will affect the quality and characteristics of handwriting input. Furthermore, different users have significantly different handwriting styles. Even when using the same device, abrupt changes in handwriting style, such as when students and teachers alternate using the device in a classroom setting, can lead to changes in writing speed, tilt angle, and stroke continuity. The mechanism continuously tracks and analyzes input changes, calculating a correction confidence factor within each recognition cycle. This correction confidence factor is based on a comprehensive evaluation of multiple input variables, including symbol contour matching degree, operator structure compliance rate, and grammatical logic fit value.

[0080] Over time, the value of the correction confidence factor is affected by changes in input data, such as handwriting style and input device switching. Based on the relationship between input features and the output of the formula recognition correction model, the influence of each feature on the correction decision is learned using weights acquired during training, generating an influence weight vector for the correction confidence factor. A sliding window method is used to monitor the changing trend of the correction confidence factor across multiple recognition periods, and the correlation coefficient of the correction confidence factor is calculated, i.e., the degree of linear correlation between the confidence factor in the current period and the confidence factors in historical periods. The correlation coefficient reflects the impact of input changes in the current recognition period on the correction decision, helping the formula recognition correction model determine whether the recognition is stable or whether there are sudden changes in the input environment. The influence weight vector and correlation coefficient of the corrected confidence factor after the change are obtained to measure the influence of different factors on the correction confidence factor. When the influence weight vector and the correlation coefficient of the correction confidence factor exceed the normal range, the normal range is determined by analyzing the fluctuation range of the correlation coefficient in historical data, for example, by statistically analyzing the standard deviation, mean, and distribution range of the correlation coefficient in historical data, a dynamic normal range is set. If the current correlation coefficient value exceeds the set normal range, such as ±2 standard deviations, it indicates a significant change in the input data. In this case, an incremental learning mechanism is automatically triggered to adapt to the new input conditions and correct the formula recognition. Incremental learning is a learning method that allows fine-tuning of local parameters of a model without retraining the entire model. Through incremental learning, the model can quickly adapt to new input devices and handwriting styles, fine-tuning local recognition parameters, thereby improving the accuracy and stability of the formula recognition and correction model in different environments.

[0081] The multi-scale feature fusion unit provides more comprehensive and accurate feature information for the formula recognition and correction model by fusing shallow symbolic texture features with deep syntactic and semantic features, which helps improve the accuracy of formula error recognition and correction. The formula error level classification results and error correction probability values ​​output by the fully connected layer provide intuitive and detailed error information. The user interaction environment change response mechanism and incremental learning mechanism enable the formula recognition and correction model to adapt to different input devices and handwriting styles, enhance the generalization ability and practicality of the formula recognition and correction model, ensure that the formula recognition and correction model can maintain good performance in various real-world scenarios, improve the accuracy, stability and correction accuracy of handwritten mathematical formula recognition, and thus improve the overall effectiveness and user experience of the formula recognition and correction scheme.

[0082] Using the aforementioned correction confidence factor, the formula error level is determined for graded correction feedback. The graded correction feedback is used to output a standard formula that conforms to mathematical expression norms and drive the voice broadcasting unit to provide synchronous voice correction prompts.

[0083] Specifically, the correction confidence factor is divided into different level intervals based on a pre-set threshold range. For example, a high confidence interval of 0.8-1.0, a medium confidence interval of 0.5-0.8, and a low confidence interval of 0-0.5 are set. The candidate correction generation strategy is determined based on the level of the correction confidence factor. When the correction confidence factor is in the high confidence interval, it indicates that the correction of the formula error is quite accurate, and minor adjustments are made, such as adjusting the position of symbols, fixing minor formatting issues, or optimizing the formula layout. For example, when the correction confidence factor is in the high confidence interval, a limited number of candidate results are generated only for local low-bias areas, such as correcting missing strokes, slight offsets, or formatting errors of individual symbols. In this case, the original formula structure is kept unchanged, and only the top 2-3 candidate standard symbols are selected from the candidate set corresponding to the current erroneous symbol for replacement and verification based on contour matching degree, stroke direction similarity, and character similarity matrix. At this time, the formula recognition and correction model directly outputs the corrected standard formula, ensuring compliance with mathematical expression standards.

[0084] When the correction confidence factor is in the medium confidence interval, it indicates that the formula recognition and correction model has some confidence in recognizing and correcting formulas, but the correction results may still need further optimization and more detailed checks on the formulas. For example, when the correction confidence factor is in the medium confidence interval, in addition to symbol-level correction, candidate correction results for operators and local structures are further generated. For instance, by combining the spatial distance between adjacent symbols, operation priority rules, and syntactic dependencies, easily confused operators such as "×" and "x", and "-" and fraction lines are replaced and combined. At the same time, candidate expression trees are reconstructed for subscript and superscript positions, bracket closure relationships, and local fractional structures, and multiple local candidate formula structures are generated. When the correction confidence factor is in the low confidence interval, it indicates low confidence in correcting the formula, and there may be significant errors in the formula, or even a combination of multiple errors. In this case, more complex correction operations are performed, such as re-parsed the formula structure, corrected operator combinations, bracket matching, etc., or recalculated and adjusted some symbols or expressions in the formula to ensure the effectiveness and accuracy of handwritten mathematical formula recognition and correction. For example, when the correction confidence factor is in the low confidence interval, it is considered that the original recognition result has a large structural error. The fraction region, matrix region and nested bracket level are re-divided, and multiple global candidate formula structures are recursively generated based on the expression tree. At the same time, the existing mathematical syntax rule library is called to perform bracket matching, operation order, function parameter integrity and subscript / superscript dependency verification on the candidate formulas to achieve formula correction.

[0085] After formula correction and the generation of a standard formula, the voice broadcast unit provides synchronized voice correction prompts. This voice broadcast unit is a device or software module capable of converting text information into voice signals for output. By utilizing TTS (Text-to-Speech) technology, it converts text information into voice signals for playback. Specifically, based on the corrected standard formula, it conveys correction information to the user in a clear and accurate voice format, including pointing out the location and type of error in the original formula, as well as the correct expression. For example, the voice broadcast might indicate that the third symbol in the original formula is incorrectly written and should be changed to ×. This synchronized voice feedback of the formula correction process and result ensures that the user clearly understands the formula's correction and the basis for the error, thereby improving the interactive experience.

[0086] Through a tiered correction mechanism, the system can flexibly adjust its correction strategy based on different ranges of the correction confidence factor, improving the accuracy, stability, and correction precision of handwritten mathematical formula recognition, further ensuring the accuracy and practicality of formula recognition and correction. Simultaneously, synchronized voice feedback further enhances the intuitiveness of the interaction and the user experience.

[0087] Example 2, based on the same inventive concept as the intelligent handwritten mathematical formula recognition and correction method in the foregoing examples, such as... Figure 2As shown, this application provides an intelligent handwritten mathematical formula recognition and correction system, wherein the intelligent handwritten mathematical formula recognition and correction system includes: Image data acquisition module 11 is used to acquire the original image data of handwritten mathematical formulas, optimize the image quality through denoising and binarization preprocessing, and obtain standardized image data; coupling analysis module 12 is used to perform coupling analysis based on the contour matching threshold corresponding to the symbol stroke, the structural specification threshold corresponding to the operator, and the grammatical logic deviation coefficient corresponding to the expression, combined with the formula grammatical topology, to formulate a structural feature matrix; error type identification module 13 is used to identify the formula error type pattern based on the formula grammatical topology and the changing trend of the grammatical logic deviation coefficient in multiple recognition periods; correction confidence factor configuration module 14 is used to configure a correction confidence factor based on the structural feature matrix, combined with the formula error type pattern and standardized image data, wherein the correction confidence factor is strongly correlated with the input layer weight of the formula recognition correction model; standard formula output module 15 is used to use the correction confidence factor to determine the formula error level and perform graded correction feedback, wherein the graded correction feedback is used to output a standard formula that conforms to the mathematical expression specification and drive the voice broadcasting unit to perform voice synchronous correction prompts.

[0088] Furthermore, the error type identification module 13 is also used to: classify the formula error type pattern into symbol morphology errors, operator contradiction errors, grammatical logic errors, and expression structure errors; in the standardized image data, distinguish between symbol morphology errors and expression structure errors in the formula error type pattern by contour similarity; and simultaneously, identify operator contradiction errors and grammatical logic errors in the formula error type pattern based on the rule matching degree and semantic association matrix in the syntax analysis results.

[0089] Furthermore, the error type identification module 13 is also used to: decompose the standardized image data into multiple image pyramid layers based on the formula syntax topology relationship, and extract edge feature components related to the symbol stroke contours; compare and analyze the expression structure error in the formula error type pattern with the edge feature components; if the edge feature components have a symbol connection break length exceeding the break threshold and the structure fitting error increases sharply in the preset identification area, then trigger the dynamic correction mechanism of the syntax logic deviation coefficient corresponding to the expression.

[0090] Furthermore, the correction confidence factor configuration module 14 is also used to: use symbol contour matching degree, operator structure compliance rate, and expression syntax adaptation value as input variables; based on the input variables, fuse the structural feature matrix and update the support function for formula recognition correction; and use the support function to perform correlation coefficient feedback verification on the correction confidence factor.

[0091] Furthermore, the correction confidence factor configuration module 14 is also used to: set the mapping relationship matrix between the support function and the input layer weights of the formula recognition correction model, decompose and determine the influence weight vector of the correction confidence factor; when the correlation coefficient of the correction confidence factor is lower than the correlation coefficient threshold for Q consecutive recognition periods, trigger the weight adaptive update mechanism, wherein the Q recognition periods are the empirically calibrated sliding window length.

[0092] Furthermore, the correction confidence factor configuration module 14 is also used to: dynamically adjust the learning rate according to changes in the handwriting input environment, and adopt a cosine annealing strategy: ,in, Let be the learning rate for the t-th iteration. and Here, denoted as minimum learning rate and maximum learning rate, respectively, and T represents the total number of iterations.

[0093] Furthermore, the correction confidence factor configuration module 14 is also used to: set up a formula recognition correction model under a bidirectional channel attention mechanism; wherein, the first unidirectional channel uses a convolutional neural network to extract symbol morphological features of standardized image data, combines contour matching threshold and symbol similarity to construct a morphological correction factor, and uses the channel attention mechanism to adaptively adjust the weights of different symbol types to highlight easily confused character features; the second unidirectional channel converts symbol sequences into semantic vectors through a Transformer encoder, and uses a spatial attention mechanism to identify the dependency structure rules between operators and expressions.

[0094] Furthermore, the system is also used to: set up a multi-scale feature fusion unit to perform tensor product operation on shallow symbol texture features and deep syntax and semantic features; at the same time, output the classification results of formula error level and error correction probability value through fully connected layers respectively; combined with the user interaction environment change response mechanism, when the input device switching or handwriting style change is detected, the influence weight vector of the correction confidence factor and the correlation coefficient of the correction confidence factor exceed the normal range, automatically triggering the incremental learning mechanism to fine-tune the local recognition parameters.

[0095] In Embodiment 3, based on the same inventive concept as the intelligent handwritten mathematical formula recognition and correction method in the foregoing embodiments, this application also provides a computer-readable storage medium storing a computer program, which, when executed, implements the steps of the intelligent handwritten mathematical formula recognition and correction method described in any one of Embodiment 1 above.

[0096] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0097] Obviously, those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. An intelligent method for recognizing and correcting handwritten mathematical formulas, characterized in that, The method includes: The original image data of handwritten mathematical formulas were collected, and the image quality was optimized by denoising and binarization preprocessing to obtain standardized image data. Based on the contour matching threshold corresponding to symbol strokes, the structural specification threshold corresponding to operators, and the grammatical logic deviation coefficient corresponding to expressions, coupled analysis is performed in conjunction with the topological relationship of formula syntax to formulate a structural feature matrix. Simultaneously, based on the formula syntax topology relationship and the changing trend of the syntax logic deviation coefficient over multiple recognition periods, formula error type patterns are identified; Based on the structural feature matrix, combined with the formula error type pattern and standardized image data, a correction confidence factor is configured, and the correction confidence factor is strongly correlated with the input layer weights of the formula recognition and correction model. Using the aforementioned correction confidence factor, the formula error level is determined for graded correction feedback. The graded correction feedback is used to output a standard formula that conforms to mathematical expression norms and drive the voice broadcasting unit to provide synchronous voice correction prompts.

2. The intelligent handwritten mathematical formula recognition and correction method as described in claim 1, characterized in that, The formula error type patterns are classified as symbolic errors, operator contradiction errors, syntax and logic errors, and expression structure errors; In the standardized image data, the symbolic form error and the expression structure error in the formula error type pattern are distinguished by contour similarity; Simultaneously, based on the rule matching degree and semantic association matrix in the syntax analysis results, operator contradiction errors and syntactic logic errors in the formula error type pattern are identified.

3. The intelligent handwritten mathematical formula recognition and correction method as described in claim 2, characterized in that, Based on the formula syntax topological relationship, the standardized image data is decomposed into multiple image pyramid layers, and edge feature components related to the outline of symbol strokes are extracted. Based on the expression structure error in the formula error type pattern, a comparative analysis is performed with the edge feature components; If the symbol connection break length of the edge feature component exceeds the break threshold and the structure fitting error increases sharply within the preset recognition area, the dynamic correction mechanism of the syntax logic deviation coefficient corresponding to the expression is triggered.

4. The intelligent handwritten mathematical formula recognition and correction method as described in claim 1, characterized in that, Based on the structural feature matrix, combined with the formula error type pattern and standardized image data, a correction confidence factor is configured, and the method further includes: Use symbol contour matching degree, operator structure compliance rate, and expression syntax adaptation value as input variables; Based on the input variables, the structural feature matrix is ​​fused, and the support function for formula recognition and correction is updated; The corrected confidence factor is then subjected to correlation coefficient feedback verification using the support function.

5. The intelligent handwritten mathematical formula recognition and correction method as described in claim 4, characterized in that, The correction confidence factor is strongly correlated with the input layer weights of the formula recognition correction model, and the method includes: Set the mapping relationship matrix between the support function and the input layer weights of the formula recognition correction model, and decompose and determine the influence weight vector of the correction confidence factor; When the correlation coefficient of the corrected confidence factor is lower than the correlation coefficient threshold for Q consecutive identification periods, the weight adaptive update mechanism is triggered, wherein the Q identification periods are the empirically calibrated sliding window length.

6. The intelligent handwritten mathematical formula recognition and correction method as described in claim 5, characterized in that, The learning rate is dynamically adjusted based on changes in the handwriting input environment, employing a cosine annealing strategy. ,in, Let be the learning rate for the t-th iteration. and Here, denoted as minimum learning rate and maximum learning rate, respectively, and T represents the total number of iterations.

7. The intelligent handwritten mathematical formula recognition and correction method as described in claim 6, characterized in that, Set up a formula recognition and correction model under a two-way channel attention mechanism; Among them, the first unidirectional channel uses a convolutional neural network to extract symbol morphological features of standardized image data, combines contour matching threshold and symbol similarity to construct a morphological correction factor, and uses a channel attention mechanism to adaptively adjust the weights of different symbol types to highlight easily confused character features. The second unidirectional channel converts the symbol sequence into a semantic vector through a Transformer encoder and uses a spatial attention mechanism to identify the dependency structure patterns between operators and expressions.

8. The intelligent handwritten mathematical formula recognition and correction method as described in claim 7, characterized in that, The method further includes: A multi-scale feature fusion unit is set up to perform tensor product operation on shallow symbolic texture features and deep syntactic and semantic features; Meanwhile, the classification results of the formula error level and the error correction probability value are output through the fully connected layer respectively. Combined with the user interaction environment change response mechanism, when the input device switch or handwriting style change is detected, the influence weight vector of the correction confidence factor and the correlation coefficient of the correction confidence factor exceed the normal range, and the incremental learning mechanism is automatically triggered to fine-tune the local recognition parameters.

9. An intelligent handwritten mathematical formula recognition and correction system, characterized in that, The steps for implementing the intelligent handwritten mathematical formula recognition and correction method according to any one of claims 1 to 8 include: The image data acquisition module is used to collect the original image data of handwritten mathematical formulas, and optimize the image quality through denoising and binarization preprocessing to obtain standardized image data. The coupling analysis module is used to perform coupling analysis based on the contour matching threshold corresponding to the symbol stroke, the structural specification threshold corresponding to the operator, and the grammatical logic deviation coefficient corresponding to the expression, combined with the formula grammatical topology, and to formulate the structural feature matrix. The error type identification module is used to identify formula error type patterns based on the formula syntax topology relationship and the changing trend of the syntax logic deviation coefficient over multiple identification periods. The correction confidence factor configuration module is used to configure a correction confidence factor based on the structural feature matrix, combined with the formula error type pattern and standardized image data. The correction confidence factor is strongly correlated with the input layer weights of the formula recognition and correction model. The standard formula output module is used to determine the formula error level using the correction confidence factor and perform graded correction feedback. The graded correction feedback is used to output a standard formula that conforms to the mathematical expression standard and drive the voice broadcast unit to provide synchronous voice correction prompts.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed, implements the steps of the intelligent handwritten mathematical formula recognition and correction method according to any one of claims 1 to 8.

Images