Handwritten mathematical examination subjective question marking method based on image recognition and fuzzy matching

Abstract

The application discloses a handwriting mathematical test paper subjective question marking method based on image recognition and fuzzy matching, and comprises the following steps: obtaining a test question image, and performing a pretreatment operation on the obtained test question image; a handwriting mathematical formula multi-scale recognition model based on an encoder-decoder is constructed; a test question region is recognized based on a multi-scale recognition model and an OCR model; and a candidate's answer and a standard answer are compared step by step in combination with mathematical grammar characteristics, a multi-feature similarity calculation method and a fuzzy matching technology. The handwriting mathematical test paper subjective question marking method based on image recognition and fuzzy matching is used to efficiently and accurately mark subjective questions of a handwriting mathematical test paper, solves the problems of complex formula recognition, multi-solution path matching and logical deduction scoring, significantly improves marking efficiency, reduces manual intervention, ensures the fairness and consistency of scoring, is suitable for the automatic marking needs of education field examinations, and greatly reduces the work burden of teachers.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and pattern recognition technology, and in particular to a method for grading subjective questions on handwritten math exam papers based on image recognition and fuzzy matching. Background Technology

[0002] In education, examinations are a crucial means of assessing and evaluating students' mastery of knowledge and skills, and an essential part of the teaching and learning process. Whether in primary, secondary, or higher education, there are numerous examinations and tests each semester. However, with the increasing number of students and the diversification of examination formats, the grading of examinations, especially subjective questions, still relies heavily on manual labor.

[0003] Taking mathematics exams as an example, especially in large-scale exams at the university and high school levels, grading subjective questions consumes a significant amount of teachers' energy and time. Subjective mathematics questions not only involve the accuracy of calculation results but also the rationality and rigor of the solution steps and logical deduction processes. This means that teachers need to review each student's solution process individually and judge the correctness of each step. At the same time, mathematics problems often have multiple solution paths, requiring teachers to flexibly determine the reasonableness of different solutions, further demonstrating the complexity of grading.

[0004] In large-scale examinations, manual grading is slow and prone to delays in releasing results. Furthermore, subjective factors such as fatigue during the grading process can lead to inconsistencies or errors in scoring, affecting fairness. Teachers' energy is often focused on grading, reducing the time available for interaction with students and problem-solving. Therefore, improving the efficiency and accuracy of grading while reducing teachers' workload has become a pressing issue in the education field.

[0005] In recent years, with the development of information technology and artificial intelligence, automated grading systems have been gradually applied to examinations, especially in grading objective questions such as multiple-choice questions, where they have shown significant advantages. However, for handwritten subjective questions, particularly in mathematics involving formula derivation, symbol recognition, and complex problem-solving steps, existing automation technologies still have limitations. Current automated grading systems for subjective questions often focus only on the correctness of the final answer, failing to identify and evaluate the rationality of the solution steps, and neglecting key steps and logical rigor in the problem-solving process.

[0006] Therefore, a method is needed that can automatically recognize handwritten subjective math questions and intelligently analyze and grade them. This method can not only recognize the mathematical symbols and formulas written by the examinees, but also tolerate certain mathematical deviations through fuzzy matching technology, flexibly handle different tiered paths, and ensure that while improving grading efficiency, the accuracy and fairness of the scoring are maintained. Summary of the Invention

[0007] The purpose of this invention is to provide a method for grading subjective questions on handwritten math exams based on image recognition and fuzzy matching. By combining deep learning models with fuzzy matching technology, it can efficiently and accurately grade subjective questions on handwritten math exams, solving the problems of complex formula recognition, multiple solution path matching, and logical derivation scoring. This significantly improves grading efficiency, reduces human intervention, and ensures the fairness and consistency of scoring. It is suitable for the automated grading needs of examinations in the education field and greatly reduces the workload of teachers.

[0008] To achieve the above objectives, this invention provides a method for grading subjective questions on handwritten mathematics exam papers based on image recognition and fuzzy matching, comprising the following steps:

[0009] Step S1: Acquire the test question image and perform preprocessing operations on the acquired test question image;

[0010] Step S2: Construct a multi-scale recognition model for handwritten mathematical formulas based on encoder-decoder;

[0011] Step S3: Based on the multi-scale recognition model and OCR model constructed in step S2, identify the test question area;

[0012] Step S4: Combining mathematical grammar characteristics, the multi-feature similarity calculation method based on the BLEU algorithm, and fuzzy matching technology, the candidate's answer and the standard answer are compared step by step.

[0013] Preferably, in step S1, the test question image is acquired, and the acquired test question image is preprocessed. The specific process is as follows:

[0014] Step S11: Acquisition and segmentation of the test question image;

[0015] First, the candidates' paper exam papers are converted into electronic image format using a scanner; then, the acquired exam question images are segmented according to the question area, and the subjective question area is extracted.

[0016] The coordinates of the question area are the coordinates used when the answer sheet was made, and the cutting is done by scaling the scanned image proportionally according to the anchor points in the answer sheet.

[0017] Step S12: Perform preprocessing operations on the acquired test question images, specifically including grayscale conversion, binarization, image noise reduction, tilt correction, and image enhancement.

[0018] Preferably, in step S2, a multi-scale recognition model for handwritten mathematical formulas based on an encoder-decoder is constructed, and the specific process is as follows:

[0019] Step S21, Encoder Design: The encoder is based on the ResNet-50 network, combined with the feature enhancement module FEM and the adaptive feature fusion module AFF to achieve multi-scale feature extraction;

[0020] Step S22, Decoder Design: The decoder decodes the multi-scale feature map generated by the encoder into a symbol sequence. The decoder adopts a bidirectional GRU structure and combines the enhanced attention mechanism EAM to model the symbol position and contextual relationship.

[0021] Step S23: Train and optimize the constructed encoder-decoder-based multi-scale recognition model for handwritten mathematical formulas.

[0022] Preferably, in step S21, the encoder is designed, wherein the input of the encoder is an image of a handwritten mathematical formula, and the output is multi-scale features. The specific process is as follows:

[0023] Step S211: Introduce the improved activation function Swish-ReLU after each residual block of ResNet to construct the Feature Enhancement Module (FEM), which specifically includes:

[0024] (1) The Swish-ReLU activation function is as follows:

[0025]

[0026] Where k represents a single value at a certain position in the feature map output by each residual block convolutional layer in the encoder, i.e., the input value; sigmoid is a smooth non-linear function; β is a parameter that determines the smoothness of the activation function; γ is a scaling factor used to scale the output in the negative range.

[0027] (2) In the upper part of the Swish-ReLU activation function, the input x is compressed to the range (0,1) based on the smooth nonlinear function Sigmoid, as shown below:

[0028]

[0029] Based on this, multiplying by k makes the activation function continuous and smooth in the positive range;

[0030] (3) In the lower half of the Swish-ReLU activation function, the activation function introduces a scaling factor γ for negative input values ​​to preserve negative value information;

[0031] Step S212: Add the Adaptive Feature Fusion (AFF) module to achieve the fusion of shallow and deep features in the handwritten mathematical formula, specifically including:

[0032] (1) Input features: The feature maps of the encoder are output by different convolutional layers, as shown below:

[0033] F1, F2, F3, ..., F l ;

[0034] Among them, F l The feature is of layer l, with dimension H. l ×W l ×C l ;

[0035] (2) The importance of the feature layer is dynamically adjusted by the Softmax function, the dynamic weight is calculated, and the fusion effect of multi-scale features is optimized.

[0036] A. Feature map dimensionality reduction: For each layer of feature map F l It is a multidimensional tensor with shape H×W×C, and its relationship to F l Applying global average pooling, we obtain a scalar, as shown below:

[0037]

[0038] Among them, g(F l ) represents the feature map F l Global features; F l (i,j) is the feature map F l The value at position (i,j); i represents the index of the feature map in the vertical direction; j represents the index of the feature map in the horizontal direction;

[0039] B. The dynamic adjustment coefficient is used to control the importance of each feature layer. The coefficient is automatically optimized based on the data to dynamically adjust the weight distribution of the feature layer.

[0040] C. Softmax normalization: This reduces γ l ·g(F l As the weight factor of the feature layer, the final weight ω is calculated by normalizing all layers using the Softmax function. l As shown below:

[0041]

[0042] Where, γ l γ represents the dynamic adjustment coefficient of the features in the l-th layer; k G(F) represents the dynamic adjustment coefficient of the k-th layer feature; l ) represents the global information of the feature map of the l-th layer; g(F) k () represents the global information of the k-th feature map; N is the total number of feature layers;

[0043] (3) Feature fusion: Multi-layer features are fused according to dynamic weight ωl The weighted sums generate a unified feature F across multiple scales. multi ;

[0044] First, for each layer of features F l According to the corresponding dynamic weight ω l The weighting is performed as follows:

[0045] F weighted,l =ω l ·F l ;

[0046] Among them, F weighted,l Indicates after ω l Weighted feature map;

[0047] Subsequently, all weighted feature maps are accumulated and fused layer by layer to generate the final multi-scale feature F. multi As shown below:

[0048]

[0049] Finally, the encoder outputs the fused feature F. multi .

[0050] Preferably, in step S22, the decoder decodes the multi-scale feature map generated by the encoder into a symbol sequence. The decoder adopts a bidirectional GRU structure and combines an enhanced attention mechanism (EAM) to model the symbol position and contextual relationships. The specific process is as follows:

[0051] Step S221: The decoder directly uses the fused feature F output by the encoder. multi As input;

[0052] (1) Forward GRU processes F step-by-step. multi Generate the forward hidden state sequence as follows:

[0053]

[0054] in, This represents the hidden state generated by the forward GRU at time step T, which contains the up and down information of the previous T time steps;

[0055] (2) Backward GRU reverse processing F multi Generate the backward hidden state sequence as follows:

[0056]

[0057] in, This represents the hidden state generated by the backward GRU at time step T, which contains context information from time step T to 1;

[0058] (3) Concatenate the forward and backward hidden states to generate the complete hidden state h. t As shown below:

[0059]

[0060] Among them, h t ⊕ represents the complete hidden state after concatenation at time step t; ⊕ represents the concatenation operation, and the hidden state after concatenation contains the contextual information of the fused features in both the forward and reverse directions.

[0061] Step S222: Enhanced attention mechanism (EAM) is achieved by calculating attention weight α. t The position that contributes most to the prediction of the current symbol in the encoder feature map is selected, and the learnable position encoding is combined to enhance the modeling of the relationship between symbols.

[0062] Attention weight α t The following represents the degree of attention the decoder pays to each position of the encoder output at time t:

[0063]

[0064] Among them, h t It is the hidden state at the current time t; W a It is a trainable weight matrix; p t It is a learnable positional encoding used to enhance the decoder's awareness of symbol positions;

[0065] Learnable positional encoding p t As shown below:

[0066] p t =W p ·Pos(t);

[0067] Where Pos(t) is the relative position vector of the t-th symbol; W p It is a positional encoding weight matrix;

[0068] Based on attention weight α t The encoder features are weighted and summed to generate a context vector c. t As shown below:

[0069]

[0070] Where M represents the number of spatial dimensions in the encoder output feature map;

[0071] Step S223, Symbol Prediction: Combining context vector c t and the current hidden state h tThe probability distribution for generating symbols is shown below:

[0072] y t =Softmax(W y ·h t );

[0073] Among them, W y It is a trainable matrix used to map hidden states to symbol classes, outputting symbol y at each time t. t This ultimately generates a complete symbol sequence y1, y2, ..., y T ;

[0074] Step S224, symbol resolution and formula structure generation, specifically includes:

[0075] (1) Operator precedence resolution: sorting the operators in the symbol sequence according to the mathematical operation rules;

[0076] (2) Determine the superscript and subscript relationship: Based on the position of the symbol in the sequence and combined with the position encoding information, determine the superscript and subscript relationship of the symbol;

[0077] (3) Constructing tree-like expressions: Organizing symbols into a tree structure according to logical relationships to represent the hierarchical relationship of the formula;

[0078] (4) Recursive traversal of the output structure: By traversing the tree expression, the tree structure is reorganized into a linear output form;

[0079] After the above steps, the symbol sequence generated by the decoder is parsed into a complete mathematical formula tree.

[0080] Preferably, in step S23, the constructed encoder-decoder-based multi-scale recognition model for handwritten mathematical formulas is trained and optimized, and the specific process is as follows:

[0081] Step S231, Data Preparation: Collect a large number of image samples containing handwritten formulas, and generate a corresponding mathematical formula LaTeX format expression for each handwritten formula image;

[0082] Among them, image data is augmented by randomly combining and sampling multiple augmentation methods such as rotation, scaling, translation, affine transformation, noise addition, brightness adjustment, and contrast adjustment.

[0083] Step S232, Model Training, specifically includes:

[0084] First, the dataset is divided into a training set and a validation set in an 8:2 ratio;

[0085] Then, the Adam optimizer is used, with an initial learning rate of 0.001, and the learning rate is dynamically adjusted using a cosine annealing learning rate adjuster, as shown below:

[0086]

[0087] Where, η t η is the current learning rate. min η is the minimum learning rate. max The initial learning rate is G, the maximum number of training epochs is G, and the current training epoch is g.

[0088] To ensure the accuracy of the predicted symbols and the rationality of the logical structure, two loss functions were designed: character-level cross-entropy loss and structure matching loss, as shown below:

[0089] (1) Character-level cross-entropy loss is used to supervise symbol sequence generation, ensuring that the predicted results match the real symbol sequences character by character, as shown below:

[0090]

[0091] Where Q represents the total length of the symbol sequence; y i It is the actual label (one-hot encoded form) of the i-th symbol; p i It is the probability distribution of the i-th symbol predicted by the model;

[0092] (2) The structural matching loss evaluates the logical consistency of the formula by the matching degree of the syntax tree. It is suitable for evaluating the priority, nesting relationship and semantic logic between symbols, as shown below:

[0093]

[0094] in, It is the prediction syntax tree generated by the model; It is the syntax tree of the standard answer; match(·) is the number of nodes matched in the syntax tree; It is the total number of nodes in the standard answer syntax tree;

[0095] (3) To balance the accuracy of symbol prediction and the consistency of logical structure, a weighted combination loss function is used during training, as shown below:

[0096]

[0097] Here, λ1 and λ2 are weight hyperparameters used to adjust the contribution ratio of the two losses.

[0098] Preferably, in step S3, the test question area is identified based on the handwritten mathematical formula multi-scale recognition model and the OCR model. The specific process is as follows:

[0099] Step S31, Formula Region Detection: Using the constructed multi-scale recognition model, directly detect the regions of handwritten formulas in the test paper, and directly output the sequence R1, R2, ..., R of the formula region coordinates R = (x, y, w, h). n Where x and y are the coordinates of the top-left corner of the region; w and h are the width and height, respectively;

[0100] Step S32, Region Merging and Optimization, specifically includes:

[0101] First, determine proximity relationships and calculate the spatial distance between detection boxes; for every two detection boxes R i and R j Calculate their boundary distance d(R) i ,R j ), as shown below:

[0102]

[0103] in, and The detection box R is respectively i and R j The coordinates of the right and left boundaries;

[0104] Then, set a distance threshold δ, if d(R) i ,R j If ) < δ, then the two detection boxes are considered to belong to the same formula;

[0105] Finally, the formula area is masked;

[0106] Step S33, Text Region Detection: Utilize an OCR model to directly detect the image area occluded by the formula region, directly outputting the region where the text exists, consistent with the formula region, and outputting the text region coordinates T = (x...). t ,y t ,w t ,h t The sequence T1, T2, ..., T n ;where x t and y t This indicates the coordinates of the top-left corner of the text area; w t and h t Indicates the width and height of the respective text areas;

[0107] Step S34: Recognize the formulas and text, specifically including:

[0108] First, the image is segmented according to the formula region sequence and the text region sequence. The segmented formula region images are then input into a multi-scale recognition model, which outputs the corresponding formula region sequence, as shown below:

[0109] {(R1,w1),(R2,w2),…,(R n ,w n )};

[0110] Then, the segmented text region image is input into the OCR model, which outputs the corresponding text sequence, as shown below:

[0111] {(T1,w1),(T2,w2),…,(T n ,w n )};

[0112] Finally, the formula recognition results and text recognition results are spliced ​​together in the original order of the regions to restore the complete test content.

[0113] Preferably, in step S4, the candidate's answer is compared with the standard answer step by step by combining mathematical grammar characteristics, multi-feature similarity, and fuzzy matching technology. The specific process is as follows:

[0114] Step S41, step-by-step analysis of the mathematical solution, specifically including:

[0115] Step S411: Construct a dictionary of mathematical symbols.

[0116] Step S412: The reverse maximum matching algorithm based on a custom mathematical symbol dictionary is used to parse the handwritten mathematical formula text step by step; the specific word segmentation process is as follows:

[0117] a. Input: Steps to solve a math problem;

[0118] b. Scanning: Scanning character by character from back to front;

[0119] c. Match: Prioritizes matching the longest word in the dictionary; if no match is found, returns a single character.

[0120] Step S413: Based on the word segmentation results, construct a mathematical syntax tree by parsing the priority and logical relationship between symbols, and generate a tree structure suitable for formula calculation and logical derivation;

[0121] Step S414: Extract the calculation logic from the formula layer by layer through the structure of the syntax tree, decompose the complete mathematical solution into independent logical operations, until the final answer is obtained;

[0122] Step S42: Using a multi-feature similarity calculation method based on the BLEU algorithm, combined with the grammatical properties of mathematical formulas, the similarity between the candidate's answer and the standard answer is evaluated. Specifically, this includes:

[0123] Step S421: Perform character-level decomposition on the formula strings of the candidate's answer and the standard answer, parsing the formulas into sequences composed of symbols and variables; then, use the BLEU algorithm to perform step-by-step matching of the character sequences and calculate the character-level n-gram similarity S. char As shown below:

[0124]

[0125] Where N is the maximum value of the calculated n-gram; w n Let be the weighting coefficient, satisfying p n The percentage overlap between the candidate's answer and the standard answer in the n-gram format;

[0126] Step S422: Perform point-by-point matching between the syntax trees of the candidate's answer and the standard answer, determine whether the symbol types and logical relationships of each node are consistent, and calculate the structural similarity S based on the similarity of each formula. struct As shown below:

[0127]

[0128] Where Match(A,B) represents the nodes that match the standard answer A and the student's answer B; sum(A) represents the total number of nodes in the standard answer.

[0129] Step S423: Introduce a logical order similarity calculation method to ensure the fairness and accuracy of scoring by verifying the logicality of the problem-solving steps and the equivalence between them; wherein, the logical order similarity S is calculated. logic (A,B), as shown below:

[0130]

[0131] Where sam(A,B) represents the number of matches between the standard answer and the candidate's answer; MaxRev(A,B) represents the maximum number of inversions in a sequence of natural numbers with the same number of solution steps; and sum(A,B) represents the total number of solution steps in the standard answer.

[0132] Step S424: Extract the crucial key steps k1, k2, ..., k from the standard answer. n And assign independent weights w1, w2, ..., w to each key step. n The total weights satisfy the following condition:

[0133]

[0134] Among them, w i Let P represent the weight of the i-th critical step, and P be the total number of critical steps.

[0135] The test takers' answers are analyzed step-by-step using the same rules as the standard answers to extract potential key steps. These extracted steps are then matched with the key steps in the standard answers to calculate the key step similarity S. key As shown below:

[0136]

[0137] Where, match(k) i (k) is the key step i The degree of matching in the candidates' answers, with the following range of values:

[0138] Exact match: k i =1; Partial match: k i =0.5; mismatch: k i =0;

[0139] Step S425: The overall score is based on character-level similarity S. char Structural similarity S struct Logical order similarity S logic Similarity S to key steps key The result is obtained by weighted fusion of the similarity of the four features, as shown below:

[0140] S total =α1·S char +α2·S struct +α3·S logic +α4·S key ;

[0141] Among them, S total The final fusion score is α1, α2, α3, and α4 are the feature weights, which satisfy the following constraints:

[0142]

[0143] Step S43: Processing multiple solutions to the standard answer. By calculating the total score of each solution to the candidate's answer and the standard answer, the solution with the highest score is selected as the final score.

[0144] The multiple solutions to the standard answer are organized into a set {A1, A2, ..., A...} n}, each A i This represents a complete solution path; based on the multi-feature fusion scoring formula in step S425, calculate the candidate's answer B and various solutions A respectively. i Overall score S total (B,A i ), as shown below:

[0145] Stotal (B,A i )=α1·S char +α2·S struct +α3·S logic +α4·S key ;

[0146] Then, select the standard solution A with the highest fusion score. best The final scoring criteria are as follows:

[0147]

[0148] Among them, A best The final standard solution chosen; S total (B,A i (A) is the candidate's answer and the standard solution A. i The overall score.

[0149] Therefore, this invention adopts the above-mentioned method for grading subjective questions on handwritten math exams based on image recognition and fuzzy matching. Combining deep learning models and fuzzy matching technology, it can efficiently and accurately grade subjective questions on handwritten math exams, solving the problems of complex formula recognition, multiple solution path matching, and logical derivation scoring. It significantly improves grading efficiency, reduces human intervention, and ensures the fairness and consistency of scoring. It is suitable for the automated grading needs of examinations in the education field and greatly reduces the workload of teachers.

[0150] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0151] Figure 1 This is a flowchart of a method for grading subjective questions in handwritten mathematics exam papers based on image recognition and fuzzy matching, according to the present invention.

[0152] Figure 2 This is the result of sequence L1 priority parsing;

[0153] Figure 3 This is the result of parsing the L2 subscript-superscript relationship of the sequence;

[0154] Figure 4 It is a tree-like expression constructed from the L3 sequence. Detailed Implementation

[0155] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0156] like Figure 1 As shown, the method for grading subjective questions on handwritten math exams based on image recognition and fuzzy matching includes the following steps:

[0157] Step S1: Obtain the test question image and perform preprocessing operations on the obtained test question image so that the subsequent steps can more accurately recognize handwritten mathematical formulas and text.

[0158] Step S11: Acquisition and segmentation of the test question image.

[0159] Step S111: Scan the test paper: Convert the candidate's paper test paper into an electronic image format through a scanner, usually in JPGE or PNG format.

[0160] Among them, to ensure that the image clarity is high enough for subsequent recognition processing, the scanning accuracy of the scanning device should not be lower than 300 DPI.

[0161] Step S112: Segmentation of the test question area: Segment the obtained image according to the test question area and extract the subjective question area.

[0162] The coordinates of the test question area are the coordinates when making the answer sheet, and the cutting is accurately performed by scaling the scanned image according to the anchor points in the answer sheet.

[0163] Step S12: The purpose of image preprocessing is to optimize the image quality, make the image features more obvious, and improve the recognition accuracy.

[0164] Step S121: Grayscale conversion: Convert the test question area image into a grayscale image, which can retain important features while reducing the computational amount. Among them, the grayscale conversion formula is:

[0165] I gray = 0.299×I R + 0.587×I G + 0.114×I B [[ID=三十二]];

[0166] Among them, I gray represents the grayscale image; I R , I G and I B respectively represent the red, green, and blue channels of the image. In the grayscale image I gray , each pixel has only one brightness value, which is suitable for subsequent binarization processing.

[0167] Step S122: Binarization: Perform binarization processing using the OTSU algorithm. Convert the grayscale image into a black-and-white image by setting a threshold T, that is, adjust the pixels with pixel value > T to the maximum grayscale value 255, and adjust the pixels with pixel value < T to the minimum grayscale value zéro, so that only black and white colors exist on the image.

[0168] The OTSU algorithm can automatically select the optimal threshold T based on image conditions, clearly separating the foreground and background and ensuring excellent segmentation results, especially suitable for images with significant gray-level differences between the foreground and background. Based on the OTSU algorithm, the optimal threshold T is determined by minimizing the intra-class variance. This is obtained by... The definition of within-class variance is as follows:

[0169]

[0170] Where ω0(T) and ω1(T) are the pixel proportions of the foreground and background, respectively; and The variances are for the foreground and background, respectively.

[0171] This process can also be viewed as maximizing the inter-class variance. As shown below:

[0172]

[0173] Where μ0(T) and μ1(T) are the average gray values ​​of the foreground and background. Through this optimization process, the OTSU algorithm can automatically obtain the optimal binarization threshold for the image.

[0174] Step S123, Image Denoising: During the acquisition of test question images, various noises often appear in the scanned images due to external factors (such as uneven lighting, paper texture, etc.), such as Gaussian noise, salt-and-pepper noise, or speckle noise. In order to improve the accuracy of subsequent image recognition, effective noise reduction processing must be performed on the scanned images.

[0175] Image denoising is performed using a Gaussian filtering algorithm. A two-dimensional Gaussian filter is used to smooth the image, eliminating noise while preserving key information. The two-dimensional Gaussian filter is shown below:

[0176]

[0177] Where σ is the standard deviation of the Gaussian filter, which controls the strength of the filter; x represents the offset of a pixel in the image from the center coordinates of the filter in the horizontal direction; and y represents the offset of a pixel in the image from the center coordinates of the filter in the vertical direction.

[0178] To improve computational efficiency and further optimize the filtering process, the two-dimensional Gaussian filtering is decomposed into two one-dimensional convolution operations. First, a one-dimensional Gaussian filter is applied to each row of the image, as shown below:

[0179]

[0180] Then, perform a one-dimensional Gaussian filter on each column, as shown below:

[0181]

[0182] This decomposition method significantly reduces computational complexity, improves processing speed, and ensures both the efficiency and accuracy of image recognition while maintaining noise reduction effectiveness.

[0183] Step S124, Tilt Correction: During image acquisition, the scanned or captured test image may be tilted, which can affect subsequent image segmentation and content recognition. Therefore, tilt correction is introduced to automatically detect and adjust the tilt angle of the image to restore it to a standard horizontal or vertical state.

[0184] First, an edge detection algorithm (such as Canny edge detection) is applied to the image to extract straight edges in the image.

[0185] Then, the Hough transform is used to identify the main straight line contours in the image, and the overall tilt angle of the image is estimated by analyzing the tilt angle of these contours.

[0186] The formula for estimating the tilt angle is as follows:

[0187]

[0188] Where θ is the tilt angle; Δy and Δx represent the changes in the line in the vertical and horizontal directions, respectively. By calculating the average tilt angle of the main lines, the overall rotation angle of the image can be determined.

[0189] Finally, after obtaining the overall rotation angle of the image, the image is rotated counterclockwise or clockwise to correct the rotation and restore the image content to a horizontal state.

[0190] The correction formula is as follows:

[0191] x = xcos(θ) - ysin(θ);

[0192] y = xcos(θ) + ysin(θ);

[0193] Where (x,y) are the coordinates before correction, (x',y') are the coordinates after correction, and θ is the tilt angle. This step accurately corrects the image content, providing a high-quality input image for subsequent character and formula recognition.

[0194] Step S125, Image Enhancement: In image preprocessing, image enhancement is used to improve image clarity and make features more prominent. The image enhancement process includes operations such as contrast adjustment, sharpening, and brightness balancing, which can make blurry characters and details clearer and improve recognition accuracy.

[0195] The image enhancement method used in this invention is as follows:

[0196] (1) Contrast Enhancement: Image contrast is enhanced through histogram equalization. Histogram equalization adjusts the grayscale distribution of the image to a wider range, making the contrast between light and dark areas of the image more obvious, as shown below:

[0197]

[0198] Where I represents the original image; I min and I max I represents the minimum and maximum gray values ​​of the image, respectively. enhanced This is the image after contrast enhancement. Contrast enhancement improves the overall clarity of the image and highlights the handwritten content.

[0199] (2) Sharpening: The Laplacian operator is used to sharpen the image to enhance edges and details, making the outlines of handwritten characters more distinct. The Laplacian operator formula is shown below:

[0200]

[0201] Where α is the sharpening coefficient, which controls the degree of sharpening; The result of applying the Laplacian operator to an image; I sharp This is the image after sharpening. Sharpening enhances the edges of handwritten characters and symbols, making image details clearer.

[0202] Step S2: Construct a multi-scale recognition model for handwritten mathematical formulas based on encoder-decoder.

[0203] The constructed encoder-decoder-based multi-scale recognition model for handwritten mathematical formulas solves the problems of inaccurate parsing of complex symbol structures and insufficient ability to handle multiple solution paths in existing technologies. By introducing a feature enhancement module (FEM), an adaptive feature fusion module (AFF), an enhanced attention mechanism (EAM), and a tree structure generator (TG), high-precision recognition and parsing of handwritten formulas are achieved.

[0204] Step S21, Encoder Design: The encoder is based on the ResNet-50 network and combines a Feature Enhancement Module (FEM) and an Adaptive Feature Fusion Module (AFF) to achieve efficient multi-scale feature extraction.

[0205] The encoder's input is an image of a handwritten mathematical formula. Where H and W represent the height and width of the image, respectively, and the number of channels (RGB image) is 3; the encoder output is a multi-scale feature. Among them, H 'W ' This is the size of the encoded feature map, and C is the number of channels.

[0206] Step S211: Construct the Feature Enhancement Module (FEM). An improved activation function, Swish-ReLU, is introduced after each residual block of the ResNet. The goal is to optimize the expressiveness of feature extraction in the neural network, making it more suitable for handling complex symbols and structures in handwritten mathematical formulas. By using the Swish-ReLU activation function, the FEM module combines the advantages of existing activation functions (the sparsity of ReLU and the smoothness of Swish), improving feature representation capabilities while enhancing the perception of details and complex symbols.

[0207] (1) The Swish-ReLU activation function is as follows:

[0208]

[0209] Where k represents a single value at a certain position in the feature map output by each residual block convolutional layer in the encoder, i.e., the input value; sigmoid is a smooth non-linear function; β is a parameter that determines the smoothness of the activation function; γ is a scaling factor used to scale the output in the negative range.

[0210] (2) In the upper part of the Swish-ReLU activation function, the input k is compressed to the range (0,1) based on the smooth nonlinear function Sigmoid, as shown below:

[0211]

[0212] Building upon this, multiplying by k makes the activation function continuous and smooth within the positive range, resolving the "dead neurons" problem of ReLU. Furthermore, Swish-ReLU exhibits stronger gradient propagation capabilities when β > 1. The value of β ranges from [0.5, 2.0], determining the smoothness of the activation function, as shown below:

[0213] When β = 1.0, the activation function changes most smoothly near the activation value of zero;

[0214] When β > 1.0, the activation function curve becomes steeper, and the network responds more strongly to inputs with larger positive values;

[0215] When β < 1.0, the activation function curve becomes gentler, and the network can produce significant activation values ​​even for inputs with small positive values, making it suitable for handling weak features.

[0216] (3) In the lower half of the Swish-ReLU activation function, a scaling factor γ is introduced for negative input values, allowing the network to retain negative information instead of directly setting it to zero (as in ReLU). This avoids information loss, which is especially important for sign edges and sparse features. γ controls the degree to which the activation function retains negative information, as shown below:

[0217] When γ = 0, the activation function degenerates into standard ReLU in the negative region;

[0218] When γ = 1, the negative information of Swish-ReLU is completely preserved, which is equivalent to Leaky ReLU;

[0219] When γ = 0.1, it is a compromise value. Swish-ReLU can suppress excessively large negative values ​​while retaining some negative value information.

[0220] In this embodiment, β = 1.5 and γ = 0.1 are preferred; the Swish-ReLU activation function improves symbol recognition by 2.5% and formula parsing by 3.6% compared to ReLU.

[0221] Step S212: Add the Adaptive Feature Fusion (AFF) module to achieve the fusion of shallow and deep features in the handwritten mathematical formula.

[0222] Shallow features (low-level convolutional layers) are typically good at capturing edges and local details; deep features (high-level convolutional layers) are better at expressing global context and semantic information. To accurately interpret the formula, it is necessary to fuse shallow and deep features, while assigning different weights to each feature layer to highlight the features most relevant to the current task.

[0223] Traditional feature fusion methods use simple weighted averaging or fixed rules, which are difficult to dynamically adapt to symbol complexity. The Adaptive Feature Fusion (AFF) module in this invention improves the Feature Pyramid Network (FPN) by adjusting the weights according to the importance of features in each layer through a dynamic weighting mechanism, thus achieving more flexible feature combinations.

[0224] (1) Input features: The feature maps of the encoder are output by different convolutional layers, as shown below:

[0225] F1, F2, F3, ..., F l ;

[0226] Among them, F l The feature is of layer l, with dimension H. l ×W l ×C l .

[0227] For example, F1 and F2, which have larger sizes, are shallow features, focusing on the edges and details of the symbols; F l-1 ,F l Smaller features are deep features, capturing the global semantics and contextual information of the symbols.

[0228] (2) Dynamic Weight Calculation: Dynamic weight calculation is a key step in the Adaptive Feature Fusion (AFF) module. Its core objective is to assign reasonable dynamic weights to each feature map layer to ensure that the fusion process can adapt to the differences in feature distribution of the input formula. The importance of the feature layers is dynamically adjusted through the Softmax function, thereby optimizing the fusion effect of multi-scale features.

[0229] A. Feature map dimensionality reduction: For each layer of feature map F l It is a multidimensional tensor, typically of shape H×W×C. To reduce computational complexity and extract global information from the feature map, F... l Applying the Global Average Pooling (GAP) operation yields a scalar, as shown below:

[0230]

[0231] Among them, g(F l ) represents the feature map F l Global features; F l (i,j) is the feature map F l The value at position (i,j); i represents the index of the feature map in the vertical direction; j represents the index of the feature map in the horizontal direction.

[0232] B. Dynamic adjustment coefficient: Dynamic adjustment coefficient γ l It is a learnable parameter used to control the importance of features in each layer. Through training, the model automatically optimizes these coefficients based on the data to dynamically adjust the weight distribution of feature layers.

[0233] C. Softmax normalization: This reduces γ l ·g(F l As the weight factor of the feature layer, the final weight ω is calculated by normalizing all layers using the Softmax function. l As shown below:

[0234]

[0235] Where, γ l γ represents the dynamic adjustment coefficient of the features in the l-th layer; k G(F) represents the dynamic adjustment coefficient of the k-th layer feature; l ) represents the global information of the feature map of the l-th layer; g(F) k() represents the global information of the k-th feature map; N is the total number of feature layers. This process uses the softmax function for normalization to ensure that the weights ω l The sum is 1.

[0236] (3) Feature fusion: Feature fusion is the process of combining multi-layer features according to dynamic weights ω. l The weighted sums generate a unified feature F across multiple scales. multi The purpose of this step is to combine the detailed information of shallow features with the global semantics of deep features, making the output feature map more expressive.

[0237] First, for each layer of features F l According to the corresponding dynamic weight ω l The weighting is performed as follows:

[0238] F weighted,l =ω l ·F l ;

[0239] Among them, F weighted,l Indicates after ω l Weighted feature map.

[0240] Subsequently, all weighted feature maps are accumulated and fused layer by layer to generate the final multi-scale feature F. multi As shown below:

[0241]

[0242] Finally, the encoder outputs the fused feature F. multi .

[0243] Step S22, Decoder Design: The decoder's task is to decode the multi-scale feature maps generated by the encoder into symbol sequences. To achieve high-precision parsing of complex formulas, the decoder adopts a bidirectional GRU (Bi-GRU) structure and combines an enhanced attention mechanism (EAM) to model symbol positions and contextual relationships.

[0244] Step S221, Bi-GRU Architecture: The decoder directly uses the fused features output by the encoder. As input. Where T represents the time step of the fused features (length of the symbol sequence), and C is the feature dimension (such as the number of output channels of the encoder).

[0245] (1) Forward GRU processes F step-by-step. multi Generate the forward hidden state sequence as follows:

[0246]

[0247] in, This represents the hidden state generated by the forward GRU at time step T, which contains context information from the previous T time steps.

[0248] (2) Backward GRU reverse processing F multi Generate the backward hidden state sequence as follows:

[0249]

[0250] in, This represents the hidden state generated by the backward GRU at time step T, which contains context information from time step T to 1.

[0251] (3) Concatenate the forward and backward hidden states to generate the complete hidden state h. t As shown below:

[0252]

[0253] Among them, h t ⊕ represents the complete hidden state after concatenation at time step t; ⊕ represents the concatenation operation, and the hidden state after concatenation contains the contextual information of the fused features in both the forward and reverse directions.

[0254] Step S222: Enhanced Attention Mechanism (EAM), by calculating attention weight α t The algorithm selects the position in the encoder feature map that contributes most to the prediction of the current symbol, and combines it with learnable positional encoding to enhance the modeling of the relationship between symbols.

[0255] Attention weight α t The following represents the degree of attention the decoder pays to each position of the encoder output at time t:

[0256]

[0257] Among them, h t It is the hidden state at the current time t; W a It is a trainable weight matrix; p t It is a learnable positional encoding used to enhance the decoder's awareness of symbol positions.

[0258] Learnable positional encoding p t As shown below:

[0259] p t =W p ·Pos(t);

[0260] Where Pos(t) is the relative position vector of the t-th symbol; W p It is the positional encoding weight matrix.

[0261] Based on attention weight α t The encoder features are weighted and summed to generate a context vector c. t As shown below:

[0262]

[0263] Where M represents the number of spatial dimensions in the encoder output feature map.

[0264] Step S223, Symbol Prediction: Combining context vector c t and the current hidden state h t The probability distribution for generating symbols is shown below:

[0265] y t =Softmax(W y ·h t );

[0266] Among them, W y It is a trainable matrix used to map hidden states to symbol classes, outputting symbol y at each time t. t This ultimately generates a complete symbol sequence y1, y2, ..., y T .

[0267] Step S224, Symbol Resolution and Formula Structure Generation: The symbol sequence generated by the decoder is linear, but mathematical formulas need to express the logical relationships between symbols (such as operation precedence, subscript and superscript relationships, etc.). The resolution steps include:

[0268] (1) Operator priority resolution: The operators (such as +, -, *, ÷) in the symbol sequence are sorted according to the mathematical operation rules.

[0269] For example, the input symbol sequence is: L1 = ["x", "=", "y", "+", "z", "*", "w"]; then, according to the mathematical operation precedence rule "multiplication and division take precedence over addition and subtraction", the operators in the symbol sequence are grouped and sorted. The priority resolution result of sequence L1 is as follows: Figure 2 As shown.

[0270] (2) Determine the superscript and subscript relationship: Based on the position of the symbol in the sequence and combined with the position encoding information, determine the superscript and subscript relationship of the symbol.

[0271] For example, given the input symbol sequence L2 = ["x","^","2","+","y"], the subscript-superscript relationship is determined based on the relative positions of the symbols "^ follows x" and the position information provided by the decoder. The parsing result of the subscript-superscript relationship in sequence L2 is as follows: Figure 3 As shown.

[0272] (3) Constructing tree-like expressions: Organize the symbols into a tree-like structure according to logical relationships to represent the hierarchical relationship of the formula.

[0273] For example, the input symbol sequence after priority parsing and subscript / superscript relationship processing is: L3 = ["x","^","2","+","y","*","z"]; tree nodes are generated from the parsing result, with the symbols used as the node values; then subtrees are constructed according to priority, ensuring that nodes with higher operators are processed first; finally, all nodes are merged to generate a complete formula tree. The tree-like expression constructed from sequence L3 is as follows: Figure 4 As shown.

[0274] (4) Recursive traversal of the output structure: By traversing the tree expression, the tree structure is reorganized into a linear output form.

[0275] Inorder traversal is used to convert the tree-like expression of mathematical formulas into a human-readable sequence of symbols. When an operator node is encountered, appropriate parentheses are inserted to ensure the order of operations.

[0276] For example, the output of an inorder traversal of the tree expression generated by sequence L3 is:

[0277] (x^2)+(y*z);

[0278] After the above steps, the symbol sequence generated by the decoder is parsed into a complete mathematical formula tree.

[0279] Step S23, Model Training and Optimization: To ensure that the constructed encoder-decoder-based multi-scale recognition model for handwritten mathematical formulas can efficiently parse complex formulas and adapt to the diversity of handwriting styles, the model is trained and optimized.

[0280] Step S231, data preparation, the specific process is as follows:

[0281] (1) Data collection: Collect a large number of image samples containing handwritten formulas to ensure data diversity, including different writing styles and formula complexity.

[0282] (2) Tag generation: Generate a corresponding mathematical formula LaTeX format expression for each handwritten formula image.

[0283] To improve the generalization ability of the handwritten mathematical formula recognition model, this invention performs data augmentation on image data during model training. Multiple augmentation methods, including rotation, scaling, translation, affine transformation, noise addition, brightness adjustment, and contrast adjustment, are randomly combined and parameters are sampled. Simultaneously, multiple augmentation methods are applied to each sample image, thereby generating diverse training samples and enhancing the model's robustness and generalization ability.

[0284] Step S232: Model training.

[0285] First, the dataset is divided into training and validation sets in an 8:2 ratio. To ensure that the model can generalize to different problem-solving styles and formula structures, the validation set includes different categories of handwritten mathematical formulas.

[0286] Then, the Adam optimizer is used, with an initial learning rate of 0.001, and the learning rate is dynamically adjusted using a cosine annealing learning rate adjuster, as shown below:

[0287]

[0288] Where, η t η is the current learning rate. min η is the minimum learning rate. max G is the initial learning rate, G is the maximum number of training epochs, and g is the current training epoch.

[0289] To ensure the accuracy of predicted symbols and the rationality of logical structure, this invention designs two loss functions: character-level cross-entropy loss and structure matching loss. These two loss functions optimize model performance at different levels, and the final training objective is achieved through weight fusion.

[0290] (1) Character-level cross-entropy loss is used to supervise symbol sequence generation, ensuring that the predicted results match the real symbol sequences character by character, as shown below:

[0291]

[0292] Where Q represents the total length of the symbol sequence; y i It is the actual label (one-hot encoded form) of the i-th symbol; p i This represents the probability distribution of the i-th symbol predicted by the model. Character-level cross-entropy loss ensures that the model accurately predicts each symbol.

[0293] (2) Structural matching loss evaluates the logical consistency of formulas by matching the syntax tree. It is particularly suitable for evaluating the priority, nesting relationship and semantic logic between symbols, as shown below:

[0294]

[0295] in, It is the prediction syntax tree generated by the model; It is the syntax tree of the standard answer; match(·) is the number of nodes matched in the syntax tree; This represents the total number of nodes in the standard answer syntax tree. The structure matching loss is sensitive to the logical relationships between symbols and can effectively capture errors in subscripts, superscripts, fractions, and nested formulas.

[0296] (3) To balance the accuracy of symbol prediction and the consistency of logical structure, a weighted combination loss function is used during training, as shown below:

[0297]

[0298] Wherein, λ1 and λ2 are weight hyperparameters used to adjust the contribution ratio of the two losses; preferably, λ1 = 0.7 and λ2 = 0.3.

[0299] The final performance results of the model training show that the symbol recognition accuracy is as high as 98.5% and the formula structure parsing accuracy is 96.2%. The model can efficiently recognize handwritten symbols of different styles and can highly restore the logical relationship of complex formulas, providing reliable technical support for subsequent automatic grading.

[0300] Step S3: Based on the multi-scale recognition model and OCR model constructed in step S2, the test question area is identified.

[0301] Step S31, Formula Region Detection: Use the constructed multi-scale recognition model to directly detect the regions of handwritten formulas in the test paper, and directly output the sequence R1, R2, ..., R of the formula region coordinates R = (x, y, w, h). n Where x and y are the coordinates of the top-left corner of the region; w and h are the width and height, respectively.

[0302] Step S32, Region Merging and Optimization: To ensure the integrity and uniqueness of the formula, it is necessary to merge adjacent or overlapping regions.

[0303] First, determine proximity relationships and calculate the spatial distance between detection boxes. For every two detection boxes R... i and R j Calculate their boundary distance d(R) i ,R j ), as shown below:

[0304]

[0305] in, and The detection box R is respectively i and R j The coordinates of the right and left boundaries.

[0306] Then, set a distance threshold δ, if d(R) i ,R j If δ < δ, then the two detection boxes are considered to belong to the same formula; preferably δ = 10.

[0307] Finally, the formula area is masked. This step is performed on a copied image and does not affect the original image.

[0308] Step S33, Text Region Detection: Utilize an OCR model to directly detect the image area occluded by the formula region, directly outputting the region where the text exists, consistent with the formula region, and outputting the text region coordinates T = (x...). t ,y t ,w t ,h t The sequence T1, T2, ..., T n Among them, x t and y t This indicates the coordinates of the top-left corner of the text area; w t and h t This indicates the width and height of the respective text areas.

[0309] Step S34: Recognize the formulas and text.

[0310] First, the image is segmented according to the formula region sequence and the text region sequence. The segmented formula region images are then input into a multi-scale recognition model, which outputs the corresponding formula region sequence, as shown below:

[0311] {(R1,w1),(R2,w2),…,(R n ,w n )};

[0312] Then, the segmented text region image is input into the OCR model, which outputs the corresponding text sequence, as shown below:

[0313] {(T1,w1),(T2,w2),…,(T n ,w n )};

[0314] Finally, the formula recognition results and text recognition results are spliced ​​together in the original order of the regions to restore the complete test content.

[0315] Step S4: Combining the characteristics of mathematical grammar, the multi-feature similarity calculation method based on the BLEU algorithm, and fuzzy matching technology, the candidate's answer is compared with the standard answer step by step to ensure the scientific nature and flexibility of the scoring.

[0316] Step S41: Step-by-step analysis of the mathematical solution.

[0317] Step S411: Construct a mathematical symbol dictionary. This dictionary should cover the following specific domain-specific mathematical symbols:

[0318] a. Function symbols: such as sin, cos, tan, ln, e x ;

[0319] b. Operators: such as +, -, ×, ÷, =;

[0320] c. Variables and constants: such as x, y, z, π, e;

[0321] d. Mathematical terms: integral, derivative, limit, etc.

[0322] Step S412: The reverse maximum matching algorithm based on a custom mathematical symbol dictionary is used to parse the handwritten mathematical formula text step by step. The specific word segmentation process is as follows:

[0323] a. Input: Steps to solve a math problem;

[0324] b. Scanning: Scanning character by character from back to front;

[0325] c. Match: Prioritizes matching the longest word in the dictionary; if no match is found, returns a single character.

[0326] Step S413: Construct a mathematical syntax tree. Based on the word segmentation results, generate a tree structure suitable for formula calculation and logical derivation by parsing the priority and logical relationship between symbols. This efficiently expresses the hierarchical relationship and operation steps of the formula, providing a solid foundation for subsequent step-by-step parsing and similarity calculation.

[0327] The input consists of a sequence of segments representing mathematical formulas, including operators, parentheses, variables, and constants. The priority order of the symbols (from highest to lowest) is: function > parentheses > exponent > multiplication and division > addition and subtraction.

[0328] Step S414: Extract the calculation logic from the formula layer by layer through the structure of the syntax tree, decompose the complete mathematical solution into independent logical operations, that is, break down a complex mathematical formula step by step and process it according to the rules of mathematical operations (such as "multiplication and division before addition and subtraction") until the final answer is obtained.

[0329] For example: When given the formula x + y * z, calculate y * z first, then add it to x. The steps after parsing are as follows:

[0330] (1) Calculate y*z;

[0331] (2) Calculate x + (y * z).

[0332] Step S42: Using a multi-feature similarity calculation method based on the BLEU algorithm, combined with the grammatical characteristics of mathematical formulas, the similarity between the candidate's answer and the standard answer is evaluated to achieve efficient comparison.

[0333] Step S421: Calculate character-level similarity.

[0334] First, the formula strings of the candidates' answers and the standard answers are decomposed at the character level, parsing the formulas into a sequence {s1,s2,…,s} consisting of symbols and variables. n For example, the formula a+b=c can be decomposed into ["a","+","b","=","c"].

[0335] The BLEU algorithm is used to perform step-by-step matching of character sequences and calculate the character-level n-gram similarity S. char As shown below:

[0336]

[0337] Where N is the maximum value of the calculated n-gram; w n Let be the weighting coefficient, satisfying p n This represents the overlap ratio between the candidate's answer and the standard answer under the n-gram.

[0338] Step S422: Calculate structural similarity.

[0339] Based on the mathematical syntax tree constructed in step S413, the structural similarity of each formula is further evaluated. The syntax trees of the test takers' answers and the standard answers are matched point by point to determine whether the symbol types and logical relationships of each node are consistent. Based on the similarity of each formula, the structural similarity S is calculated. struct As shown below:

[0340]

[0341] Match(A,B) represents the nodes that match the standard answer A and the student's answer B; sum(A) represents the total number of nodes in the standard answer.

[0342] Because each test taker has different answering habits, there may be cases where the variable names differ from the standard answer, but the actual solution steps are correct. To address this, a symbol substitution and equivalent expression mechanism is introduced. Fuzzy matching is used to tolerate differences in the representation of different symbols, addressing the differences between variables and constants. For example, in a formula, the positions of x and y can be interchanged, or equivalent expressions can be used.

[0343] Determine if mappings are equivalent, allowing matching of variables with different symbol names but consistent context, such as x+y matching a+b; allowing expressions that satisfy the commutative law, such as a+b matching b+a.

[0344] Step S423: Calculation of the score for logical order.

[0345] When comparing the standard answer and the student's answer, it was found that the standard answer might provide n steps, while the student's solution might be more concise, containing only n-1 steps. However, the student had accurately grasped the key knowledge points in their solution and ultimately arrived at the correct result. If the similarity calculation only performs content matching while ignoring the logic of the solution steps and the inconsistency of the steps, it will have a significant impact on the final result.

[0346] Therefore, to improve the accuracy of calculations, a logical order similarity calculation method needs to be introduced. This method verifies the logicality of the problem-solving steps and the equivalence between them, comprehensively considering the similarity of the answers, thereby effectively reducing errors and ensuring the fairness and accuracy of scoring. The logical order similarity S is calculated. logic (A,B), as shown below:

[0347]

[0348] Where sam(A,B) represents the number of matches between the standard answer and the candidate's answer; MaxRev(A,B) represents the maximum number of inversions in a sequence of natural numbers with the same number of solution steps; and sum(A,B) represents the total number of solution steps in the standard answer.

[0349] Step S424: Calculation of scores for key steps.

[0350] To assess the consistency between test takers' answers and standard answers in terms of problem-solving steps, a scoring method based on key steps is proposed, taking into account the characteristics of subjective mathematics questions. This method ensures the accuracy and fairness of scoring by identifying and evaluating the matching of key steps in the problem-solving process.

[0351] Extract the crucial key steps k1, k2, ..., k from the standard answer. n And assign independent weights w1, w2, ..., w to each key step. n The total weights satisfy the following condition:

[0352]

[0353] Among them, w i Let P represent the weight of the i-th critical step, and P be the total number of critical steps.

[0354] The same rules as the standard answer are used to analyze the candidates' answers step by step, extracting potential key steps. These extracted steps are then matched with the key steps in the standard answer. If a step in the candidate's answer completely matches a key step in the standard answer, it is recorded as a complete match; if a step in the candidate's answer partially matches a key step in the standard answer (e.g., different solutions but the same result), it is recorded as a partial match; otherwise, it is recorded as a mismatch. The key step similarity S is calculated.key As shown below:

[0355]

[0356] Where, match(k) i (k) is the key step i The degree of matching in the candidates' answers, with the following range of values:

[0357] Exact match: k i =1; Partial match: k i =0.5; mismatch: k i =0.

[0358] Step S425: Calculate the multi-feature fusion score.

[0359] To ensure the final score is more accurate and reflects reality, the overall score is based on character-level similarity S. char Structural similarity S struct Logical order similarity S logic Similarity S to key steps key The result is obtained by weighted fusion of the similarity of the four features, as shown below:

[0360] S total =α1·S char +α2·S struct +α3·S logic +α4·S key ;

[0361] Among them, S total The final fusion score is α1, α2, α3, and α4 are the feature weights, which satisfy the following constraints:

[0362]

[0363] Among them, the preferred values ​​are α1 = 0.3, α2 = 0.3, α3 = 0.2, and α4 = 0.2.

[0364] Step S43: Handling multiple solutions to the standard answer.

[0365] In subjective math questions, the standard answer may have multiple solutions. To efficiently and fairly evaluate candidates' answers, a multi-solution processing method based directly on fusion scoring is proposed. By calculating the sum of scores for each solution in the candidate's answer and the standard answer, the solution with the highest score is selected as the final score.

[0366] Organize the possible solutions to the standard answer into a set {A1, A2, ..., A...} n}, each A iThis represents a complete solution path. Based on the multi-feature fusion scoring formula in step S425, the candidate's answer B and various solutions A are calculated respectively. i Overall score S total (B,A i ), as shown below:

[0367] S total (B,A i )=α1·S char +α2·S struct +α3·S logic +α4·S key ;

[0368] Then, select the standard solution A with the highest fusion score. best The final scoring criteria are as follows:

[0369]

[0370] Among them, A best The final standard solution chosen; S total (B,A i (A) is the candidate's answer and the standard solution A. i The overall score.

[0371] Therefore, this invention adopts the above-mentioned method for grading subjective questions on handwritten math exams based on image recognition and fuzzy matching. Combining deep learning models and fuzzy matching technology, it can efficiently and accurately grade subjective questions on handwritten math exams, solving the problems of complex formula recognition, multiple solution path matching, and logical derivation scoring. It significantly improves grading efficiency, reduces human intervention, and ensures the fairness and consistency of scoring. It is suitable for the automated grading needs of examinations in the education field and greatly reduces the workload of teachers.

[0372] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for grading subjective questions on handwritten math exam papers based on image recognition and fuzzy matching, characterized in that: Includes the following steps: Step S1: Acquire the test question image and perform preprocessing operations on the acquired test question image; Step S2: Construct a multi-scale recognition model for handwritten mathematical formulas based on an encoder-decoder. The specific process is as follows: Step S21, Encoder Design: The encoder is based on the ResNet-50 network, combined with the feature enhancement module FEM and the adaptive feature fusion module AFF to achieve multi-scale feature extraction; Step S211: Introduce the improved activation function Swish-ReLU after each residual block of ResNet to construct the feature enhancement module FEM; Step S212: Add the adaptive feature fusion module AFF to achieve the fusion of shallow and deep features in the handwritten mathematical formula; Step S22, Decoder Design: The decoder decodes the multi-scale feature map generated by the encoder into a symbol sequence. The decoder adopts a bidirectional GRU structure and combines an enhanced attention mechanism (EAM) to model the symbol position and contextual relationship. Step S221: The decoder directly uses the fused features output by the encoder as input; Step S222: Enhanced Attention Mechanism (EAM) is implemented by calculating attention weights. The position that contributes most to the prediction of the current symbol in the encoder feature map is selected, and the learnable position encoding is combined to enhance the modeling of the relationship between symbols. Step S223, Symbol Prediction: Combining Context Vectors and the current hidden state The probability distribution of generated symbols; Step S224, Symbol parsing and formula structure generation: The symbol sequence generated by the decoder is parsed into a complete mathematical formula tree; Step S23: Train and optimize the constructed encoder-decoder-based multi-scale recognition model for handwritten mathematical formulas; Step S3: Based on the multi-scale recognition model and OCR model constructed in step S2, identify the test question area; Step S4: Combining mathematical grammar characteristics, the multi-feature similarity calculation method based on the BLEU algorithm, and fuzzy matching technology, the candidate's answer and the standard answer are compared step by step. The specific process is as follows: Step S41: Define a custom mathematical symbol dictionary and use the reverse maximum matching algorithm to parse the handwritten mathematical formula text step by step; construct a mathematical syntax tree and generate a tree structure suitable for formula calculation and logical derivation based on the word segmentation results to express the hierarchical relationship of the formula and the operation steps; extract the calculation logic in the formula layer by layer through the structure of the syntax tree, decompose the complete mathematical solution into independent logical operations, and realize the step-by-step parsing of the mathematical solution. Step S42: Employ a multi-feature similarity calculation method based on the BLEU algorithm, combining the grammatical characteristics of mathematical formulas, and evaluate the similarity between the candidate's answer and the standard answer through multi-feature fusion scoring calculation to achieve efficient comparison; wherein, multi-feature similarity includes: character-level similarity, structural-level similarity, logical order similarity, and key step similarity; Step S43: For cases with multiple solutions to the standard answer, based on the multi-solution processing method of fusion scoring, the solution with the highest score is selected as the final score by calculating the sum of the scores of the candidate's answer and each solution to the standard answer.

2. The method for grading subjective questions in handwritten mathematics exam papers based on image recognition and fuzzy matching according to claim 1, characterized in that, In step S1, the test question image is acquired, and the acquired test question image is preprocessed. The specific process is as follows: Step S11: Acquisition and segmentation of the test question image; First, the candidates' paper exam papers are converted into electronic image format using a scanner; then, the acquired exam question images are segmented according to the question area, and the subjective question area is extracted. The coordinates of the question area are the coordinates used when the answer sheet was made, and the cutting is done by scaling the scanned image proportionally according to the anchor points in the answer sheet. Step S12: Perform preprocessing operations on the acquired test question images, specifically including grayscale conversion, binarization, image noise reduction, tilt correction, and image enhancement.

3. The method for grading subjective questions in handwritten mathematics exams based on image recognition and fuzzy matching according to claim 1, characterized in that, In step S21, the encoder is designed, where the input to the encoder is an image of a handwritten mathematical formula, and the output is multi-scale features. The specific process is as follows: Step S211: Introduce the improved activation function Swish-ReLU after each residual block of ResNet to construct the Feature Enhancement Module (FEM), which specifically includes: (1) The activation function is as follows: ； in, This represents a single value at a certain location in the feature map output by each residual block convolutional layer in the encoder, i.e., the input value; It is a smooth nonlinear function; These are parameters that determine the smoothness of the activation function; This is the scaling factor, used to scale the output in the negative value range; (2) In In the upper part of the activation function, a smooth nonlinear function is used. , will input Compress to Within the range, as shown below: ； Based on this, multiply by This makes the activation function continuous and smooth within the positive range; (3) In In the lower half of the activation function, a scaling factor is introduced for negative input values. Retain negative value information; Step S212: Add the Adaptive Feature Fusion (AFF) module to achieve the fusion of shallow and deep features in the handwritten mathematical formula, specifically including: (1) Input features: The feature maps of the encoder are output by different convolutional layers, as shown below: ； in, For the first Layer features, dimension 1 ; (2) The importance of the feature layer is dynamically adjusted by using the Softmax function, the dynamic weights are calculated, and the fusion effect of multi-scale features is optimized; A. Feature map dimensionality reduction: Each layer of feature maps It is a multidimensional tensor, for Applying global average pooling, we obtain a scalar, as shown below: ； in, Representation of feature map Global features; It is a feature map In position The value at; Indicates the index of the feature map in the vertical direction; Indicates the horizontal index of the feature map; B. Dynamic Adjustment Coefficient Used to control the importance of each feature layer, the coefficients are automatically optimized based on the data to dynamically adjust the weight distribution of the feature layers; C. Softmax Normalization: [This part is incomplete and likely refers to a function or method.] As weight factors for the feature layers, the final weights are calculated by normalizing all layers using the Softmax function. As shown below: ； in, Indicates the first Dynamic adjustment coefficients for layer features; Indicates the first Dynamic adjustment coefficients for layer features; Indicates the first Global information of the layer feature map; Indicates the first Global information of the layer feature map; This represents the total number of feature layers; (3) Feature fusion: Multi-layer features are fused according to dynamic weights Weighted summation generates a unified feature across multiple scales. ; First, for each layer of features According to the corresponding dynamic weight The weighting is performed as follows: ； in, ; Subsequently, all weighted feature maps are accumulated and fused layer by layer to generate the final multi-scale features. As shown below: ； Finally, the encoder outputs the fused features. .

4. The method for grading subjective questions in handwritten math exams based on image recognition and fuzzy matching according to claim 1, characterized in that, In step S22, the decoder decodes the multi-scale feature map generated by the encoder into a symbol sequence. The decoder adopts a bidirectional GRU structure and combines an enhanced attention mechanism (EAM) to model the symbol positions and contextual relationships. The specific process is as follows: Step S221: The decoder directly uses the fused features output by the encoder. As input; (1) Forward GRU time-step processing Generate the forward hidden state sequence as follows: ； in, This represents the hidden state generated by the forward GRU at time step t; This represents the total number of time steps; the forward hidden state sequence contains context information from the first T time steps. (2) Reverse processing of backward GRU Generate the backward hidden state sequence as follows: ； in, This represents the hidden state generated by the backward GRU at time step t; the backward hidden state sequence contains context information from time step T to 1; (3) Concatenate the forward and backward hidden states to generate the complete hidden state. As shown below: ； in, This represents the complete hidden state after splicing at time step t; This indicates a splicing operation, and the hidden state after splicing contains the contextual information of the fused features in both the positive and negative directions. Step S222: Enhanced Attention Mechanism (EAM) is implemented by calculating attention weights. The position that contributes most to the prediction of the current symbol in the encoder feature map is selected, and the learnable position encoding is combined to enhance the modeling of the relationship between symbols. Attention weight Indicates the decoder at time step The degree of attention given to each position of the encoder output is as follows: ； in, It is the current time step The hidden state; It is a trainable weight matrix; It is a learnable positional encoding used to enhance the decoder's awareness of symbol positions; Learnable positional encoding As shown below: ； in, It is the current time step The relative position vector; It is a positional encoding weight matrix; Based on attention weight The encoder features are weighted and summed to generate a context vector. As shown below: ； in, This indicates the number of spatial dimensions in the encoder's output feature map; An index representing a spatial dimension; Step S223, Symbol Prediction: Combining Context Vectors and the current hidden state The probability distribution for generating symbols is shown below: ； in, It is a trainable matrix used to map hidden states to symbol classes at each time step. Output symbols Finally, a complete symbol sequence is generated. ; Step S224, symbol resolution and formula structure generation, specifically includes: (1) Operator precedence resolution: sort the operators in the symbol sequence according to the mathematical operation rules; (2) Determine the superscript and subscript relationship: Based on the position of the symbol in the sequence and combined with the position encoding information, determine the superscript and subscript relationship of the symbol; (3) Constructing tree-like expressions: Organizing symbols into a tree structure according to logical relationships to represent the hierarchical relationship of the formula; (4) Recursive traversal of the output structure: By traversing the tree expression, the tree structure is reorganized into a linear output form; After the above steps, the symbol sequence generated by the decoder is parsed into a complete mathematical formula tree.

5. The method for grading subjective questions in handwritten mathematics exam papers based on image recognition and fuzzy matching according to claim 1, characterized in that, In step S23, the constructed encoder-decoder-based multi-scale recognition model for handwritten mathematical formulas is trained and optimized. The specific process is as follows: Step S231, Data Preparation: Collect a large number of image samples containing handwritten formulas, and generate a corresponding mathematical formula LaTeX format expression for each handwritten formula image; Among them, image data is augmented by randomly combining and sampling multiple augmentation methods such as rotation, scaling, translation, affine transformation, noise addition, brightness adjustment, and contrast adjustment. Step S232, Model Training, specifically includes: First, divide the dataset according to The proportions are divided into training set and validation set; Then, the Adam optimizer is used, with the initial learning rate set to... The learning rate is dynamically adjusted using a cosine annealing learning rate adjuster, as shown below: ； in, This is the current learning rate; Minimum learning rate; The initial learning rate, This is the maximum number of training rounds; This is the current training round; To ensure the accuracy of the predicted symbols and the rationality of the logical structure, two loss functions were designed: character-level cross-entropy loss and structure matching loss, as shown below: (1) Character-level cross-entropy loss is used to supervise symbol sequence generation, ensuring that the predicted results match the true symbol sequence character by character, as shown below: ； in, Indicates the total length of the symbol sequence; Indices representing the sequence of symbols; It is the first The true label of a symbol; It is the model's prediction of the first The probability distribution of each symbol; (2) The structural matching loss evaluates the logical consistency of the formula by the matching degree of the syntax tree. It is suitable for evaluating the priority, nesting relationship and semantic logic between symbols, as shown below: ； in, It is the prediction syntax tree generated by the model; It is the syntax tree of the standard answer; It is the number of matching nodes in the syntax tree; It is the total number of nodes in the standard answer syntax tree; (3) In order to balance the accuracy of symbol prediction and the consistency of logical structure, a weighted combination loss function is used during training, as shown below: ； in, and This is a weighting hyperparameter used to adjust the contribution ratio of the two losses.

6. The method for grading subjective questions in handwritten mathematics exams based on image recognition and fuzzy matching according to claim 1, characterized in that, In step S3, the question area is identified based on the multi-scale recognition model of handwritten mathematical formulas and the OCR model. The specific process is as follows: Step S31, Formula Region Detection: Use the constructed multi-scale recognition model to directly detect the regions of handwritten formulas in the test paper and directly output the coordinates of the formula regions. sequence ;in, The coordinates of the top-left corner of the region; They are width and height, respectively; This represents the total number of sequences of coordinates in the formula region; Step S32, Region Merging and Optimization, specifically includes: First, determine proximity relationships and calculate the spatial distance between detection boxes; for every two detection boxes... and Calculate their boundary distance As shown below: ； in, and Detection boxes and The coordinates of the right and left boundaries; Then, set a distance threshold. ,if If the two detection boxes belong to the same formula, then they are considered to belong to the same formula. Finally, the formula area is masked; Step S33, Text Region Detection: Utilize an OCR model to directly detect the image area occluded by the formula region, directly outputting the range of the text region, which is consistent with the formula region, and outputting the coordinates of the text region. sequence ;in, and This indicates the coordinates of the top-left corner of the text area; and Indicates the width and height of the respective text areas; Step S34: Recognize the formulas and text, specifically including: First, the image is segmented according to the formula region sequence and the text region sequence. The segmented formula region images are then input into a multi-scale recognition model, which outputs the corresponding formula region sequence, as shown below: ； Then, the segmented text region image is input into the OCR model, which outputs the corresponding text sequence, as shown below: ； Finally, the formula recognition results and text recognition results are spliced ​​together in the original order of the regions to restore the complete test content.

7. The method for grading subjective questions in handwritten mathematics exam papers based on image recognition and fuzzy matching according to claim 1, characterized in that, In step S4, combining mathematical grammar characteristics, multi-feature similarity, and fuzzy matching techniques, the candidate's answers are compared with the standard answers step by step. The specific process is as follows: Step S41, step-by-step analysis of the mathematical solution, specifically including: Step S411: Construct a dictionary of mathematical symbols; Step S412: The reverse maximum matching algorithm based on a custom mathematical symbol dictionary is used to parse the handwritten mathematical formula text step by step; the specific word segmentation process is as follows: a. Input: Steps to solve a math problem; b. Scanning: Scanning character by character from back to front; c. Match: Prioritizes matching the longest word in the dictionary; if no match is found, returns a single character. Step S413: Based on the word segmentation results, construct a mathematical syntax tree by parsing the priority and logical relationship between symbols, and generate a tree structure suitable for formula calculation and logical derivation; Step S414: Extract the calculation logic from the formula layer by layer through the structure of the syntax tree, decompose the complete mathematical solution into independent logical operations, until the final answer is obtained; Step S42: Using a multi-feature similarity calculation method based on the BLEU algorithm, combined with the grammatical properties of mathematical formulas, the similarity between the candidate's answer and the standard answer is evaluated. Specifically, this includes: Step S421: Perform character-level decomposition on the formula strings of the candidate's answer and the standard answer, parsing the formulas into sequences composed of symbols and variables; then, use the BLEU algorithm to perform step-by-step matching of the character sequences and calculate the character-level n-gram similarity. As shown below: ； in, The maximum length of the n-gram is calculated; Indicates the length index of the n-gram; For the weighting coefficients, satisfying ; The percentage overlap between the candidate's answer and the standard answer in the n-gram format; Step S422: Perform point-by-point matching between the syntax trees of the candidate's answer and the standard answer, determine whether the symbol types and logical relationships of each node are consistent, and calculate the structural similarity based on the similarity of each formula. As shown below: ； in, For the standard answer and student answers Matching nodes in; This represents the total number of nodes in the standard answer. Step S423: Introduce a logical order similarity calculation method to ensure the fairness and accuracy of scoring by verifying the logicality of the problem-solving steps and the equivalence between them; among which, the logical order similarity is calculated. As shown below: ； in, This indicates the number of matches between the standard answer and the test taker's answer; The maximum number of inversions in a sequence of natural numbers that have the same number of solution steps; This indicates the total number of steps in the solution provided in the standard answer. Step S424: Extracting crucial key steps from the standard answer. And assign independent weights to each key step. The total weights satisfy the following condition: ； in, Indicates the first The weight of each key step; Total number of critical steps; The test takers' answers are analyzed step-by-step using the same rules as the standard answers to extract potential key steps. These extracted steps are then matched with the key steps in the standard answers to calculate the key step similarity. As shown below: ； in, Key steps The degree of matching in the candidates' answers, with the following range of values: Exact match: Partial match: ; Mismatch: ; Step S425: The overall score is based on character-level similarity. Structural similarity Logical order similarity Similarity to key steps The result is obtained by weighted fusion of the similarity of the four features, as shown below: ； in, The final combined total score; The weights are the feature weights, and the weights satisfy the following constraints: ； Step S43: Processing multiple solutions to the standard answer. By calculating the total score of each solution to the candidate's answer and the standard answer, the solution with the highest score is selected as the final score. The various solutions to the standard answer are compiled into a set. Each This represents a complete solution path; based on the multi-feature fusion scoring formula in step S425, calculate the examinee's answer respectively. With various solutions Overall rating As shown below: ； Then, select the standard solution with the highest fusion score. The final scoring criteria are as follows: ； in, This is the standard solution that was ultimately chosen. The candidate's answer and the standard solution The overall score.

Images