Student handwritten form recognition system and method based on YOLOv11 and EasyOCR fusion

Abstract

The invention relates to the technical field of crossing of an artificial intelligence technology and an education digitization technology, in particular to a student handwritten form recognition system and method based on fusion of YOLOv11 and EasyOCR, and the system comprises a front-end interaction module, a rear-end processing module and a local database module. The system comprises an image uploading and previewing area, a recognition process control area and a recognition result display area and is used for receiving handwritten test paper images uploaded by a user, displaying image previews and recognition results and supporting the recognition results to be exported as CSV files. The bottlenecks of missing detection of small targets, weak recognition of connected characters and module error accumulation are broken through, and the recognition precision is improved. A special education data set is constructed, lightweight design is adapted to common equipment, subjective question marking automation is realized, the burden of teachers is reduced, structured storage data supports precise teaching, and the requirements of double reduction and education digitization are met.

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of artificial intelligence and digital education technology, specifically to a student handwriting recognition system and method based on the integration of YOLOv11 and EasyOCR. Background Technology

[0002] Digitalization of education has become an important breakthrough for opening up new tracks for educational development and shaping new advantages for development. Automated marking and handwriting recognition have become key needs in the education field.

[0003] However, traditional handwriting recognition technology is limited by rule matching and struggles to handle complex scenarios such as Chinese handwriting deformation and mixed question types. Current research faces several bottlenecks: a lack of dedicated handwriting datasets for educational scenarios results in weak generalization ability of models in actual teaching situations; insufficient detection accuracy for unstructured answer areas leads to missed detections and false detections; handwriting recognition is not robust to changes in writing style, with cursive writing and corrections significantly reducing recognition accuracy; and there is a significant problem of error accumulation between recognition modules. Furthermore, existing research mostly focuses on character-level recognition and lacks a semantic association mechanism between questions and answers, failing to meet actual grading needs. Therefore, it is necessary to propose a student handwriting recognition system and method based on the fusion of YOLOv11 and EasyOCR. Summary of the Invention

[0004] To address the problems in the existing technology, this invention provides a student handwriting recognition system and method based on the fusion of YOLOv11 and EasyOCR.

[0005] The technical solution adopted by this invention to solve its technical problem is: a student handwriting recognition system based on the fusion of YOLOv11 and EasyOCR, including a front-end interaction module, a back-end processing module, and a local database module. The front-end interaction module is built on the Streamlit framework and includes an image upload and preview area, a recognition process control area, and a recognition result display area. It is used to receive handwritten test paper images uploaded by users, display image previews and recognition results, and supports exporting recognition results to CSV files. The backend processing module includes a preprocessing unit, a YOLOv11 target detection unit, and an EasyOCR recognition unit; The preprocessing unit is used to perform format normalization, noise suppression, adaptive binarization, and perspective correction on the uploaded image. The YOLOv11 object detection unit employs an improved CSP-DarkNet backbone network. It optimizes feature extraction through a cross-stage feature reorganization formula, introduces an SPPF feature fusion module and depthwise separable convolution, and generates custom anchor boxes using a K-means clustering loss formula. A confidence-weighted selection formula is used to retain high-confidence detection results, enabling the detection of question number regions, character boxes, and question box boundaries in the exam paper, outputting bounding boxes containing the target coordinates. The cross-stage feature reorganization formula is as follows: ,in This is the input feature map after the CSP module is separated. These are the feature weight coefficients. For convolution operations, For batch normalization, It is the ReLU activation function. For bias terms, The recombined feature map is output. The K-means clustering loss formula is as follows: ,in The number of clusters. For the first A set of target bounding boxes for each cluster. The width and height of a single target bounding box within the cluster. For the first The center of a cluster; The confidence-weighted screening formula is as follows: ,in For the final confidence level, Predict the initial confidence level for the model. These are the weighting coefficients. For the intersection-union comparison of the predicted bounding box and the labeled bounding box, It is the minimum value; The EasyOCR recognition unit uses a CNN+BiLSTM+CTC architecture as its core, employs an improved ResNet-34 as its front-end backbone network, strengthens the temporal correlation of cursive characters through a temporal dependency weight formula, and corrects common writing errors using a rule-based confidence formula. It then performs handwritten character recognition on the character regions detected by YOLOv11. The temporal dependency weight formula is as follows: , ,in For the current moment, For a historic moment, For BiLSTM hidden state, This is the attention weight matrix. To hide the state dimension, The length of the character sequence; The formula for the rule base correction confidence level is: ,in To correct the confidence level, As the initial identification confidence level, For correction factors, To identify the similarity between the text and the standard text; The local database module uses an SQLite database and creates an answer_results table to store the recognition results. It supports querying by question number and time, and uses a transaction mechanism to control data writing. The backend processing module also optimizes the propagation of detection-recognition errors through a cross-module attention alignment formula, which is: ,in The number of questions. For the first Question number feature vector For the first The eigenvectors of the answer to the question.

[0006] Specifically, the improved CSP-DarkNet backbone network of the YOLOv11 object detection unit separates convolutional layer channels and reassembles features through the CrossStagePartial mechanism. The value range is [0.3, 0.7]; the SPPF feature fusion module uses four pooling kernels of different scales (1×1, 2×2, 3×3, 6×6); the depthwise separable convolution replaces the standard convolutions in layers 5-8, reducing the number of model parameters by 30%; the number of K-means clusters... The sample size is set to 9, and the sample size is taken from 32,000 handwritten annotation boxes. The iteration count is 100. In the confidence-weighted screening formula... Set it to 0.6. The threshold is set to 0.8. .

[0007] Specifically, the two-stage annotation strategy of the YOLOv11 object detection unit includes: the first stage uses the LabelImg tool to annotate three types of targets in handwritten text: title area, character box, and title box boundary, generating YOLO format annotation files, accumulating 32,000 target boxes, covering 12,000 handwritten text images; the second stage uses the K-means clustering algorithm to generate custom anchor boxes with a height-to-width ratio of 1:1.2 for Chinese handwritten characters; the inference process includes: resizing the preprocessed image to 640×640, using the feature pyramid network to detect small characters of 16×16 pixels, medium characters of 32×32 pixels, and large characters of 64×64 pixels on feature maps of three different scales, filtering overlapping boxes through non-maximum suppression with a threshold of 0.5, and retaining detection results with a confidence level ≥0.8.

[0008] Specifically, the domain-adaptive training strategy of the EasyOCR recognition unit includes: adding ±15° rotation and scaling of 0.8-1.2 times to the original character image. Gaussian blur data enhancement perturbation of 1.5; freezing the first 10 layers of the CNN backbone network, and only fine-tuning the BiLSTM and CTC layers; training using a dedicated dataset for educational scenarios containing 100,000+ characters; the time-dependent weight formula in the above formula. =256, =40; in the rule base correction confidence formula Set it to 0.3. The distance is calculated using edit distance, ranging from [0,1].

[0009] Specifically, the backend processing module employs an asynchronous loading mechanism to optimize the model and data processing flow, avoids redundant calculations through a state caching mechanism, and supports batch processing of multiple graphs; the frontend interaction module utilizes `st.session_state` for persistent state management and uses the `@st.cache_data` decorator to optimize preprocessing and model inference; the cross-module attention alignment formula... , All dimensions are 128.

[0010] Specifically, the fields of the answer_results data table include id (primary key), image_id (image unique identifier UUID), question_number (question number), recognized_text (recognized content), confidence (recognition confidence level), bbox (character region coordinates), timestamp (recognition time), and is_corrected (whether it was manually modified).

[0011] Implementation of a student handwriting recognition system based on the fusion of YOLOv11 and EasyOCR: S1. Data Preparation: Construct a dataset containing 12,000 character regions, covering various question types, font styles, and interference factors. Use a two-stage annotation mechanism to generate YOLO format detection annotation files and OCR recognition annotation files, and divide the training set and validation set in an 8:2 ratio. S2. Model Training: Train the YOLOv11 object detection model, implemented in PyTorch. Initialize the backbone network with pre-trained weights and use cross-stage feature recombination formulas. Optimize feature extraction and combine it with the K-means clustering loss formula. Generate custom anchor boxes, setting the input size to 640×640, batch size to 16, initial learning rate to 0.001 (cosinedecay), optimizer to SGD (momentum=0.937), number of epochs to 100, and loss function to CIoULoss+BCELoss, combined with a warm-up mechanism and Mosaic data augmentation; fine-tune the EasyOCR recognition model, freeze the CNN feature extraction layer, and apply a time-dependent weight formula. , To enhance the recognition of cursive characters, the batch size was set to 32, the learning rate to 0.0005 (fixed), the optimizer to Adam, the number of epochs to 80, the loss function to CTCLoss, and the EarlyStopping strategy was adopted. S3. Image Upload and Preprocessing: Users upload handwritten test paper images through the front-end interaction module, and the back-end processing module performs format normalization, noise suppression, adaptive binarization and perspective correction on the images. S4. Character Region Detection: The YOLOv11 object detection unit performs multi-scale inference on the preprocessed image and uses a confidence-weighted selection formula. Retain high-confidence results and output bounding boxes containing question numbers and character region coordinates; S5. Handwritten Character Recognition: The EasyOCR recognition unit extracts and preprocesses character regions, and corrects the confidence formula using a rule base. Correct common errors and perform character recognition; S6. Result Optimization and Storage: Through the cross-module attention alignment loss formula To optimize detection and recognition errors, the "question number-answer" key-value pairs are stored in a structured SQLite database, and the recognition results are displayed on the front end. Exporting CSV files is also supported.

[0012] Specifically, in step S1, the YOLO format annotation file is generated using the LabelImg tool, with each line containing...<class_id><x_center><y_center> <width> <height>(Relative normalized coordinates); The OCR recognition annotation file is in .csv or .json format, containing image path, recognized text, character length information, and the recognized text is processed according to UTF-8 encoding standard, simplified and traditional Chinese characters, and full-width and half-width numbers.

[0013] Specifically, in step S2, the warm-up period for YOLOv11 training is the first 5 epochs, with the learning rate gradually increasing from 0.0001 to 0.001; the EarlyStopping strategy for EasyOCR training is to automatically stop training if the accuracy on the validation set does not improve after 10 consecutive epochs. The training data comes from cropped images of YOLOv11 detection boxes, combined with manual annotations and enhancements such as rotation and noise addition. Specifically, in step S4, the three scale feature maps of the YOLOv11 feature pyramid network correspond to downsampling rates of 1 / 8, 1 / 16, and 1 / 32 of the input image, respectively, with a non-maximum suppression threshold of 0.5; in step S5, EasyOCR's character region preprocessing includes resizing to 32×128 pixels, grayscale stretching, and threshold segmentation; in step S6, the cross-module attention alignment loss formula... This represents the total number of questions identified in a single scan. Extracted from YOLOv11 detection bounding boxes. Extracted from EasyOCR recognition results.

[0014] The beneficial effects of this invention: The student handwriting recognition system and method based on the fusion of YOLOv11 and EasyOCR described in this invention overcomes bottlenecks such as missed detection of small objects, weak robustness of cursive handwriting recognition, and accumulation of module errors through formulaic designs such as cross-stage feature recombination, custom anchor box clustering, and cross-module attention alignment, significantly improving recognition accuracy and stability. In terms of scenario adaptation, a dedicated educational dataset covering diverse question types and handwriting styles is constructed. Combined with a lightweight design, it can be deployed on ordinary teaching equipment, reducing the computational barrier. In terms of practical value, it automates the entire process of subjective question grading, significantly reducing the burden on teachers. Simultaneously, the structured storage of recognition data provides support for precise teaching analysis, aligning with the "double reduction" policy and the needs of digital education. Attached Figure Description

[0015] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0016] Figure 1 This is a flowchart illustrating the student handwriting recognition system and method based on the fusion of YOLOv11 and EasyOCR provided by the present invention. Detailed Implementation

[0017] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0018] like Figure 1 As shown, the present invention provides the following technical solution: Example: A student handwriting recognition system based on the fusion of YOLOv11 and EasyOCR, including a front-end interaction module, a back-end processing module, and a local database module: The front-end interaction module is built on the Streamlit framework and includes an image upload and preview area, a recognition process control area, and a recognition result display area. It is used to receive handwritten test paper images uploaded by users, display image previews and recognition results, and supports exporting recognition results to CSV files. The back-end processing module includes a preprocessing unit, a YOLOv11 target detection unit, and an EasyOCR recognition unit; The preprocessing unit is used to perform format normalization, noise suppression, adaptive binarization, and perspective correction on the uploaded image; The YOLOv11 object detection unit employs an improved CSP-DarkNet backbone network. It optimizes feature extraction through a cross-stage feature reorganization formula, introduces an SPPF feature fusion module and depthwise separable convolution, and generates custom anchor boxes using a K-means clustering loss formula. A confidence-weighted selection formula retains high-confidence detection results, enabling the detection of question number regions, character boxes, and question box boundaries in the exam paper, outputting bounding boxes containing the target coordinates. The cross-stage feature reorganization formula is as follows: ,in , This is the input feature map after the CSP module is separated. , These are the feature weight coefficients. For convolution operations, For batch normalization, It is the ReLU activation function. For bias terms, The recombined feature map is output. The K-means clustering loss formula is: ,in The number of clusters. For the first A set of target bounding boxes for each cluster. The width and height of a single target bounding box within the cluster. For the first The center of a cluster; The confidence-weighted screening formula is: ,in For the final confidence level, Predict the initial confidence level for the model. These are the weighting coefficients. For the intersection-union comparison of the predicted bounding box and the labeled bounding box, It is the minimum value; The EasyOCR recognition unit uses a CNN+BiLSTM+CTC architecture as its core, employing an improved ResNet-34 as its front-end backbone network. It strengthens the temporal correlation of cursive characters through a temporal dependency weight formula, and corrects common writing errors using a rule-based confidence formula. It then performs handwritten character recognition on the character regions detected by YOLOv11. The temporal dependency weight formula is as follows: , ,in For the current moment, For a historic moment, , For BiLSTM hidden state, This is the attention weight matrix. To hide the state dimension, The length of the character sequence; The formula for the confidence level of rule base correction is: ,in To correct the confidence level, As the initial identification confidence level, For correction factors, To identify the similarity between the text and the standard text; The local database module uses an SQLite database and creates an answer_results table to store the recognition results. It supports querying by question number and time, and uses a transaction mechanism to control data writing. The backend processing module also optimizes the propagation of detection-recognition errors through a cross-module attention alignment formula. The attention alignment loss formula is as follows: ,in The number of questions. For the first Question number feature vector For the first The eigenvectors of the answer to the question.

[0019] Among them, the improved CSP-DarkNet backbone network of the YOLOv11 object detection unit separates convolutional layer channels and reassembles features through the CrossStagePartial mechanism. The value range is [0.3, 0.7]; the SPPF feature fusion module uses four pooling kernels of different scales (1×1, 2×2, 3×3, 6×6); depthwise separable convolutions replace the standard convolutions in layers 5-8, reducing the number of model parameters by 30%; K-means cluster number The sample size is set to 9, with 32,000 handwritten annotation boxes as the sample size, and the number of iterations is 100; the confidence-weighted selection formula is as follows: Set it to 0.6. The threshold is set to 0.8. .

[0020] The two-stage annotation strategy of the YOLOv11 object detection unit includes: the first stage uses the LabelImg tool to annotate three types of targets in handwritten text: title area, character box, and title box boundary, generating YOLO format annotation files, accumulating 32,000 target boxes, covering 12,000 handwritten text images; the second stage uses the K-means clustering algorithm to generate custom anchor boxes with a height-to-width ratio of 1:1.2 for Chinese handwritten characters; the inference process includes: resizing the preprocessed image to 640×640, using the feature pyramid network to detect small characters of 16×16 pixels, medium characters of 32×32 pixels, and large characters of 64×64 pixels on feature maps of three different scales, respectively, filtering overlapping boxes through non-maximum suppression with a threshold of 0.5, and retaining detection results with a confidence score ≥0.8.

[0021] The domain-adaptive training strategy of the EasyOCR recognition unit includes: adding ±15° rotation, 0.8-1.2x scaling, and Gaussian blur data enhancement perturbation with σ=1.5 to the original character image; freezing the first 10 layers of the CNN backbone network and only fine-tuning the BiLSTM and CTC layers; training using a dedicated dataset for educational scenarios containing over 100,000 characters; and using time-dependent weight formulas. =256, =40; In the rule base correction confidence formula Set it to 0.3. The distance is calculated using edit distance, ranging from [0,1].

[0022] The backend processing module employs an asynchronous loading mechanism to optimize model and data processing workflows, avoids redundant calculations through state caching, and supports batch processing of multiple graphs. The frontend interaction module utilizes `st.session_state` for persistent state management and uses the `@st.cache_data` decorator to optimize preprocessing and model inference. The cross-module attention alignment formula includes... , All dimensions are 128.

[0023] The fields in the answer_results table include id (primary key), image_id (unique image identifier UUID), question_number (question number), recognized_text (recognized content), confidence (recognition confidence level), bbox (character region coordinates), timestamp (recognition time), and is_corrected (whether it was manually modified).

[0024] Implementation of a student handwriting recognition system based on the fusion of YOLOv11 and EasyOCR: S1. Data Preparation: Construct a dataset containing 12,000 character regions, covering various question types, font styles, and interference factors. Use a two-stage annotation mechanism to generate YOLO format detection annotation files and OCR recognition annotation files, and divide the training set and validation set in an 8:2 ratio. S2. Model Training: Train the YOLOv11 object detection model, implemented in PyTorch. Initialize the backbone network with pre-trained weights and use cross-stage feature recombination formulas. Optimize feature extraction and combine it with the K-means clustering loss formula. Generate custom anchor boxes, setting the input size to 640×640, batch size to 16, initial learning rate to 0.001 (cosinedecay), optimizer to SGD (momentum=0.937), number of epochs to 100, and loss function to CIoULoss+BCELoss, combined with a warm-up mechanism and Mosaic data augmentation; fine-tune the EasyOCR recognition model, freeze the CNN feature extraction layer, and apply a time-dependent weight formula. , To enhance the recognition of cursive characters, the batch size was set to 32, the learning rate to 0.0005 (fixed), the optimizer to Adam, the number of epochs to 80, the loss function to CTCLoss, and the EarlyStopping strategy was adopted. S3. Image Upload and Preprocessing: Users upload handwritten test paper images through the front-end interaction module, and the back-end processing module performs format normalization, noise suppression, adaptive binarization and perspective correction on the images. S4. Character Region Detection: The YOLOv11 object detection unit performs multi-scale inference on the preprocessed image and uses a confidence-weighted selection formula. Retain high-confidence results and output bounding boxes containing question numbers and character region coordinates; S5. Handwritten Character Recognition: The EasyOCR recognition unit extracts and preprocesses character regions, and corrects the confidence formula using a rule base. Correct common errors and perform character recognition; S6. Result Optimization and Storage: Through the cross-module attention alignment loss formula To optimize detection and recognition errors, the "question number-answer" key-value pairs are stored in a structured SQLite database, and the recognition results are displayed on the front end. Exporting CSV files is also supported.

[0025] In step S1, the YOLO format annotation file is generated using the LabelImg tool, with each line containing...<class_id><x_center><y_center> <width> <height>(Relative normalized coordinates); The OCR recognition annotation file is in .csv or .json format, containing image path, recognized text, character length information, and the recognized text is processed according to UTF-8 encoding standard, simplified and traditional Chinese characters, and full-width and half-width numbers.

[0026] In step S2, the warm-up period for YOLOv11 training is the first 5 epochs, with the learning rate gradually increasing from 0.0001 to 0.001. The EarlyStopping strategy for EasyOCR training automatically stops training if the accuracy on the validation set does not improve after 10 consecutive epochs. The training data comes from cropped images of YOLOv11 detection boxes, combined with manual annotations and enhancements such as rotation and noise addition. In step S4, the three scale feature maps of the YOLOv11 feature pyramid network correspond to downsampling rates of 1 / 8, 1 / 16, and 1 / 32 of the input image, respectively, with a non-maximum suppression threshold of 0.5. In step S5, EasyOCR's character region preprocessing includes resizing to 32×128 pixels, grayscale stretching, and threshold segmentation. In step S6, the cross-module attention alignment loss formula... This represents the total number of questions identified in a single scan. Extracted from YOLOv11 detection bounding boxes. Extracted from EasyOCR recognition results.

[0027] When using it, the following steps are included: The first step involves breaking down the entire system workflow. The system uses "image input → preprocessing → character detection → text recognition → result optimization → storage and display" as its core pipeline. Each step achieves efficient collaboration through modular design and formulaic optimization. The specific workflow is as follows: 1. Front-end image upload and initialization (user interaction layer): 2. Startup and Model Loading: After the teacher opens the system, the front end is automatically initialized based on the Streamlit framework, and the back end loads the YOLOv11 detection model and EasyOCR recognition model through the state caching mechanism (@st.cache_data decorator). The first loading takes about 30 seconds due to reading the pre-trained weights, and subsequent loading directly reuses the cache (time ≤ 2 seconds). 3. Image Upload and Preview: Teachers can select 1-5 handwritten test paper images (supporting JPG / PNG format, single image ≤10MB) through the front-end "Image Upload Area". The system automatically verifies the format and displays the original image preview through the st.image() function, while generating a unique identifier for each image (UUID as image_id) for subsequent result association; The second step is backend image preprocessing (data cleaning layer). The preprocessing unit standardizes common interferences (tilt, shadow, noise) in the test paper images. The process and key operations are as follows: 1. Format standardization: Automatically convert PNG format images to JPG and unify image color channels (RGB to grayscale) to avoid format differences affecting model inference; 2. Noise Suppression: Gaussian filtering (kernel_size=(3,3), sigmaX=1.0) is used to remove salt-and-pepper noise generated during scanning, improving character edge sharpness by 20%. 3. Adaptive Binarization: By using the cv2.adaptiveThreshold() function (adaptiveMethod=GAUSSIAN_C, blockSize=11, C=2), the threshold is dynamically adjusted according to the local brightness of the image to separate characters from the background, improving contrast by 35%. 4. Perspective Correction: The edges of the test paper are detected by Hough transform, and the perspective matrix is ​​calculated to correct the tilted image (supports tilt correction of ≤15°) to ensure that the answer area is not deformed and to provide a regular image for subsequent detection; The third step, YOLOv11 character region detection (target localization layer), achieves accurate localization of the "question number-answer area" based on the improved YOLOv11 model. The core process and formula application are as follows: 1. Image size adaptation: Resize the preprocessed image to 640×640 (standard input size of the model), and record the aspect ratio of the original image for subsequent coordinate mapping restoration; 2. Multi-scale feature inference: The Feature Pyramid Network (FPN) is used to detect targets of different sizes on feature maps at three scales (corresponding to downsampling rates of 1 / 8, 1 / 16, and 1 / 32 of the input image). Small-scale feature maps (1 / 8 downsampling): Detect small targets of 16×16~32×32 pixels (e.g., title "1.""(2)"); Mesoscale feature map (1 / 16 downsampling): detects medium-sized targets (e.g., answer areas for fill-in-the-blank questions) with a resolution of 32×32~64×64 pixels. Large-scale feature maps (1 / 32 downsampling): Detect large targets ≥64×64 pixels (such as the answer area of ​​essay questions); 3. Custom Anchor Box Matching: Based on K-means Clustering Loss Formula Nine sets of generated Anchor boxes (adapted to Chinese handwriting with a 1:1.2 aspect ratio) are matched with the detected target to improve the bounding box fit. 4. Confidence-based screening and post-processing: A confidence-weighted screening formula is used. Filter out low-confidence results and retain Detection boxes with an area ratio ≥ 0.8 are selected, and overlapping boxes are removed through non-maximum suppression (NMS, threshold 0.5). Finally, a set of bounding boxes containing "question number - character region coordinates" is output; Step 4: EasyOCR handwritten text recognition (content extraction layer). Based on the fine-tuned EasyOCR model, the detected character regions are converted into text. The core process and formula applications are as follows: 1. Character region cropping: According to the bounding box coordinates output by YOLOv11, independent character regions are cropped from the original image, resized to 32×128 pixels (the standard input size of EasyOCR), and at the same time, gray stretching (pixel values are mapped to 0 - 255) and threshold segmentation (removing residual noise) are performed; 2. Feature extraction and temporal modeling: Character features are extracted by improving the ResNet-34 backbone network (adding dilated convolutions), and then the temporal dependence weight formula is combined with the BiLSTM layer to capture the temporal associations of connected characters and strengthen the "stroke connection" features; 3. Text decoding and error correction: The initial recognition text and confidence are generated through the CTC decoder , and then the confidence formula is corrected by combining with a rule library to correct common errors (such as "以经” → "已经”). Among them, the text similarity is calculated through the edit distance, and finally the "question number - recognition text - " corresponding relationship is output; Step 5: Cross-module error optimization and result storage (data integration layer): 1. Error optimization: The semantic association between the question number feature vector and the answer feature vector is established through the cross-module attention alignment loss formula to correct the "question number - answer" mismatch caused by detection offset, and the overall question association accuracy is improved to 98.5%; 2. Data storage: Information such as "image_id - question number - recognition text - - bounding box coordinates - recognition time - is_corrected" is structured and stored in the answer_results table of the SQLite database, supporting the transaction mechanism (ensuring data write consistency) and querying by "question number / time"; 3. Front-end display and export: The front-end reads the results from the database and displays each record (including the question number, recognition text, ) through st.dataframe(). At the same time, a "Export CSV" button (supporting UTF-8 encoding) is provided, which is convenient for teachers to file or conduct subsequent analysis.

[0028] Implementation case data and effect verification: Using "junior high school Chinese subjective question grading" as a practical application scenario, 200 student handwritten test papers (including 300 fill-in-the-blank questions, 150 short answer questions, and 50 essay questions, covering styles such as neat handwriting, cursive writing, and corrections) were selected and tested on a regular teaching computer (Intel i7-12700H CPU, 16GB RAM, no dedicated GPU). Key data are as follows: Time spent on each step (single A4 exam paper, 300 DPI):

[0029] Recognition accuracy data (categorized by handwriting style):

[0030] YOLOv11 detection accuracy data (categorized by target type):

[0031] Teacher user experience data:

[0032] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of protection claimed by the present invention. The scope of protection of the present invention is defined by the appended claims and their equivalents.< / height> < / width> < / height> < / width>

Claims

1. A student handwriting recognition system based on the fusion of YOLOv11 and EasyOCR, comprising a front-end interaction module, a back-end processing module, and a local database module, characterized in that: The front-end interaction module is built on the Streamlit framework and includes an image upload and preview area, a recognition process control area, and a recognition result display area. It is used to receive handwritten test paper images uploaded by users, display image previews and recognition results, and supports exporting recognition results to CSV files. The backend processing module includes a preprocessing unit, a YOLOv11 target detection unit, and an EasyOCR recognition unit; The preprocessing unit is used to perform format normalization, noise suppression, adaptive binarization, and perspective correction on the uploaded image. The YOLOv11 target detection unit adopts an improved CSP-DarkNet backbone network, optimizes feature extraction through a cross-stage feature recombination formula, introduces an SPPF feature fusion module and depthwise separable convolution, generates custom anchor boxes by combining the K-means clustering loss formula, and retains high-confidence detection results through a confidence weighted screening formula, thereby realizing the detection of question number areas, character boxes and question box boundaries in the test paper, and outputting bounding boxes containing target coordinates; The cross-stage feature recombination formula is as follows: ,in This is the input feature map after the CSP module is separated. These are the feature weight coefficients. For convolution operations, For batch normalization, It is the ReLU activation function. For bias terms, The recombined feature map is output. The K-means clustering loss formula is as follows: ,in The number of clusters. For the first A set of target bounding boxes for each cluster. The width and height of a single target bounding box within the cluster. For the first The center of a cluster; The confidence-weighted screening formula is as follows: ,in For the final confidence level, Predict the initial confidence level for the model. These are the weighting coefficients. To perform an intersection-union comparison between the predicted bounding box and the labeled bounding box, It is the minimum value; The EasyOCR recognition unit uses a CNN+BiLSTM+CTC architecture as its core, employs an improved ResNet-34 as its front-end backbone network, strengthens the temporal correlation of cursive characters through a temporal dependency weight formula, and corrects common writing errors using a rule-based confidence formula. It then performs handwritten character recognition on the character regions detected by YOLOv11. The temporal dependency weight formula is as follows: , ,in For the current moment, For a historic moment, For BiLSTM hidden state, Here is the attention weight matrix. To hide the state dimension, The length of the character sequence; The formula for the rule base correction confidence level is: ,in To correct the confidence level, As the initial identification confidence level, For correction factors, To identify the similarity between the text and the standard text; The local database module uses an SQLite database and creates an answer_results table to store the recognition results. It supports querying by question number and time, and uses a transaction mechanism to control data writing. The backend processing module also optimizes the propagation of detection-recognition errors through a cross-module attention alignment formula, which is: ,in The number of questions. For the first Question number feature vector For the first The eigenvectors of the answer to the question.

2. The student handwriting recognition system based on the fusion of YOLOv11 and EasyOCR as described in claim 1, characterized in that: The improved CSP-DarkNet backbone of the YOLOv11 object detection unit separates convolutional layer channels and reassembles features through the CrossStagePartial mechanism. The value range is [0.3, 0.7]; the SPPF feature fusion module uses four pooling kernels of different scales; the depthwise separable convolution replaces the standard convolutions in layers 5-8, reducing the number of model parameters by 30%; the number of K-means clusters... The sample size is set to 9, and the sample size is taken from 32,000 handwritten annotation boxes. The iteration count is 100. In the confidence-weighted screening formula... Set it to 0.

6. The threshold is set to 0.

8. .

3. The student handwriting recognition system based on the fusion of YOLOv11 and EasyOCR as described in claim 1, characterized in that: The two-stage annotation strategy of the YOLOv11 object detection unit includes: the first stage uses the LabelImg tool to annotate three types of targets in handwritten text: title area, character box, and title box boundary, generating YOLO format annotation files, accumulating 32,000 target boxes, covering 12,000 handwritten text images; the second stage uses the K-means clustering algorithm to generate custom anchor boxes with a height-to-width ratio of 1:1.2 for Chinese handwritten characters; the inference process includes: resizing the preprocessed image to 640×640, using the feature pyramid network to detect small characters of 16×16 pixels, medium characters of 32×32 pixels, and large characters of 64×64 pixels on feature maps of three different scales, respectively, filtering overlapping boxes through non-maximum suppression with a threshold of 0.5, and retaining detection results with a confidence of ≥0.

8.

4. The student handwriting recognition system based on the fusion of YOLOv11 and EasyOCR as described in claim 1, characterized in that: The domain-adaptive training strategy of the EasyOCR recognition unit includes: adding ±15° rotation and 0.8-1.2x scaling to the original character image. Gaussian blur data enhancement perturbation of 1.5; freezing the first 10 layers of the CNN backbone network, and only fine-tuning the BiLSTM and CTC layers; training using a dedicated dataset for educational scenarios containing 100,000+ characters; the time-dependent weight formula in the above formula... =256, =40; in the rule base correction confidence formula Set it to 0.

3. Calculated using edit distance, ranging from [0,1].

5. The student handwriting recognition system based on the fusion of YOLOv11 and EasyOCR as described in claim 1, characterized in that: The backend processing module employs an asynchronous loading mechanism to optimize model and data processing workflows, avoids redundant calculations through a state caching mechanism, and supports batch processing of multiple graphs. The frontend interaction module utilizes `st.session_state` for persistent state management and uses the `@st.cache_data` decorator to optimize preprocessing and model inference. The cross-module attention alignment formula... , All dimensions are 128.

6. The student handwriting recognition system based on the fusion of YOLOv11 and EasyOCR as described in claim 1, characterized in that: The fields in the answer_results table include id, image_id, question_number, recognized_text, confidence, bounding box, timestamp, and is_corrected.

7. A student handwriting structure recognition method based on the fusion of YOLOv11 and EasyOCR, wherein the method is implemented using the student handwriting recognition system based on the fusion of YOLOv11 and EasyOCR as described in any one of claims 1-6, characterized in that: S1. Data preparation: Construct a dataset containing 12,000 character regions, covering various question types, font styles and interference factors. Use a two-stage annotation mechanism to generate YOLO format detection annotation files and OCR recognition annotation files, and divide the training set and validation set in an 8:2 ratio. S2. Model Training: Train the YOLOv11 object detection model, implemented in PyTorch. Initialize the backbone network with pre-trained weights and use cross-stage feature recombination formulas. Optimize feature extraction and combine it with the K-means clustering loss formula. Generate custom anchor boxes, setting the input size to 640×640, batch size to 16, initial learning rate to 0.001, optimizer to SGD, number of epochs to 100, and loss function to CIoULoss+BCELoss, combined with a warm-up mechanism and Mosaic data augmentation; fine-tune the EasyOCR recognition model, freeze the CNN feature extraction layer, and apply a time-dependent weight formula. , To enhance the recognition of cursive characters, the batch size was set to 32, the learning rate to 0.0005, the optimizer to Adam, the number of epochs to 80, the loss function to CTCLoss, and the EarlyStopping strategy was adopted. S3. Image Upload and Preprocessing: Users upload handwritten test paper images through the front-end interaction module, and the back-end processing module performs format normalization, noise suppression, adaptive binarization and perspective correction on the images. S4. Character Region Detection: The YOLOv11 object detection unit performs multi-scale inference on the preprocessed image and uses a confidence-weighted selection formula. Retain high-confidence results and output bounding boxes containing question numbers and character region coordinates; S5. Handwritten Character Recognition: The EasyOCR recognition unit extracts and preprocesses character regions, and corrects the confidence formula using a rule base. Correct common errors and perform character recognition; S6. Result Optimization and Storage: Through the cross-module attention alignment loss formula To optimize detection and recognition errors, the "question number-answer" key-value pairs are stored in a structured SQLite database, and the recognition results are displayed on the front end. Exporting CSV files is also supported.

8. The student handwriting structure recognition method based on the fusion of YOLOv11 and EasyOCR as described in claim 7, characterized in that: In step S1, the YOLO format annotation file is generated using the LabelImg tool, with each line containing...<class_id><x_center><y_center> <width> <height> The OCR recognition annotation file is in .csv or .json format, containing image path, recognized text, character length information, and the recognized text is processed according to UTF-8 encoding standard, simplified and traditional Chinese characters, and full-width and half-width numbers.< / height> < / width> 9. The student handwriting structure recognition method based on the fusion of YOLOv11 and EasyOCR as described in claim 7, characterized in that: In step S2, the warm-up period for YOLOv11 training is the first 5 epochs, with the learning rate gradually increasing from 0.0001 to 0.001; the EarlyStopping strategy for EasyOCR training is to automatically stop training if the accuracy on the validation set does not improve after 10 consecutive epochs. The training data comes from cropped images of YOLOv11 detection boxes, combined with manual annotation and enhanced by rotation, noise addition, etc.

10. The student handwriting structure recognition method based on the fusion of YOLOv11 and EasyOCR according to claim 7, characterized in that: In step S4, the three scale feature maps of the YOLOv11 feature pyramid network correspond to downsampling rates of 1 / 8, 1 / 16, and 1 / 32 of the input image, respectively, with a non-maximum suppression threshold of 0.

5. In step S5, EasyOCR's character region preprocessing includes resizing to 32×128 pixels, grayscale stretching, and threshold segmentation. In step S6, the cross-module attention alignment loss formula... This represents the total number of questions identified in a single scan. Extracted from YOLOv11 detection bounding boxes. Extracted from EasyOCR recognition results.

Images