AI classroom teaching quality evaluation method and system based on image recognition

Abstract

The invention relates to the technical field, in particular to an AI classroom teaching quality evaluation method and system based on image recognition, and the method comprises the steps: obtaining a test paper RGB image and structured metadata, and generating multi-modal input data through graying, denoising and affine transformation; dynamically adjusting the weight through a double-branch feature extraction network (CNN + ViT), and extracting the global structure of the printed form and the local detail features of the handwritten form; generating a mixed font segmentation mask by using space attention, channel attention and dynamic routing regulation; optimizing handwritten answer recognition in combination with an educational knowledge graph; and fusing the answer expression and the attention data to generate a teaching quality evaluation report. The system comprises a data acquisition unit, a preprocessing unit, a feature extraction unit, a segmentation unit, an identification verification unit and an evaluation unit. According to the scheme, the accuracy of character segmentation and recognition in a mixed font scene is improved, and data support is provided for precise teaching in an education scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image recognition mode classification and the field of education scene data processing in the field of digital data processing technology, and in particular to an AI classroom teaching quality evaluation method and system based on image recognition. BACKGROUND

[0002] In the education scene, the core technology path of the AI classroom based on image recognition involves multi-stage data processing procedures such as visual information acquisition, feature extraction, pattern classification, and education scene adaptation. Specifically, for example, a mapping relationship between student attention fluctuation curve and knowledge point explanation rhythm is established to provide quantitative basis for teachers to dynamically adjust teaching progress and optimize interaction strategies. Through the synergistic optimization of multi-modal data fusion and deep learning models, the intelligent transformation of classroom visual information from perception to cognition is realized, and the evolution of the education scene to a data-driven precise teaching mode is promoted.

[0003] In the student test paper grading scene, the core technical pain points of the AI classroom based on image recognition are the insufficient accuracy of text segmentation and recognition in the mixed scene of handwritten characters and printed characters. Specifically, the test paper image usually includes printed text (such as standard fonts such as Songti and Kaishu) and student handwritten answers (such as continuous writing, tilting, and uneven thickness of personalized writing). The existing image recognition models based on convolutional neural network (CNN) or visual Transformer (ViT) have difficulty in accurately distinguishing the boundary features of the two fonts in the feature extraction stage due to the limitations of local receptive fields or information redundancy of global attention mechanisms. For example, when a student writes the answer "x = 7" next to the printed question stem "Solve the equation: x + 5 = 12", if the handwritten "x" and the printed "x" in the question stem differ in stroke curvature and ink depth, the model may mistakenly include the printed "x" in the question stem into the answer recognition area, resulting in the incorrect segmentation of "x = 7" as "x + 5 = 12x = 7", and further causing the grading system to misjudge the completeness of the answer. In addition, if the student modifies the answer with overlapping strokes (such as changing "3" to "8" resulting in overlapping strokes), the model may not accurately separate the handwritten features in the overlapping area, and may recognize "8" as "3" or "∞", causing the grading result to deviate. Such problems are essentially the lack of differentiated representation and effective segmentation technology of multi-modal text features in the education scene adaptation, which affects the practicality of the AI classroom in the test paper grading link. SUMMARY

[0004] To address the deficiencies in the prior art, the present application provides an AI classroom teaching quality evaluation method and system based on image recognition, which solves the problem of insufficient accuracy of text segmentation and recognition in image recognition.

[0005] To solve the above technical problems, the specific technical solutions of the present application are as follows: In a first aspect, the present application provides an AI classroom teaching quality evaluation method based on image recognition, comprising: Step 1: Obtain the RGB image data of the test paper and the structured metadata including the question area coordinates, question type label, font type, and standard answer template; perform grayscale, median filter denoising on the image data, map the standard area coordinates of the metadata to the actual image pixel coordinates through affine transformation, and generate multi-modal input data labeled with the question stem area, answer area, and modification area; Step 2: input the multi-modal input data generated in step 1 into a double-branch feature extraction network, wherein the convolutional neural network branch extracts the global structural features of printed characters, and the visual Transformer branch extracts the local detail features of handwritten characters; dynamically adjust the feature output weight of the double-branch according to the pixel ratio of printed and handwritten characters, and generate a differentiated feature map that fuses the global structure of printed characters and the local details of handwritten characters; Step 3: input the differentiated feature map generated in step 2 into a segmentation network, suppress the printed pixel response in the question stem area and enhance the boundary pixel activation in the answer area through spatial attention mechanism, adjust the feature channel weight according to the font type through channel attention mechanism, and dynamically fuse spatial and channel attention according to the font pixel ratio to output a segmentation mask for the labeled printed question stem, handwritten answer, and modification overlay area; Step 4: extract the answer area and modification area sub-image based on the segmentation mask generated in step 3, input it into a pre-trained handwritten-printed mixed OCR model to generate a character sequence; call the pre-built education knowledge graph to obtain the standard answer template and common modification mode corresponding to the real-time question, and if the character sequence does not match the standard answer template or the modification area feature is detected, adjust the segmentation mask and re-extract the local feature to optimize the recognition result; Step 5: compare the optimized recognition result in step 4 with the pre-stored standard answer library, and calculate evaluation indicators such as answer accuracy and modification misrecognition rate; fuse classroom attention analysis data, and mine the association rules between answer performance and attention level through a random forest model; adjust the index weight according to the evaluation target, and generate a teaching quality evaluation report including student knowledge weaknesses, overall class level, and teacher teaching suggestions.

[0006] Further, in the AI classroom teaching quality evaluation method based on image recognition, the image data in step 1 is obtained by an industrial-grade image acquisition device deployed in the classroom or the test paper area, which supports automatic focusing and illumination compensation; the structured metadata is obtained synchronously through the API interface of the educational administration system, including the standard area coordinates of the question in the test paper template, the question type label of the multiple-choice question or the answer question, the printed font type of Songti or Kaishu, and the standard answer template in the preset format, which is used for coordinate mapping in subsequent affine transformation.

[0007] Furthermore, in the AI ​​classroom teaching quality assessment method based on image recognition described in this invention, in step 2, the CNN branch of the dual-branch feature extraction network takes the multimodal input data generated in step 1 as input and uses a 3×3 convolutional kernel to extract global structural features such as stroke regularity and row-column alignment of printed characters; the ViT branch divides the image into 16×16 pixel blocks through block embedding and uses a self-attention layer to extract local detail features such as curvature of the stroke trajectory and ink gradient of handwritten characters; the dynamic routing mechanism calculates the pixel ratio of printed and handwritten characters based on the area ratio of the question stem area and the answer area marked in step 1. If the proportion of printed characters is high, the feature output weight of the CNN branch is enhanced; if the proportion of handwritten characters is high, the feature fusion ratio of the ViT branch is increased, and finally, a differentiated feature map is generated.

[0008] Furthermore, in the AI ​​classroom teaching quality evaluation method based on image recognition described in this invention, step 3 of the spatial attention mechanism takes the differential feature map generated in step 2 as input, calculates the gray-level difference and positional distance between adjacent pixels, and generates contextual association weights to suppress the response of printed pixels in the question stem area and enhance the activation of handwritten boundary pixels in the answer area; the channel attention mechanism assigns higher weights to the handwritten feature channels based on the font type labels in the metadata of step 1; the dynamic routing control mechanism increases the spatial attention weight when the handwritten proportion is high and the channel attention weight when the printed proportion is high based on the font pixel ratio statistically obtained in step 2, and finally outputs a pixel-level labeled segmentation mask including the printed question stem, the handwritten answer, and the modified coverage area.

[0009] Furthermore, in the AI-based classroom teaching quality assessment method based on image recognition described in this invention, the pre-trained handwritten and printed hybrid OCR model in step 4 takes the answer area and modification area sub-images generated in step 3 as input to generate a character sequence; the educational knowledge graph stores the standard answer template and the modification pattern of stroke coverage; if the character sequence does not match the standard answer template or the modification area feature is detected, the segmentation mask generated in step 3 is adjusted and local features are re-extracted, and the recognition result is optimized by combining the rule that the gray value of the intersection pixel in the modification pattern is higher than that of the surrounding pixels.

[0010] Furthermore, in the AI-based classroom teaching quality assessment method based on image recognition described in this invention, step 5 involves storing standard answers for each question type in a standard answer database; the random forest model uses the optimized recognition results from step 4 and classroom attention analysis data as input to analyze the accuracy improvement patterns of students with a focus duration of ≥10 minutes; the dynamic routing mechanism adds weights to accuracy and error rate when assessing students' knowledge mastery, and adds weights to the accuracy distribution of questions of different difficulty when assessing the rationality of question design; the assessment report is displayed in the form of tables and charts via a web interface.

[0011] Furthermore, in the AI ​​classroom teaching quality assessment method based on image recognition described in this invention, the educational knowledge graph in step 4 stores the association between multi-disciplinary knowledge nodes and questions. When the modification area feature is detected, a high-frequency incorrect answer database of the same question type is extracted based on the educational knowledge graph, and the similarity between the character sequence and the high-frequency incorrect answer is calculated using the edit distance algorithm. If the similarity exceeds the preset threshold, skip step 4 of adjusting the segmentation mask and re-extracting local features, directly label the recognition result as the corresponding error type, and associate it with the knowledge weakness marker in the educational knowledge graph.

[0012] Furthermore, the AI-based classroom teaching quality assessment method based on image recognition described in this invention, in addition to calculating the spatial distribution entropy of handwritten pixels based on the statistical ratio of printed and handwritten pixels, incorporates a dynamic routing mechanism: Information entropy is calculated based on the pixel coordinates of the handwritten characters. When the entropy value is higher than a set threshold, the handwriting is judged to be illegible. In the case of illegible handwriting, the local feature weights of the visual Transformer branch are increased to more than 70% of the total output of the two branches, while the weight ratio of the convolutional neural network branch is reduced simultaneously.

[0013] Furthermore, in the AI-based classroom teaching quality assessment method based on image recognition described in this invention, the modified area feature optimization recognition includes: Based on the stroke trajectory data of the standard answer in the educational knowledge graph, the gray-level gradient direction of the intersection point of the modified area is calculated. If the offset angle from the standard trajectory is greater than 15°, it is determined to be a valid modification. Based on the modified pattern library type called in step 4: For stroke-covering modifications, a contour-filling algorithm based on connected component analysis is used to restore the covered original answer strokes; For erased modifications, the boundary contrast is enhanced by using the Laplacian operator, and then the handwritten and printed text is re-entered for recognition by the hybrid OCR model.

[0014] Secondly, the image recognition-based AI classroom teaching quality assessment system provided by the present invention, applied to the image recognition-based AI classroom teaching quality assessment method, includes: The data acquisition unit is configured to acquire RGB image data of the test paper and structured metadata including question area coordinates, question type labels, font type, and standard answer template; The data preprocessing unit is configured to perform grayscale conversion and median filtering on RGB image data, map the standard region coordinates of structured metadata to the actual image pixel coordinates through affine transformation, generate multimodal input data with annotations for the question stem area, answer area, and modification area, and transmit it to the feature extraction unit. The feature extraction unit, including a convolutional neural network branch and a visual Transformer branch, is configured to extract the global structural features of printed characters and the local detail features of handwritten characters in multimodal input data. The feature output weights of the two branches are dynamically adjusted according to the pixel ratio of printed and handwritten characters to generate differential feature maps and transmit them to the multi-attention segmentation unit. The multi-attention segmentation unit is configured to suppress the response of printed pixels in the question stem area and enhance the activation of handwritten boundary pixels in the answer area through a spatial attention mechanism. It adjusts the feature channel weights according to the font type through a channel attention mechanism and dynamically fuses spatial and channel attention according to the font pixel ratio to generate a segmentation mask that annotates the printed text of the question stem, the handwritten text of the answer, and modifies the coverage area, and transmits it to the context verification and recognition unit. The context verification recognition unit is configured to extract answer area and modification area sub-images based on segmentation mask, input a pre-trained handwritten and printed hybrid OCR model to generate character sequences, call a pre-built educational knowledge graph to obtain the standard answer template corresponding to the real-time question and the modification pattern of "3→8" stroke coverage, if the character sequence does not match the standard answer template or the modification area features are detected, adjust the segmentation mask and re-extract local features to optimize the recognition results, and transmit them to the quality assessment generation unit; The quality assessment generation unit is configured to compare the optimized recognition results with the standard answer database, statistically analyze evaluation indicators such as answer accuracy and correction error rate, integrate classroom attention analysis data, mine the correlation rules between answer performance and attention level through a random forest model, adjust the indicator weights according to the evaluation objectives, and generate a teaching quality assessment report that includes students' knowledge weaknesses, the overall level of the class, and teachers' teaching suggestions.

[0015] Beneficial effects of this invention; This invention achieves precise mapping of standard region coordinates through synchronous acquisition and alignment preprocessing of multimodal data, combined with affine transformation, providing prior localization basis for text segmentation and effectively solving the coordinate offset problem caused by the tilt of the test paper. The dynamic routing mechanism dynamically adjusts the weights of the dual-branch feature extraction network based on the pixel ratio of printed and handwritten text, integrating the ability of convolutional neural networks to capture global structural features of printed text with the advantages of visual Transformers in extracting local detail features of handwritten text, avoiding feature confusion caused by the limitations of local receptive fields or global attention redundancy in single models. The multi-dimensional attention-enhanced segmentation network suppresses the printed text stem through spatial attention. Disturbance and channel attention focus on handwritten features, combined with dynamic routing control to optimize the attention fusion ratio, significantly improving the segmentation accuracy of mixed font boundaries; the context constraint recognition and verification process combines standard answer templates and modification patterns from educational knowledge graphs, and optimizes the feature extraction of modification areas through secondary recognition, reducing the false recognition rate of handwritten modification traces; the multi-source data fusion teaching quality assessment method integrates answer performance and attention analysis data, mines association rules through a random forest model and dynamically adjusts indicator weights, providing teachers with multi-dimensional assessment results such as students' knowledge weaknesses, overall class level, and question design suggestions, promoting the evolution of educational scenarios towards a data-driven precision teaching model. Attached Figure Description

[0016] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on the drawings without creative effort.

[0017] Figure 1 A flowchart of an AI-based classroom teaching quality assessment method and system based on image recognition provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention. The technical solutions provided by various embodiments of this invention will be described in detail below with reference to the accompanying drawings. To better understand the objectives of this invention, it will be described in further detail below.

[0019] In a first aspect, the AI-based classroom teaching quality assessment method based on image recognition provided by this invention includes: Step 1: Obtain the RGB image data of the test paper and the structured metadata including the coordinates of the question area, question type label, font type, and standard answer template; perform grayscale conversion and median filtering to remove noise from the image data, and map the standard area coordinates of the metadata to the actual image pixel coordinates through affine transformation to generate multimodal input data with annotations for the question stem area, answer area, and modification area; Step 2: Input the multimodal input data generated in Step 1 into the dual-branch feature extraction network, where the convolutional neural network branch extracts the global structural features of printed characters and the visual Transformer branch extracts the local detail features of handwritten characters; dynamically adjust the feature output weights of the dual branches according to the pixel ratio of printed and handwritten characters to generate a differential feature map that integrates the global structure of printed characters and the local details of handwritten characters. Step 3: Input the differential feature map generated in Step 2 into the segmentation network. Use the spatial attention mechanism to suppress the response of printed pixels in the question stem area and enhance the activation of pixels at the boundary of handwritten characters in the answer area. Use the channel attention mechanism to adjust the feature channel weights according to the font type and dynamically fuse spatial and channel attention according to the font pixel ratio. Output the segmentation mask that annotates the printed text of the question stem, the handwritten text of the answer, and the modified coverage area. Step 4: Extract the answer area and modification area sub-images based on the segmentation mask generated in Step 3, input them into the pre-trained handwritten and printed hybrid OCR model to generate character sequences; call the pre-built educational knowledge graph to obtain the standard answer template and common modification patterns corresponding to the real-time questions; if the character sequence does not match the standard answer template or modification area features are detected, adjust the segmentation mask and re-extract local features to optimize the recognition results. Step 5: Compare the optimized recognition results from Step 4 with the pre-stored standard answer database, and statistically analyze evaluation indicators such as answer accuracy and correction error rate; integrate classroom attention analysis data, and use a random forest model to mine the correlation rules between answer performance and attention level; adjust the indicator weights according to the evaluation objectives, and generate a teaching quality evaluation report that includes students' knowledge weaknesses, the overall level of the class, and teachers' teaching suggestions.

[0020] In Step 1, the data acquisition and preprocessing stage, RGB three-channel image data of the exam paper is acquired using industrial-grade image acquisition equipment. This equipment supports autofocus and illumination compensation to adapt to changes in the classroom environment. Structured metadata is synchronously acquired through the academic affairs system API interface, including the coordinate positioning data of the questions in the standard exam paper template (such as the boundary coordinates of the question stem area), question type classification labels (such as multiple choice / shortcut questions), font type identifiers (such as SimSun or KaiTi), and predefined format standard answer templates. The preprocessing process first performs grayscale conversion on the RGB images to generate single-channel grayscale images to reduce computational complexity, and then uses a median filter to eliminate salt-and-pepper noise interference. Based on the affine transformation algorithm, the standard area coordinates in the metadata are mapped to the pixel coordinate system of the actual acquired image, solving the problem of physical placement offset of the exam paper. Finally, multimodal input data with annotations for the question stem area, answer area, and modification area are output, providing spatial positioning basis for subsequent processing.

[0021] Step 2, the feature extraction stage, employs a parallel dual-branch architecture. The convolutional neural network branch scans the multimodal input data with 3×3 convolutional kernels to extract global structural features of printed characters, including spatial distribution characteristics such as stroke direction consistency, character spacing uniformity, and row / column alignment. The visual Transformer branch divides the image into a 16×16 pixel block sequence and captures local detail features of handwritten characters through a multi-head self-attention mechanism, including changes in curvature of the stroke trajectory, ink density gradient distribution, and stroke width fluctuations. A dynamic routing mechanism calculates the area ratio of the question stem area to the answer area in real time, thereby statistically determining the pixel ratio of printed and handwritten characters. When the pixel ratio of printed characters exceeds a preset threshold, the weight coefficient of the convolutional neural network branch in feature fusion is increased; when the pixel ratio of handwritten characters is dominant, the feature contribution of the visual Transformer branch is enhanced. Weighted fusion generates a differentiated feature map containing both the global structure of printed characters and the local details of handwritten characters.

[0022] In step 3, during the segmentation network processing stage, the spatial attention module calculates the gray-level difference and Euclidean distance between adjacent pixels in the differential feature map to generate a spatial weight matrix. This matrix suppresses the activation response of printed pixels in the question stem area while enhancing the feature expression intensity of handwritten boundary pixels in the answer area. The channel attention module assigns higher weight coefficients to handwritten-related feature channels based on the font type labels in the metadata. The dynamic routing control module, based on the font pixel ratio data collected in step 2, increases the spatial attention weight when the handwritten pixel ratio is high and increases the channel attention weight when the printed pixel ratio is high. A gating mechanism is used to fuse the two attention outputs to generate a pixel-level labeled segmentation mask, clearly distinguishing the printed question stem area, the handwritten answer area, and the modified coverage area.

[0023] Step 4, the recognition and verification stage, extracts sub-images of the answer area and the modification area based on the segmentation mask. A pre-trained handwritten and printed hybrid OCR model performs optical character recognition on the sub-images to generate an initial character sequence. Simultaneously, it calls the educational knowledge graph to obtain the standard answer template and modification pattern library for the current question (e.g., "3→8" type stroke coverage features). When the character sequence logically conflicts with the standard answer template (e.g., numerical range discrepancies) or when intersecting stroke features are detected in the modification area, feedback optimization is performed: the boundary range of the segmentation mask is adjusted to re-extract the local region, and the recognition result is optimized by combining the physical characteristic that the gray value of the intersection point of the modification area is higher than that of the surrounding pixels. For stroke coverage type modifications, a connected component analysis algorithm is used to reconstruct the original stroke trajectory that was covered.

[0024] Step 5, the quality assessment stage, compares the optimized recognition results with the standard answer database, calculating quantitative indicators such as answer accuracy and correction error rate. Classroom attention analysis data (such as focus duration and eye-tracking data) is correlated with answering behavior through time alignment. A random forest model analyzes the association rules between answering performance and attention characteristics, such as the statistical significance of improved accuracy in samples with focus duration exceeding 10 minutes. A dynamic routing mechanism allocates indicator weights according to the assessment objectives: student knowledge mastery assessment focuses on accuracy and correction error rate, while question design assessment adds weights to the accuracy distribution across different difficulty levels. Finally, a structured assessment report is generated, including an analysis of knowledge weaknesses based on educational knowledge graph associations, a heatmap of class ability level distribution, and teaching strategy suggestions.

[0025] Each step is tightly integrated through data flow: preprocessed multimodal data drives the feature extraction network; bi-branch feature maps provide differentiated inputs for the segmentation network; segmentation masks guide sub-image extraction; recognition results are verified by a knowledge graph and then fed back to optimize segmentation accuracy; finally, multi-source data are fused to generate an evaluation report.

[0026] Specifically, in the AI-based classroom teaching quality assessment method based on image recognition described in this invention, the image data in step 1 is acquired by an industrial-grade image acquisition device deployed in the classroom or marking area. The industrial-grade image acquisition device supports autofocus and illumination compensation. The structured metadata is synchronously acquired through the teaching affairs system API interface, including the standard area coordinates of the questions in the test paper template, the question type labels of multiple-choice or problem-solving questions, the printed font type of Song or Kai, and the standard answer template in a preset format, which are used for coordinate mapping of subsequent affine transformations.

[0027] Industrial-grade image acquisition equipment is deployed in the classroom ceiling or physical space of the marking area. This equipment integrates an autofocus module and a wide dynamic range sensor, adjusting the lens focal length and sensor exposure parameters in real time to adapt to fluctuations in ambient light. The acquired RGB three-channel image data contains complete visible light spectrum information of the exam paper, providing raw visual input for subsequent processing. The academic affairs system API interface synchronously transmits structured metadata via HTTP protocol. Metadata fields include normalized coordinate data of the questions in the standard exam paper template (X / Y axis coordinates based on the top-left origin), question type classification codes (e.g., binary identifiers distinguishing between multiple-choice and problem-solving questions), enumerated values ​​of printed font types (e.g., Song typeface corresponding to type identifier 1, Kai typeface corresponding to identifier 2), and a standard answer template stored in JSON format (containing expected answer structure and regular expression constraints). This metadata set provides prior semantic information for image processing.

[0028] The preprocessing stage first performs color space conversion, transforming the RGB three-channel image into a single-channel grayscale image to reduce data dimensionality. The conversion process uses a weighted average method to calculate pixel brightness values. Median filtering scans the grayscale image using a 5×5 pixel window, and a sorting filtering algorithm eliminates impulse noise and local outliers. An affine transformation algorithm establishes a projection matrix based on the standard region coordinates in the metadata, fits the coordinate offset of the corresponding region in the actual image using the least squares method, and performs rotation and scaling transformations to align the actual image coordinate system with the standard template. This process generates multimodal input data containing annotations for the question stem, answer area, and modification area. The annotation information is superimposed on the preprocessed image data in the form of a mask matrix.

[0029] The necessity of coordinate mapping stems from the pose deviations of exam papers in physical scenarios. Affine transformation, through a linear transformation model (including translation, rotation, and scaling parameters), maps the template coordinates provided by the academic affairs system to the actual pixel coordinate system of the acquired image, resolving regional positioning errors caused by paper tilt or perspective. The region annotation information in the multimodal input data serves as guiding signals for subsequent steps, enabling the feature extraction network to selectively process text features in different functional areas; for example, focusing on extracting printed features in the question stem area while strengthening handwritten character analysis in the answer area. This data structuring processing based on prior knowledge lays the spatial foundation for feature decoupling in mixed font scenarios.

[0030] Specifically, in the AI ​​classroom teaching quality assessment method based on image recognition described in this invention, in step 2, the CNN branch of the dual-branch feature extraction network takes the multimodal input data generated in step 1 as input and uses a 3×3 convolutional kernel to extract global structural features such as stroke regularity and row-column alignment of printed characters; the ViT branch divides the image into 16×16 pixel blocks through block embedding and uses a self-attention layer to extract local detail features such as curvature of the stroke trajectory and ink gradient of handwritten characters; the dynamic routing mechanism calculates the pixel ratio of printed and handwritten characters based on the area ratio of the question stem area and the answer area marked in step 1. If the proportion of printed characters is high, the feature output weight of the CNN branch is enhanced; if the proportion of handwritten characters is high, the feature fusion ratio of the ViT branch is increased, and finally, a differentiated feature map is generated.

[0031] The convolutional neural network branch processes multimodal input data, constructing a hierarchical structure including convolutional layers, activation function layers, and pooling layers. This branch uses 3×3 convolutional kernels for a sliding window scanning operation, learning the consistency features of stroke direction in printed characters through kernel weights. During the progression through successive convolutional layers, the feature map spatial size decreases progressively while the number of channels increases, thereby capturing global structural features such as character row and column alignment and paragraph layout in higher-level feature maps. Finally, a dense feature vector representing the overall spatial distribution pattern of the printed text is output through global spatial pooling.

[0032] The visual Transformer branch employs a block embedding process, dividing the input image into a sequence of non-overlapping blocks of 16×16 pixels. Each block is transformed into a feature vector through linear projection, and learnable positional encoding information is superimposed to preserve spatial relationships. A self-attention mechanism calculates the association weight matrix between the feature vectors of all blocks, focusing on local detail features within the handwritten text region: including curvature variations at stroke inflection points, gradient differences in ink density at the start and end points of strokes, and the ink overlay effect at stroke intersections. A multi-layered Transformer encoder stack enables multi-scale abstract representation of local detail features.

[0033] The dynamic routing mechanism performs quantitative analysis of font pixel proportions. Based on the polygon coordinate data of the question stem and answer areas marked in step 1, this mechanism calculates the actual pixel area of ​​each region in the image. Combining preset font type density parameters (printed characters have a higher pixel count per unit area than handwritten characters), it derives the pixel proportion weights of printed and handwritten characters in the entire image. The routing control module generates branch weight coefficients based on the proportion calculation results: when the printed pixel proportion exceeds the 65% threshold, the weight of the convolutional neural network branch in feature fusion is increased to above 0.7; when the handwritten pixel proportion exceeds 60%, the fusion weight of the visual Transformer branch is correspondingly enhanced. The feature fusion layer performs a linear weighted operation based on the weight coefficients on the dual-branch outputs, generating a differentiated feature map that fuses global structural features and local detail features.

[0034] The technical solution addresses the interference problem of mixed font features through dual-path parallel processing: the translation invariance of convolutional neural network branches adapts to the regular spatial distribution of printed characters, while the self-attention mechanism of the visual Transformer adapts to the irregular local shapes of handwritten characters. A dynamic routing mechanism dynamically optimizes the feature extraction strategy based on the font distribution in the actual scene, avoiding feature confusion caused by a single model in mixed regions. The output differential feature map contains complementary information on spatial structure and local details, providing a foundation for subsequent segmentation networks to distinguish different font types.

[0035] Specifically, in the AI ​​classroom teaching quality evaluation method based on image recognition described in this invention, step 3 involves a spatial attention mechanism that takes the differential feature map generated in step 2 as input, calculates the grayscale difference and positional distance between adjacent pixels, and generates contextual association weights to suppress the response of printed pixels in the question stem area and enhance the activation of handwritten boundary pixels in the answer area. The channel attention mechanism assigns higher weights to the handwritten feature channels based on the font type labels in the metadata from step 1. The dynamic routing control mechanism increases the spatial attention weight when the handwritten proportion is high and the channel attention weight when the printed proportion is high, based on the font pixel ratio statistically analyzed in step 2. The final output includes a pixel-level labeled segmentation mask for the printed question stem, the handwritten answer, and the modified coverage area.

[0036] The spatial attention mechanism processes the differential feature map output from step 2, constructing a pixel-level association matrix by calculating the gray-level difference vector and Euclidean distance parameter between adjacent pixels in the feature map. This mechanism generates a context-related weight map: negative weight coefficients are applied to the printed pixels in the question stem area to suppress their activation response, while positive weight coefficients are assigned to the handwritten boundary pixels in the answer area to enhance the feature expression intensity. The weight map and the original feature map are subjected to a Hadamard product operation, which strengthens the detailed features of the handwritten boundary area and weakens the interference areas of the printed text.

[0037] The channel attention mechanism performs channel selection based on the font type labels in the metadata from step 1. This mechanism encodes the font type labels as one-hot vectors and maps them through a fully connected layer to generate channel weight coefficient vectors. Handwritten feature channels are assigned higher weight coefficients than printed channels (e.g., a 30% increase in weight for handwritten channels), guiding the network to focus on handwritten-related features in the mixed feature channels. The weighted feature channels are then reorganized through 1×1 convolutional layers to enhance the representational ability of handwritten features.

[0038] The dynamic routing adjustment mechanism selects attention strategies based on the font pixel ratio data obtained in step 2. This mechanism includes a routing decision module: when the handwritten pixel ratio exceeds a preset threshold, the fusion weight of the spatial attention module in the final output is increased; when the printed pixel ratio is dominant, the contribution ratio of the channel attention module is increased. The attention fusion layer dynamically weights and integrates the two attention outputs through a gating mechanism to form an optimized segmentation feature map. This feature map is input into the U-Net segmentation network, and after upsampling and skip connection operations, it outputs a pixel-level labeled segmentation mask, clearly distinguishing the printed region of the question stem, the handwritten region of the answer, and the modified coverage area.

[0039] The technical solution addresses the challenge of segmenting mixed regions through multi-level attention collaboration: spatial attention precisely locates the handwritten character boundaries, channel attention enhances the discriminative power of handwritten character features, and dynamic routing optimizes the attention combination strategy based on scene characteristics. These three mechanisms form a hierarchical optimization system of "spatial localization, feature selection, and strategy adaptation," significantly improving the segmentation accuracy of mixed font boundaries. The output segmentation mask provides accurate region segmentation data for subsequent recognition and verification.

[0040] Specifically, in the AI ​​classroom teaching quality assessment method based on image recognition described in this invention, the pre-trained handwritten and printed hybrid OCR model in step 4 takes the answer area and modification area sub-images generated in step 3 as input to generate a character sequence; the educational knowledge graph stores the standard answer template and the modification pattern of stroke coverage; if the character sequence does not match the standard answer template or the modification area features are detected, the segmentation mask generated in step 3 is adjusted and local features are re-extracted, and the recognition result is optimized by combining the rule that the gray value of the intersection pixel in the modification pattern is higher than that of the surrounding pixels.

[0041] The pre-trained handwritten and printed hybrid OCR model receives the answer region and modification region sub-images located by the segmentation mask generated in step 3 as input. This model employs a convolutional recurrent neural network architecture, extracting local morphological features of characters through convolutional layers, modeling the contextual dependencies of character sequences through bidirectional long short-term memory network layers, and finally outputting the character sequence recognition result by connecting to a temporal classification decoder. The educational knowledge graph stores structured standard answer template data, including the character composition rules of the expected answers to questions (e.g., math answers must conform to numerical formats), semantic constraints (e.g., chemical equations must satisfy atomic conservation), and also stores the physical characteristics of common modification patterns (e.g., the grayscale distribution pattern of pixels at intersections in a "3→8" type modification).

[0042] When the OCR output character sequence logically conflicts with the standard answer template in the educational knowledge graph (e.g., numerical values ​​exceeding a reasonable range or character combinations violating semantic rules) or when the segmentation mask detects intersecting stroke features in the modified region, a feedback optimization mechanism is triggered. This mechanism first adjusts the boundary range of the segmentation mask generated in step 3: performing morphological dilation on the modified region to expand the region of interest, and then re-extracting the optimized local sub-image. For the features of the modified region, analysis is performed based on the physical laws of the modification patterns stored in the knowledge graph: the grayscale value of the intersection pixel is higher than that of the surrounding non-intersecting regions, forming a local extremum. This characteristic is used to separate the covered strokes using a region growing algorithm. The optimized sub-image is then input into the OCR model for recognition.

[0043] For stroke-covering modifications, a connected component analysis algorithm is used to locate the closed contours within the modified area. Based on the stroke trajectory data in the standard answer template, the original stroke direction of the covered area is fitted using Bézier curves to reconstruct the topological structure of the covered character. For erasure modifications, the edge contrast of the remaining handwriting is enhanced using the Laplacian operator, and the enhanced image is then re-identified. The optimized recognition results are then subjected to secondary matching and verification with the educational knowledge graph.

[0044] The technical solution improves recognition accuracy by combining semantic constraints and physical laws from a knowledge graph: standard answer templates provide character-level verification rules, and the modified pattern library describes the differences in the physical characteristics of handwriting. A feedback mechanism dynamically optimizes segmentation accuracy and recognition strategies, effectively solving the problem of misrecognition caused by handwritten modifications. The optimized character sequence serves as a reliable data source for the evaluation stage.

[0045] Specifically, in the AI-based classroom teaching quality assessment method based on image recognition described in this invention, step 5 stores standard answers for each question type in the standard answer database; the random forest model takes the optimized recognition results from step 4 and classroom attention analysis data as input to analyze the accuracy improvement pattern of students with a focus duration of ≥10 minutes; the dynamic routing mechanism adds weights to accuracy and error rate when assessing students' knowledge mastery, and adds weights to the accuracy distribution of questions of different difficulty when assessing the rationality of question design; the assessment report is displayed in the form of tables and charts on the web.

[0046] The standard answer database employs a tree-structured storage system, establishing index relationships based on subject knowledge points, question type (multiple choice / shortcut questions), and difficulty level. Each question is associated with predefined standard answer data and its allowed equivalent expressions. The random forest model receives input data in two dimensions: quantitative indicators generated by comparing the optimized character sequence recognition results from step 4 with the standard answer database (including single-question correctness indicators and statistics on the number of incorrect corrections); and attention data collected through a classroom behavior analysis system (including student gaze trajectory, frequency of facial orientation changes, and segmented statistics of attention duration). The model processes feature vectors in parallel using multiple decision trees, analyzes the statistical correlation of accuracy changes in a subset of samples with attention durations exceeding 10 minutes, and uncovers the association rules between attention levels and answer performance.

[0047] The dynamic routing mechanism configures the weighting coefficients of indicators based on the assessment objective. When the assessment objective is the students' knowledge mastery, the routing strategy increases the weighting of the answer accuracy rate and error correction rate indicators in the overall score. When the assessment objective shifts to the analysis of the rationality of question design, the weighting coefficient of the dispersion of the accuracy rate distribution of questions at different difficulty levels is increased. The weighting configuration results are input into the assessment report generation engine.

[0048] The assessment report generation module integrates three types of data processing units: a knowledge gap analysis unit that links to the educational knowledge graph, mapping incorrect answer patterns to knowledge nodes to form a defect topology map; a class level analysis unit that uses clustering algorithms to statistically analyze the distribution ratio of students at different ability levels; and a teaching strategy suggestion unit that generates teaching parameter adjustment suggestions based on association rules output by random forest. The final report outputs structured data through a web service interface, and the front-end rendering engine transforms the data into visual components such as tables, matrices, ability distribution heatmaps, and teaching strategy tree diagrams.

[0049] The technical solution achieves a closed loop in teaching evaluation through multi-source data fusion: a standard answer database provides a benchmark, a random forest model establishes a quantitative correlation between behavior and performance, and a dynamic routing mechanism adapts to the changing needs of different evaluation scenarios. Visualized reports transform the analysis results into actionable teaching decision-making criteria, completing the entire process from data collection to teaching optimization.

[0050] Specifically, in the AI ​​classroom teaching quality assessment method based on image recognition described in this invention, the educational knowledge graph in step 4 stores the relationship between multi-disciplinary knowledge nodes and questions. When the modification area feature is detected, a high-frequency incorrect answer database of the same question type is extracted based on the educational knowledge graph, and the similarity between the character sequence and the high-frequency incorrect answer is calculated using the edit distance algorithm. If the similarity exceeds the preset threshold, skip step 4 of adjusting the segmentation mask and re-extracting local features, directly label the recognition result as the corresponding error type, and associate it with the knowledge weakness marker in the educational knowledge graph.

[0051] The educational knowledge graph is constructed as a multi-level graph structure database, where nodes represent subject knowledge point entities (such as "solution of a linear equation in one variable"), and edge relationships define the mapping rules between questions and knowledge points. When a modified area feature is detected, the system calls the graph's association query interface to retrieve a database of frequently misused answers for the same question type based on the current question ID. This database is built through historical teaching data mining and stores typical error patterns and their associated knowledge point identifiers (such as "sign reversal error" associated with knowledge points related to equation operation rules).

[0052] The similarity calculation module uses the edit distance algorithm to process the character sequence output by OCR. This algorithm calculates the minimum number of edit operations (insertion, deletion, and replacement of characters) between the current sequence and each entry in the high-frequency incorrect answer database, and converts the number of operations into a similarity score in the range of 0-1. The preset similarity threshold is set to 0.85. When the similarity of any entry exceeds this threshold, a fast annotation mechanism is triggered.

[0053] The rapid annotation mechanism skips the conventional mask adjustment and feature re-extraction process, directly labeling the recognition results as the corresponding error type code (e.g., error type ID E201). Through the graph's relational mapping interface, the error type ID is associated with a knowledge point weakness identifier (e.g., knowledge point K0312). This identifier carries three layers of associated data: the knowledge module number, a description of common error causes, and an index of remedial teaching resources.

[0054] The technical solution utilizes a knowledge graph to achieve semantic processing of error identification: a high-frequency error answer database provides prior knowledge of typical error patterns, an edit distance algorithm quantifies the similarity between the identification result and typical errors, and a threshold mechanism controls the conditions for process jumps. The technical advantage of skipping the image-level optimization process is that for typical errors with obvious features, efficient annotation can be achieved directly using semantic information, while simultaneously associating with the knowledge point data required for teaching analysis, providing structured input for the generation of evaluation reports.

[0055] Specifically, the AI-based classroom teaching quality assessment method based on image recognition described in this invention, in addition to calculating the spatial distribution entropy of handwritten pixels based on the statistical ratio of printed and handwritten pixels, incorporates a dynamic routing mechanism: Information entropy is calculated based on the pixel coordinates of the handwritten characters. When the entropy value is higher than a set threshold, the handwriting is judged to be illegible. In the case of illegible handwriting, the local feature weights of the visual Transformer branch are increased to more than 70% of the total output of the two branches, while the weight ratio of the convolutional neural network branch is reduced simultaneously.

[0056] The dynamic routing mechanism, based on the statistical analysis of the proportion of printed / handwritten pixels, introduces a module for calculating the spatial distribution entropy of handwritten pixels. This module obtains the set of handwritten pixel coordinates in the answer area marked in step 1 and divides the image plane into M×N grid cells. By statistically analyzing the distribution frequency of handwritten pixels within each cell, the probability distribution function P(i,j) is calculated. Based on the Shannon entropy formula, the spatial distribution entropy value H = -ΣP(i,j)log2P(i,j) is calculated, where the entropy value represents the degree of spatial disorder of the handwritten pixels.

[0057] The handwriting sloppiness detection module sets an entropy threshold parameter. When the calculated entropy value H exceeds the preset threshold, the handwriting sloppiness feature in that area is determined to be significant (e.g., H > 5.0 is considered sloppiness). The routing policy controller responds to the sloppiness status signal by dynamically adjusting the dual-branch feature fusion parameters: increasing the weight coefficient of the visual Transformer branch in the final feature map to above 0.7, while simultaneously reducing the weight of the convolutional neural network branch to below 0.3. The magnitude of the weight adjustment is positively correlated with the degree to which the entropy value deviates from the threshold.

[0058] The technical solution addresses the feature extraction adaptation problem caused by changes in handwriting quality through entropy quantification: spatially distributed entropy objectively reflects the disorder of handwriting structure, with high entropy indicating high stroke position dispersion and enhanced structural irregularity. The effect of strengthening the visual Transformer branch weights in the illegible state is that its self-attention mechanism better captures unstructured local details (such as subtle gaps between overlapping strokes), while reducing the weights of the convolutional neural network branches avoids mismatches of regular convolutional kernels in disordered regions. This extension mechanism enables the dual-branch feature extraction network to adapt to handwriting quality, improving the stability of subsequent segmentation and recognition.

[0059] Specifically, the AI-based classroom teaching quality assessment method based on image recognition described in this invention includes the following: Modification area feature optimization recognition. Based on the stroke trajectory data of the standard answer in the educational knowledge graph, the gray-level gradient direction of the intersection point of the modified area is calculated. If the offset angle from the standard trajectory is greater than 15°, it is determined to be a valid modification. Based on the modified pattern library type called in step 4: For stroke-covering modifications, a contour-filling algorithm based on connected component analysis is used to restore the covered original answer strokes; For erased modifications, the boundary contrast is enhanced by using the Laplacian operator, and then the handwritten and printed text is re-entered for recognition by the hybrid OCR model.

[0060] The valid modification determination module calls upon the standard answer stroke trajectory vector data stored in the educational knowledge graph. This module first locates the coordinates of the intersection point of the modification area and calculates the grayscale gradient direction field of the 3×3 pixel area surrounding the intersection point using the Sobel operator. It then performs a dot product operation between the measured gradient direction and the standard stroke trajectory direction vector in the knowledge graph, calculating the cosine of the direction angle. When the direction offset angle exceeds a 15° threshold, a valid modification flag is triggered (e.g., a sudden change in writing direction indicates answer correction).

[0061] The stroke-covering modification processing unit performs operations based on the type identifier in the modification pattern library. A connected component analysis algorithm scans the binary image of the modification region, marking all closed contour boundary point sets. Based on the original answer stroke topology stored in the knowledge graph, a morphological reconstruction algorithm is used to restore the covered region: expanding outwards from the intersection points to fill connected regions and reconstruct the continuous trajectory of the strokes before coverage. The reconstructed stroke image is merged with the unmodified region to form a complete answer image input OCR recognition process.

[0062] The erasure-type modification processing unit applies a Laplacian convolution kernel for edge enhancement. This operator performs a second-order differential operation on the grayscale image of the modified region, highlighting the edge features of the remaining handwriting. The enhanced high-contrast image is then amplified through bilinear interpolation and re-input into a handwritten / printed hybrid OCR model for recognition. The recognition process incorporates character set constraints for this question type from the knowledge graph (e.g., math problems only contain numeric operators) to improve the accuracy of remaining handwriting recognition.

[0063] The technical solution addresses the challenge of modification recognition through a collaborative approach of physical features and semantic constraints: gradient direction analysis quantifies changes in stroke shape, connected component reconstruction restores the original structure, and edge enhancement improves the recognition rate of residual traces. The classification processing mechanism adapts to the physical characteristics of different modification types.

[0064] Secondly, the image recognition-based AI classroom teaching quality assessment system provided by the present invention, applied to the image recognition-based AI classroom teaching quality assessment method, includes: The data acquisition unit is configured to acquire RGB image data of the test paper and structured metadata including question area coordinates, question type labels, font type, and standard answer template; The data preprocessing unit is configured to perform grayscale conversion and median filtering on RGB image data, map the standard region coordinates of structured metadata to the actual image pixel coordinates through affine transformation, generate multimodal input data with annotations for the question stem area, answer area, and modification area, and transmit it to the feature extraction unit. The feature extraction unit, including a convolutional neural network branch and a visual Transformer branch, is configured to extract the global structural features of printed characters and the local detail features of handwritten characters in multimodal input data. The feature output weights of the two branches are dynamically adjusted according to the pixel ratio of printed and handwritten characters to generate differential feature maps and transmit them to the multi-attention segmentation unit. The multi-attention segmentation unit is configured to suppress the response of printed pixels in the question stem area and enhance the activation of handwritten boundary pixels in the answer area through a spatial attention mechanism. It adjusts the feature channel weights according to the font type through a channel attention mechanism and dynamically fuses spatial and channel attention according to the font pixel ratio to generate a segmentation mask that annotates the printed text of the question stem, the handwritten text of the answer, and modifies the coverage area, and transmits it to the context verification and recognition unit. The context verification recognition unit is configured to extract answer area and modification area sub-images based on segmentation mask, input a pre-trained handwritten and printed hybrid OCR model to generate character sequences, call a pre-built educational knowledge graph to obtain the standard answer template corresponding to the real-time question and the modification pattern of "3→8" stroke coverage, if the character sequence does not match the standard answer template or the modification area features are detected, adjust the segmentation mask and re-extract local features to optimize the recognition results, and transmit them to the quality assessment generation unit; The quality assessment generation unit is configured to compare the optimized recognition results with the standard answer database, statistically analyze evaluation indicators such as answer accuracy and correction error rate, integrate classroom attention analysis data, mine the correlation rules between answer performance and attention level through a random forest model, adjust the indicator weights according to the evaluation objectives, and generate a teaching quality assessment report that includes students' knowledge weaknesses, the overall level of the class, and teachers' teaching suggestions.

[0065] Please see Figure 1 The AI-based classroom teaching quality assessment method based on image recognition described in this invention includes: In educational applications of test paper grading, the AI-based classroom teaching quality assessment method based on image recognition described in this invention can be implemented as follows: First, industrial-grade image acquisition devices (supporting autofocus and illumination compensation) deployed on the ceiling of the classroom or above the marking station are used to acquire RGB image data of the exam papers in real time, and simultaneously obtain structured metadata through the academic affairs system API interface. This metadata includes the standard area coordinates of the questions in the exam paper template (such as the upper left corner of the question stem area). , bottom right corner , The data includes question type labels (multiple-choice or open-ended), font type (SimSun or KaiTi), and standard answer templates (e.g., format constraints like "x=□"). The collected RGB image data requires preprocessing: three-channel images are converted to single-channel grayscale to reduce data dimensionality; median filtering removes speckle noise generated during scanning; and affine transformation maps the standard region coordinates in the metadata to the pixel coordinates of the actual image (addressing coordinate offset issues caused by tilted test paper placement). Finally, multimodal input data labeled "Question Stem Area," "Answer Area," and "Modification Area" is generated, providing prior region localization for subsequent segmentation.

[0066] The preprocessed multimodal input data is fed into a dual-branch feature extraction network. The Convolutional Neural Network (CNN) branch uses 3×3 convolutional kernels to extract global structural features (such as stroke regularity and row / column alignment) for printed text. The Visual Transformer (ViT) branch divides the image into 16×16 pixel blocks through block embedding and uses a self-attention layer to extract local detail features of handwritten text (such as curvature changes in stroke paths and gradient distribution of ink density). A dynamic routing mechanism determines the pixel ratio of printed and handwritten text by statistically analyzing the area ratio of the question stem area to the answer area: if the printed text ratio exceeds a threshold (e.g., 60%), the feature output weights of the CNN branch are increased (to avoid information redundancy caused by global attention in the ViT branch); if the handwritten text ratio exceeds the threshold, the feature fusion ratio of the ViT branch is increased (to enhance the ability to capture local details). Finally, through dot product operations and layer normalization, a differentiated feature map that fuses the global structure of printed text and the local details of handwritten text is generated.

[0067] After the differential feature maps are input into the segmentation network, the spatial attention mechanism calculates the gray-level difference and positional distance between adjacent pixels to generate contextual weights: suppressing the response values ​​of printed pixels in the question stem area (by setting negative weights), while enhancing the activation intensity of handwritten boundary pixels in the answer area (by setting positive weights). The channel attention mechanism assigns higher weights to the handwritten feature channels based on the font type label (printed / handwritten) in the metadata, guiding the network to focus on the boundary pixels of the mixed region. The dynamic routing control mechanism adjusts the attention fusion ratio based on the pixel ratio of printed and handwritten text: when the handwritten text ratio is high, the spatial attention weight is increased (strengthening boundary localization); when the printed text ratio is high, the channel attention weight is increased (highlighting font type differentiation). The final output includes a pixel-level labeled segmentation mask that includes "printed question stem," "handwritten answer," and "modified coverage area."

[0068] Sub-images of the "answer area" and "modification area" are extracted based on the segmentation mask and input into a pre-trained handwritten and printed hybrid OCR model to generate character sequences. Simultaneously, a pre-built educational knowledge graph is invoked to obtain the standard answer template (e.g., "x=numerical value") and common modification patterns (e.g., the stroke coverage pattern of "3→8"). If the character sequence does not match the standard answer template (e.g., "x=seven" or "x=7.5" exceeds the integer range), or modification area features are detected (e.g., intersecting strokes), the boundaries of the segmentation mask are adjusted (expanding or shrinking the "modification area") and local features are re-extracted. The recognition results are optimized by combining the rule in the knowledge graph that "the gray value of the pixel at the intersection point is higher than the surrounding area" (e.g., correctly recognizing the "8" in the intersecting strokes instead of misidentifying it as "3" or "∞").

[0069] The optimized recognition results are compared with a pre-stored standard answer database to statistically analyze quantitative indicators such as answer accuracy (number of correct answers / total number of questions) and error rate (number of incorrectly identified modifications / total number of modifications). Simultaneously, classroom attention analysis data (such as students' focus duration and frequency of confused expressions) is acquired, and a random forest model is used to mine the correlation rules between answer performance and attention levels (e.g., students with focus duration ≥ 10 minutes show a significant increase in answer accuracy). A dynamic routing mechanism adjusts the indicator weights according to the assessment objective: if the assessment objective is students' knowledge mastery, the weights of accuracy and error rate are increased; if the assessment objective is the rationality of teachers' question design, the weights of the distribution analysis of accuracy rates for questions of different difficulty levels are increased. Finally, a teaching quality assessment report is generated, including individual student knowledge weaknesses (e.g., high error rate in solving linear equations), overall class mastery level (e.g., 80% of students can correctly answer basic questions), and teacher suggestions for question design (e.g., "increase the proportion of medium-difficulty questions"), and is visualized in tables and charts via a web interface.

[0070] Through the above specific implementation process, the present invention effectively solves the problem of insufficient accuracy in character segmentation and recognition in mixed handwritten and printed text scenarios, improves the practicality of AI classrooms in the test paper scoring process, and provides data support for precise teaching in educational scenarios.

[0071] In educational applications of test paper grading, the AI-based classroom teaching quality assessment system based on image recognition described in this invention can achieve its functions through the following specific implementation process: The system is deployed in classrooms or marking areas, and its core consists of a data acquisition unit, a data preprocessing unit, a feature extraction unit, a multi-attention segmentation unit, a context verification and recognition unit, and a quality assessment generation unit working together.

[0072] The data acquisition unit uses industrial-grade image acquisition equipment (deployed on the classroom ceiling or above the marking station), supporting autofocus and illumination compensation functions to acquire RGB three-channel image data of the exam paper in real time; simultaneously, it obtains structured metadata through the academic affairs system API interface, including the standard area coordinates of the questions in the exam paper template (such as the upper left corner of the question stem area). , bottom right corner , The system includes: question type labels (multiple-choice or open-ended questions), font type (SimSun or KaiTi), and standard answer templates (e.g., format constraints such as "x=□"). The collected image data and metadata are then transferred to the data preprocessing unit.

[0073] After receiving the data, the data preprocessing unit first performs grayscale processing on the RGB image, converting the three-channel image into a single-channel grayscale image to reduce data dimensionality. Then, a median filtering algorithm is used to remove speckle noise generated during the scanning process (such as local interference caused by paper wrinkles or equipment errors). Subsequently, an affine transformation algorithm is used to map the standard region coordinates in the metadata to the pixel coordinates of the actual image (solving the coordinate offset problem caused by the tilt of the test paper), generating multimodal input data labeled "question stem area," "answer area," and "modification area," and transmitting it to the feature extraction unit.

[0074] The feature extraction unit comprises a Convolutional Neural Network (CNN) branch and a Visual Transformer (ViT) branch. The CNN branch receives multimodal input data and extracts global structural features of printed text (such as stroke regularity and row / column alignment) using 3×3 convolutional kernels. The ViT branch divides the image into 16×16 pixel blocks and extracts local detail features of handwritten text (such as curvature variations in stroke paths and gradient distribution of ink density) through block embedding and self-attention layers. A dynamic routing mechanism determines the pixel ratio of printed and handwritten text by statistically analyzing the area ratio of the question stem area to the answer area: if the printed text ratio exceeds a preset threshold (e.g., 60%), the feature output weights of the CNN branch are increased (to avoid information redundancy caused by global attention in the ViT branch); if the handwritten text ratio exceeds the threshold, the feature fusion ratio of the ViT branch is increased (to enhance the ability to capture local details). Finally, a differentiated feature map fusing the global structure of printed text and the local details of handwritten text is generated and transmitted to the multi-attention segmentation unit.

[0075] The multi-attention segmentation unit, based on differential feature maps, calculates the gray-level difference and positional distance between adjacent pixels through a spatial attention mechanism to generate contextual weights: suppressing the response values ​​of printed pixels in the question stem area (by setting negative weights) and enhancing the activation intensity of handwritten boundary pixels in the answer area (by setting positive weights). The channel attention mechanism assigns higher weights to handwritten feature channels based on font type labels (printed / handwritten) in the metadata, guiding the network to focus on boundary pixels of the mixed region. The dynamic routing control mechanism adjusts the attention fusion ratio based on the pixel ratio of printed and handwritten text: increasing spatial attention weights when handwritten text has a high proportion (strengthening boundary localization), and increasing channel attention weights when printed text has a high proportion (highlighting font type distinction). The final output includes a pixel-level labeled segmentation mask containing "printed question stem," "handwritten answer," and "modified coverage area," which is then transmitted to the context verification and recognition unit.

[0076] The context verification recognition unit extracts sub-images of the "answer area" and "modification area" based on the segmentation mask, and inputs them into a pre-trained handwritten and printed hybrid OCR model to generate character sequences. Simultaneously, it calls a pre-built educational knowledge graph to obtain the standard answer template (e.g., "x=numerical value") and the modification pattern of stroke coverage "3→8" for the real-time questions. If the character sequence does not match the standard answer template (e.g., "x=seven" or "x=7.5" exceeds the integer range), or if modification area features are detected (e.g., intersecting strokes), the boundary of the segmentation mask is adjusted (expanding or shrinking the "modification area") and local features are re-extracted. The recognition result is optimized by combining the rule in the knowledge graph that "the gray value of the pixel at the intersection point is higher than the surrounding pixels" (e.g., correctly recognizing the "8" in the intersecting strokes instead of misidentifying it as "3" or "∞"). The optimized recognition result is then transmitted to the quality assessment generation unit.

[0077] The quality assessment generation unit compares the optimized recognition results with a pre-stored standard answer database, and calculates quantitative indicators such as answer accuracy rate (the ratio of correct answers to the total number of questions) and modification trace misidentification rate (the ratio of misidentified modifications to the total number of modifications). Simultaneously, it acquires classroom attention analysis data (such as students' focus duration and frequency of confused expressions while answering questions), and uses a random forest model to mine the correlation rules between answer performance and attention levels (e.g., students with focus duration ≥ 10 minutes show a significant increase in answer accuracy). A dynamic routing mechanism adjusts the indicator weights according to the assessment objective: if the assessment objective is students' knowledge mastery, the weights of accuracy rate and modification misidentification rate are increased; if the assessment objective is the rationality of teachers' question design, the weights of the distribution analysis of accuracy rates for questions of different difficulty levels are increased. Finally, it generates a teaching quality assessment report that includes individual student knowledge weaknesses (e.g., high error rate in solving linear equations), overall class mastery level (e.g., 80% of students can correctly answer basic questions), and teacher question design suggestions (e.g., "increase the proportion of medium-difficulty questions"), and visualizes the report in tables and charts via a web interface.

[0078] Through the coordinated operation of the above units, the system effectively solves the problem of insufficient accuracy in text segmentation and recognition in mixed handwritten and printed text scenarios, improves the practicality of AI classroom in the test paper scoring process, and provides data support for precise teaching in educational scenarios.

Claims

1. An AI-based classroom teaching quality assessment method based on image recognition, characterized in that, include: Step 1: Obtain the RGB image data of the test paper and the structured metadata, including the coordinates of the question area, question type labels, font type, and standard answer template; Image data is converted to grayscale and denoised using median filtering. Affine transformation is used to map the standard region coordinates of metadata to the actual image pixel coordinates, generating multimodal input data with labeled question stem area, answer area, and modification area. Step 2: Input the multimodal input data generated in Step 1 into the dual-branch feature extraction network, where the convolutional neural network branch extracts the global structural features of printed characters and the visual Transformer branch extracts the local detail features of handwritten characters. The feature output weights of the dual branches are dynamically adjusted according to the pixel ratio of printed and handwritten characters to generate a differential feature map that integrates the global structure of printed characters and the local details of handwritten characters. Step 3: Input the differential feature map generated in Step 2 into the segmentation network. Use the spatial attention mechanism to suppress the response of printed pixels in the question stem area and enhance the activation of pixels at the boundary of handwritten characters in the answer area. Use the channel attention mechanism to adjust the feature channel weights according to the font type and dynamically fuse spatial and channel attention according to the font pixel ratio. Output the segmentation mask that annotates the printed text of the question stem, the handwritten text of the answer, and the modified coverage area. Step 4: Based on the segmentation mask generated in Step 3, extract the answer area and modification area sub-images, input them into the pre-trained handwritten and printed hybrid OCR model to generate character sequences, call the pre-built educational knowledge graph to obtain the standard answer template and common modification patterns corresponding to the real-time questions, if the character sequence does not match the standard answer template or modification area features are detected, adjust the segmentation mask and re-extract local features to optimize the recognition results; Step 5: Compare the optimized recognition results from Step 4 with the pre-stored standard answer database, and statistically analyze evaluation indicators such as answer accuracy and correction error rate. Integrate classroom attention analysis data and use a random forest model to mine the correlation rules between answer performance and attention level. Adjust the indicator weights according to the evaluation objectives to generate a teaching quality evaluation report that includes students' knowledge weaknesses, the overall class level, and teachers' teaching suggestions.

2. The AI-based classroom teaching quality evaluation method based on image recognition according to claim 1, characterized in that, In step 1, the image data is acquired by industrial-grade image acquisition equipment deployed in classrooms or marking areas. The industrial-grade image acquisition equipment supports autofocus and illumination compensation. The structured metadata is synchronously acquired through the teaching affairs system API interface, including the standard area coordinates of the questions in the test paper template, the question type labels of multiple-choice or problem-solving questions, the printed font type of Song or Kai, and the standard answer template in a preset format, which is used for coordinate mapping of subsequent affine transformations.

3. The AI-based classroom teaching quality evaluation method based on image recognition according to claim 1, characterized in that, In step 2, the CNN branch of the dual-branch feature extraction network takes the multimodal input data generated in step 1 as input, uses 3×3 convolution kernels to extract global structural features of the stroke regularity and row and column alignment of printed characters, divides the image into 16×16 pixel blocks, and uses self-attention layers to extract local detail features such as curvature of the pen trajectory and ink gradient of handwritten characters. The dynamic routing mechanism is based on the area ratio of the question stem area and the answer area marked in step 1. It calculates the pixel ratio of printed and handwritten characters. If the printed character ratio is high, the feature output weight of the CNN branch is enhanced to generate a differential feature map.

4. The AI-based classroom teaching quality evaluation method based on image recognition according to claim 1, characterized in that, In step 3, the spatial attention mechanism takes the differential feature map generated in step 2 as input, calculates the gray-level difference and positional distance between adjacent pixels, and generates contextual association weights that suppress the response of printed pixels in the question stem area and enhance the activation of handwritten boundary pixels in the answer area. The channel attention mechanism assigns higher weights to the handwritten feature channels based on the font type labels in the metadata from step 1. The dynamic routing control mechanism is based on the font pixel ratio statistics in step 2. When the handwritten font ratio is high, spatial attention weight is increased, and when the printed font ratio is high, channel attention weight is increased. The output includes pixel-level annotation segmentation masks for the printed question stem, the handwritten answer, and the modified coverage area.

5. The AI-based classroom teaching quality evaluation method based on image recognition according to claim 1, characterized in that, In step 4, the pre-trained handwritten and printed hybrid OCR model takes the answer area and modification area sub-images generated in step 3 as input to generate character sequences; If the character sequence does not match the standard answer template or if modified area features are detected, adjust the segmentation mask generated in step 3 and re-extract local features. Combine the pattern that the gray value of the intersection pixel in the modified mode is higher than that of the surrounding pixels to optimize the recognition result.

6. The AI-based classroom teaching quality evaluation method based on image recognition according to claim 1, characterized in that, Step 5 involves storing the standard answers for each question type in the standard answer database; The random forest model uses the optimized recognition results from step 4 and classroom attention analysis data as input to analyze the accuracy improvement patterns of students with a focus duration of ≥10 minutes. The dynamic routing mechanism adds weights to accuracy and error rate when assessing students’ knowledge mastery, and adds weights to the accuracy distribution of questions of different difficulty when assessing the rationality of question design. The assessment report is presented in tables and charts via a web interface.

7. The AI-based classroom teaching quality evaluation method based on image recognition according to claim 6, characterized in that, In step 4, the educational knowledge graph stores the relationship between multi-disciplinary knowledge nodes and questions. When the modification area feature is detected, a high-frequency incorrect answer database of the same question type is extracted based on the educational knowledge graph, and the similarity between the character sequence and the high-frequency incorrect answer is calculated using the edit distance algorithm. If the similarity exceeds the preset threshold, skip step 4 of adjusting the segmentation mask and re-extracting local features, directly label the recognition result as the corresponding error type, and associate it with the knowledge weakness marker in the educational knowledge graph.

8. The AI-based classroom teaching quality evaluation method based on image recognition according to claim 7, characterized in that, The dynamic routing mechanism, based on the statistical ratio of printed and handwritten pixels, adds the calculation of the spatial distribution entropy value of handwritten pixels: Information entropy is calculated based on the pixel coordinates of the handwritten characters. When the entropy value is higher than a set threshold, the handwriting is judged to be illegible. In the case of illegible handwriting, the local feature weights of the visual Transformer branch are increased to more than 70% of the total output of the two branches, while the weight ratio of the convolutional neural network branch is reduced simultaneously.

9. The AI-based classroom teaching quality evaluation method based on image recognition according to claim 8, characterized in that, The modified region feature optimization identification includes: Based on the stroke trajectory data of the standard answer in the educational knowledge graph, the gray-level gradient direction of the intersection point of the modified area is calculated. If the offset angle from the standard trajectory is greater than 15°, it is determined to be a valid modification. Based on the modified pattern library type called in step 4: For stroke-covering modifications, a contour-filling algorithm based on connected component analysis is used to restore the covered original answer strokes; For erased modifications, the boundary contrast is enhanced by using the Laplacian operator, and then the handwritten and printed text is re-entered for recognition by the hybrid OCR model.

10. An AI-based classroom teaching quality assessment system based on image recognition, applied to the AI-based classroom teaching quality assessment method based on image recognition as described in any one of claims 1 to 9, characterized in that, include: The data acquisition unit is configured to acquire RGB image data of the test paper and structured metadata including question area coordinates, question type labels, font type, and standard answer template; The data preprocessing unit is configured to perform grayscale conversion and median filtering on RGB image data, map the standard region coordinates of structured metadata to the actual image pixel coordinates through affine transformation, generate multimodal input data with annotations for the question stem area, answer area, and modification area, and transmit it to the feature extraction unit. The feature extraction unit, including a convolutional neural network branch and a visual Transformer branch, is configured to extract the global structural features of printed characters and the local detail features of handwritten characters in multimodal input data. The feature output weights of the two branches are dynamically adjusted according to the pixel ratio of printed and handwritten characters to generate differential feature maps and transmit them to the multi-attention segmentation unit. The multi-attention segmentation unit is configured to suppress the response of printed pixels in the question stem area and enhance the activation of handwritten boundary pixels in the answer area through a spatial attention mechanism. It adjusts the feature channel weights according to the font type through a channel attention mechanism and dynamically fuses spatial and channel attention according to the font pixel ratio to generate a segmentation mask that annotates the printed text of the question stem, the handwritten text of the answer, and modifies the coverage area, and transmits it to the context verification and recognition unit. The context verification recognition unit is configured to extract answer area and modification area sub-images based on segmentation mask, input a pre-trained handwritten and printed hybrid OCR model to generate character sequences, call a pre-built educational knowledge graph to obtain the standard answer template and stroke coverage modification mode corresponding to the real-time question, if the character sequence does not match the standard answer template or the modification area features are detected, adjust the segmentation mask and re-extract local features to optimize the recognition results, and transmit them to the quality assessment generation unit. The quality assessment generation unit is configured to compare the optimized recognition results with the standard answer database, statistically analyze evaluation indicators such as answer accuracy and correction error rate, integrate classroom attention analysis data, mine the correlation rules between answer performance and attention level through a random forest model, adjust the indicator weights according to the evaluation objectives, and generate a teaching quality assessment report that includes students' knowledge weaknesses, the overall level of the class, and teachers' teaching suggestions.

Images