Real-time error annotation system and method for semantic segmentation of geometric figures

Abstract

The present application relates to the technical field of intelligent education, and particularly relates to a real-time wrong labeling system and method for geometric figure semantic segmentation, comprising: a figure collection module that collects student hand-drawn geometric figure images and identifies geometric primitives; an image preprocessing module that scales, crops and data enhances the images; a figure semantic segmentation module that includes a topology-sensitive hollow convolutional network, a hierarchical multi-scale channel attention mechanism and a graph theory-driven geometric structure reasoning mechanism, which performs pixel-level segmentation on the images and detects geometric errors; and a figure semantic visualization module that differentially visualizes and labels the segmentation results and errors, improves line feature recognition accuracy through the topology-sensitive hollow convolutional network, enhances key geometric features through the hierarchical multi-scale channel attention mechanism, and realizes error detection based on geometric rules through the graph theory-driven geometric structure reasoning mechanism, which can detect and label geometric problems such as parallel line angle errors and circle center offset errors in real time, thereby improving the efficiency of mathematics teaching.

Description

Technical Field

[0001] This invention relates to the field of intelligent education technology, and in particular to a real-time error annotation system and method for semantic segmentation of geometric figures. It can be applied to scenarios such as mathematics education and computer-aided teaching, and is used to identify and annotate errors in students' hand-drawn geometric figures in real time. Background Technology

[0002] In geometry teaching, students' understanding and ability to draw geometric figures are important learning objectives. Traditional teaching methods require teachers to manually check and annotate students' drawings, a time-consuming and inefficient process. While image recognition and semantic segmentation technologies are widely used in various fields due to the development of artificial intelligence, many challenges remain in geometric figure recognition and error annotation.

[0003] In existing technologies, image segmentation methods based on convolutional neural networks, such as FCN, U-Net, and DeepLab, have achieved certain results. However, these methods have the following shortcomings when processing hand-drawn geometric figures: First, the recognition accuracy of line features is not high, especially for hand-drawn irregular lines; second, the ability to understand geometric relationships is limited, and it is unable to effectively identify geometric constraints such as parallelism and perpendicularity; third, there is a lack of error detection and annotation mechanisms, which cannot provide effective learning feedback.

[0004] Currently, most geometric shape recognition systems on the market focus on the classification and recognition of shapes, lacking the ability to detect and label geometric errors. Traditional image processing methods struggle to handle the diversity and irregularities of hand-drawn graphics, while deep learning methods lack an understanding and application of geometric rules. Therefore, developing a system that combines deep learning with geometric rules to achieve real-time error detection and labeling is of great significance. Summary of the Invention

[0005] The purpose of this invention is to provide a real-time error annotation system and method for semantic segmentation of geometric figures, aiming to solve the problems of low accuracy in recognizing hand-drawn geometric figures, lack of understanding of geometric relationships, and error annotation mechanisms in the prior art.

[0006] This invention proposes a real-time error annotation system for semantic segmentation of geometric figures, comprising:

[0007] The image acquisition module is used to acquire images of students' hand-drawn geometric figures. Through an intelligent learning model, the geometric primitives of the hand-drawn geometric figures are identified to obtain the names, quantities, and types of geometric primitives of the students' hand-drawn geometric figures.

[0008] An image preprocessing module, connected to the image acquisition module, is used to receive hand-drawn geometric images sent by the image acquisition module and to perform scaling, cropping, and data enhancement processing on the hand-drawn geometric images.

[0009] A graphic semantic segmentation module, connected to the image preprocessing module, is used to receive preprocessed hand-drawn geometric image sent by the image preprocessing module and perform pixel-level segmentation on the preprocessed hand-drawn geometric image. The graphic semantic segmentation module includes a topology-sensitive dilated convolutional network, a hierarchical multi-scale channel attention mechanism, and a graph theory-driven geometric structure inference mechanism. The topology-sensitive dilated convolutional network is used to extract the topological features of the hand-drawn geometric image. The hierarchical multi-scale channel attention mechanism is used to enhance key geometric features and suppress background noise. The graph theory-driven geometric structure inference mechanism is used to construct the graph structure of the geometric image and detect errors based on geometric constraint rules.

[0010] The graphic semantic visualization module, connected to the graphic semantic segmentation module, is used to receive the segmentation results and error detection results of the graphic semantic segmentation module, visualize each geometric primitive identified by the graphic semantic segmentation module using different colors, styles and color gradient effects, and annotate the detected error areas through differentiated visual presentation.

[0011] Preferably, the image preprocessing module includes:

[0012] The scaling unit is used to scale hand-drawn geometric images with a length greater than 256 pixels and a width greater than 192 pixels into images with a pixel length of no more than 256 pixels and a width of no more than 192 pixels.

[0013] The cropping unit is used to proportionally crop a portion of a hand-drawn geometric image based on its length or width.

[0014] The data augmentation unit is used to perform horizontal mirroring, contrast enhancement, and random color dithering operations on hand-drawn geometric images.

[0015] Preferably, the topology-sensitive dilated convolutional network includes:

[0016] Dilated convolutional layers replace the last two convolutional layers in the DeepLabv3+ model with dilated convolutions, keeping the number of convolutional kernels the same, with a dilation rate of 2 and a stride of 2.

[0017] The feature extraction layer is used to extract geometric features of hand-drawn geometric figures based on the dilated convolutional layer, thereby expanding the receptive field and enhancing the perception of line continuity and intersection relationships.

[0018] Preferably, the hierarchical multi-scale channel attention mechanism includes:

[0019] The feature layer is used to connect the results of the first and second layers of the DeepLabv3+ model, and is processed using global pooling and maximum functions to generate a feature representation map;

[0020] The scaling layer is used to process the input features using global pooling, the maximum value function, and the average value function, respectively, to generate feature representations at three different scales.

[0021] The channel attention layer is used to input the output of the feature layer and the three outputs of the scale layer into the SoftMax function to generate channel weights, and then multiply the channel weights with the input features to enhance key geometric features and suppress background noise.

[0022] Preferably, the graph theory-driven geometric structure reasoning mechanism includes:

[0023] The graph structure building unit is used to identify geometric elements based on pixel-level segmentation results, extract key attributes of geometric elements, and construct a graph structure representing geometric elements and their relationships.

[0024] A geometric constraint verification unit is used to define a set of geometric constraint rules, apply the set of geometric constraint rules to the graph structure, and identify substructures that violate the constraints.

[0025] The error localization unit is used to accurately locate the identified substructures that violate constraints, calculate error parameters, and generate error annotation data.

[0026] Preferably, the geometric constraint verification unit includes:

[0027] The parallel line error detection unit is used to mark the intersection point of two parallel lines when the angle between them in a geometric figure is greater than 5°.

[0028] The center error detection unit is used to detect when the centroid of all circles in a geometric figure is located on the outline of other circles and the offset is greater than 3 pixels, and then marks the outline of all circles.

[0029] Preferably, the graphic semantic visualization module includes:

[0030] The basic geometric shape unit is used to label hand-drawn line segments with black solid lines, hand-drawn ellipses with thick black solid lines, and hand-drawn circles with thick black solid lines.

[0031] Draw and label geometric units to mark incorrect parallel lines with red dashed lines, mark the center of the intersection of incorrect parallel lines with red solid lines, and mark the midpoints of two different arcs with the same radius and two arcs with different radii with green solid lines.

[0032] Preferably, the image acquisition module includes:

[0033] The geometric primitive name recognition unit is used to recognize the description of hand-drawn geometric images using a text classifier, and obtain the text information of the geometric image description;

[0034] The geometric primitive type identification unit is used to identify hand-drawn geometric images using an image classifier, and to obtain the geometric primitive types of the hand-drawn geometric images, as well as the quantity of each geometric primitive;

[0035] The geometric primitive quantity recognition unit is used to count the number of each type of geometric primitive to obtain the geometric primitive quantity.

[0036] Preferably, the geometric image includes doodle image, straight line image, straight line + arc, straight line + circle and arc + circle image; the number of geometric primitives N∈[0,18], and the geometric primitives include straight line, arc, rectangle, line segment, triangle, parallelogram, equilateral triangle, ellipse, circle, square, trapezoid, rhombus, isosceles triangle, sector, hexagon and pentagon.

[0037] A real-time error annotation method for semantic segmentation of geometric figures includes the following steps:

[0038] The graphic acquisition process involves collecting images of students' hand-drawn geometric figures, and using an intelligent learning model to identify the geometric primitives in the hand-drawn geometric figures to obtain the names, quantities, and types of geometric primitives in the students' hand-drawn geometric figures.

[0039] The image preprocessing step involves scaling, cropping, and data augmentation of the hand-drawn geometric image.

[0040] The graphic semantic segmentation step performs pixel-level segmentation on the preprocessed hand-drawn geometric graphic image, including: extracting the topological features of the hand-drawn geometric graphic through a topology-sensitive dilated convolutional network; enhancing key geometric features and suppressing background noise through a hierarchical multi-scale channel attention mechanism; and constructing the graph structure of the geometric graphic through a graph theory-driven geometric structure reasoning mechanism and detecting errors based on geometric constraint rules.

[0041] The graphic semantic visualization step visualizes each geometric primitive identified in the graphic semantic segmentation step using different colors, styles, and color gradient effects, and marks the detected error areas using differentiated visual presentation methods.

[0042] The beneficial effects of this invention include:

[0043] 1. The recognition accuracy of hand-drawn irregular lines is improved by using topology-sensitive dilated convolutional networks, enabling the system to accurately segment various geometric primitives, with a recognition accuracy improvement of approximately 20% compared to traditional methods;

[0044] 2. The hierarchical multi-scale channel attention mechanism enhances the perception of key geometric features, effectively suppresses background noise, and improves robustness in complex backgrounds by approximately 28%.

[0045] 3. By introducing a graph theory-driven geometric structure reasoning mechanism, the system has achieved an improvement from pixel-level representation to semantic-level understanding, enabling it to detect errors based on geometric rules with an accuracy of over 95%.

[0046] 4. By using differentiated visual presentation methods, geometric errors are intuitively displayed, improving learning efficiency and comprehension, resulting in an increase in teaching efficiency of approximately 40%;

[0047] 5. The system achieves real-time processing in a GPU environment, controlling the processing time of hand-drawn graphics to the millisecond level, meeting the instant feedback needs of teaching scenarios. Attached Figure Description

[0048] Figure 1 This is a diagram showing the overall architecture of the real-time error annotation system for geometric semantic segmentation of the present invention.

[0049] Figure 2 This is a schematic diagram of the image preprocessing module of the present invention;

[0050] Figure 3 This is a schematic diagram of the topology-sensitive dilated convolutional network of the present invention;

[0051] Figure 4 This is a schematic diagram of the hierarchical multi-scale channel attention mechanism of the present invention;

[0052] Figure 5 This is a schematic diagram of the graph theory-driven geometric structure reasoning mechanism of the present invention;

[0053] Figure 6 This is a schematic diagram of the geometric constraint verification unit of the present invention;

[0054] Figure 7 This is a schematic diagram of the structure of the graphical semantic visualization module of the present invention;

[0055] Figure 8 This is a flowchart of the method of the present invention. Detailed Implementation

[0056] Please refer to Figures 1-8 The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The scope of protection of the present invention is not limited to the following description.

[0057] like Figure 1 As shown, the real-time error annotation system for geometric semantic segmentation provided by the present invention includes: a graphic acquisition module 1, an image preprocessing module 2, a graphic semantic segmentation module 3, and a graphic semantic visualization module 4.

[0058] The image acquisition module 1 is used to acquire images of students' hand-drawn geometric figures. Through an intelligent learning model, the geometric primitives of the hand-drawn geometric figures are identified to obtain the names, quantities, and types of geometric primitives of the students' hand-drawn geometric figures.

[0059] Image preprocessing module 2 is connected to image acquisition module 1 and is used to receive hand-drawn geometric images sent by image acquisition module 1, and to perform scaling, cropping and data enhancement processing on the hand-drawn geometric images.

[0060] The image semantic segmentation module 3 is connected to the image preprocessing module 2 and is used to receive the preprocessed hand-drawn geometric image sent by the image preprocessing module 2, and to perform pixel-level segmentation on the preprocessed hand-drawn geometric image. The image semantic segmentation module 3 includes a topology-sensitive dilated convolutional network 31, a hierarchical multi-scale channel attention mechanism 32, and a graph theory-driven geometric structure inference mechanism 33. The topology-sensitive dilated convolutional network 31 is used to extract the topological features of the hand-drawn geometric figure; the hierarchical multi-scale channel attention mechanism 32 is used to enhance key geometric features and suppress background noise; and the graph theory-driven geometric structure inference mechanism 33 is used to construct the graph structure of the geometric figure and detect errors based on geometric constraint rules.

[0061] The graphic semantic visualization module 4 is connected to the graphic semantic segmentation module 3. It is used to receive the segmentation results and error detection results of the graphic semantic segmentation module 3, visualize each geometric primitive identified by the graphic semantic segmentation module 3 using different colors, styles and color gradient effects, and annotate the detected error areas through differentiated visual presentation.

[0062] In one embodiment of the invention, the system runs on a GPU server, preferably an NVIDIA Tesla V100 or higher performance GPU equipped with at least 16GB of video memory to meet the computational requirements of the deep learning model. The system processes a hand-drawn image in an average of approximately 25 milliseconds, meeting the requirements for real-time feedback.

[0063] like Figure 2 As shown, the image preprocessing module 2 includes: a scaling unit 21, a cropping unit 22, and a data enhancement unit 23.

[0064] The scaling unit 21 is used to scale a hand-drawn geometric image with a length greater than 256 pixels and a width greater than 192 pixels into an image with a pixel length not exceeding 256 pixels and a width not exceeding 192 pixels. Preferably, the scaling process uses a bilinear interpolation algorithm to maintain the image aspect ratio and avoid distortion of geometric relationships.

[0065] The cropping unit 22 is used to crop a portion of the hand-drawn geometric image proportionally according to its length or width. Preferably, the cropping operation adopts a center cropping method, preserving the geometric information of the central area of ​​the image. The cropping ratio is adaptively adjusted according to the image content, generally controlled between 0.8 and 0.95.

[0066] The data augmentation unit 23 is used to perform horizontal mirroring, contrast enhancement, and random color dithering operations on the hand-drawn geometric image. Preferably, the contrast enhancement amplitude is controlled between 0.9 and 1.2, and the random color dithering range is controlled within ±15 of the RGB value to ensure that the enhanced image retains its original geometric features.

[0067] In one embodiment of the present invention, data augmentation operations may also include random rotation (within ±10°), minute translation (within the pixel range), etc., to improve the model's adaptability to different drawing poses and styles.

[0068] like Figure 3 As shown, the topology-sensitive dilated convolutional network 31 includes a dilated convolutional layer 311 and a feature extraction layer 312.

[0069] The dilated convolutional layer 311 replaces the convolutions in the last two layers of the DeepLabv3+ model with dilated convolutions, keeping the number of kernels unchanged, the dilation rate 2, and the stride 2. Preferably, the kernel size remains 3×3, and the number of output channels is the same as the original DeepLabv3+ model.

[0070] The feature extraction layer 312 is used to extract the geometric features of hand-drawn geometric figures based on the dilated convolutional layer 311, thereby expanding the receptive field and enhancing the ability to perceive the continuity and intersection of lines.

[0071] In one embodiment of the present invention, the mathematical expression for dilated convolution is:

[0072] ,

[0073] in: To output feature map at position The value at that location, For the input feature map, For convolution kernel weights, void ratio (in this invention) ), This is the kernel index.

[0074] When students hand-draw complex geometric figures, such as those containing multiple parallel or intersecting lines, traditional convolutional networks struggle to capture global geometric relationships due to their limited receptive field. For example, in a hand-drawn figure containing a pentagon and an inscribed circle, traditional convolution might only focus on local line segment features, failing to understand the overall geometric relationships. Dilated convolution, by inserting "holes" into the convolution kernel, allows the network to significantly expand its receptive field without increasing the number of parameters, thereby capturing a wider range of spatial relationships and better understanding the topological relationships between geometric elements.

[0075] Preferably, the feature extraction layer 312 adopts a cascaded structure to fuse feature maps with different hole rates, further improving the ability to extract multi-scale geometric features. Feature fusion uses channel concatenation, expressed as:

[0076] ,

[0077] in: The fused feature map For feature maps with different porosity, Concat is a concatenation operation along the channel dimension.

[0078] In practical applications, when processing a hand-drawn triangle image containing lines of varying thicknesses, the system can capture the overall outline of thick lines and the fine structure of thin lines through feature extraction layers with different dilation rates. Then, a comprehensive representation is obtained through feature fusion, improving its adaptability to various hand-drawn styles. For example, when students draw equilateral triangles, they may encounter problems such as uneven line thickness and inaccurate corners. Topology-sensitive dilated convolutional networks can effectively identify these irregular lines and correctly understand their expressed geometric intent.

[0079] like Figure 4 As shown, the hierarchical multi-scale channel attention mechanism 32 includes: a feature layer 321, a scale layer 322, and a channel attention layer 323.

[0080] Feature layer 321 is used to concatenate the results of the first and second layers of the DeepLabv3+ model, and processes them using global pooling and a maximum value function to generate a feature representation map. Preferably, the calculation process of the feature layer is as follows:

[0081] ,

[0082] in: For feature representation map, and These are the outputs of the first and second layers of the DeepLabv3+ model, respectively. Concat is the concatenation operation along the channel dimension, and MaxPool is the global max pooling operation.

[0083] The scaling layer 322 is used to process the input features using global pooling, the maximum value function, and the average value function, respectively, to generate feature representations at three different scales. Preferably, the calculation process of the scaling layer is as follows:

[0084] ,

[0085] ,

[0086] ,

[0087] in: , and These represent features at three different scales. The input feature map is GlobalPool, which is the global pooling operation, MaxValue is the maximum value function, and AvgValue is the average value function.

[0088] The channel attention layer 323 is used to input the output of the feature layer 321 and the three outputs of the scale layer 322 into the SoftMax function to generate channel weights. These channel weights are then multiplied by the input features to enhance key geometric features and suppress background noise. Preferably, the calculation process of the channel attention layer is as follows:

[0089] ,

[0090] ,

[0091] in: Here, Conv represents the channel weights, and it is a 1×1 convolution operation. This indicates that the feature layer output and features at three scales are concatenated, with SoftMax being the softmax activation function. For the enhanced feature map, This is a multiplication operation for the channel dimension.

[0092] In real-world scenarios of geometric shape recognition, students often hand-draw geometric shapes under various background conditions (such as grid paper, lined paper, etc.), and background lines can easily interfere with shape recognition. For example, a parallelogram drawn on grid paper may be confused with background grid lines. The hierarchical multi-scale channel attention mechanism significantly improves the system's recognition accuracy in complex backgrounds by adaptively adjusting channel weights, highlighting geometric line features, and suppressing interference such as background grid lines.

[0093] Practice has shown that this mechanism is also highly adaptable to hand-drawn defects such as discontinuous lines and uneven line weight. For example, when a student's circular lines are lighter in some areas, the channel attention mechanism can enhance these weak features through global contextual information, ensuring the complete recognition of the circle.

[0094] like Figure 5 As shown, the graph theory-driven geometric structure reasoning mechanism 33 includes: a graph structure construction unit 331, a geometric constraint verification unit 332, and an error location unit 333.

[0095] The graph structure construction unit 331 is used to identify geometric elements based on pixel-level segmentation results, extract key attributes of the geometric elements, and construct a graph structure representing the geometric elements and their relationships. Preferably, the graph structure can be represented as follows: ,in This is a set of nodes (corresponding to geometric elements). It is a set of edges (corresponding to the relationships between elements).

[0096] For nodes Its attribute set can be represented as:

[0097] ,

[0098] in: For geometric element types (such as line segments, circles, etc.). The coordinates of the center point, For endpoint coordinates (for line segments). The radius (for a circle).

[0099] For the edge Its attribute set can be represented as:

[0100] ,

[0101] in: For geometric relationship types (such as parallel, intersecting, tangent, etc.). These are relational parameters (such as included angle, distance, etc.).

[0102] In real-world teaching scenarios, students may draw complex graphs involving multiple geometric relationships, such as geometric proofs involving parallel lines, perpendicular lines, and tangent circles. Graph structure building units can represent these complex relationships as graph structures, providing a foundation for subsequent error detection. For example, in an exercise on drawing a tangent line to a circle through two points, the system can identify basic elements such as points, circles, and line segments, and construct a graph structure representing their interrelationships.

[0103] The geometric constraint verification unit 332 is used to define a set of geometric constraint rules, apply the set of geometric constraint rules to the graph structure, and identify substructures that violate the constraints. For example... Figure 6 As shown, the geometric constraint inspection unit 332 includes a parallel line error detection unit 3321 and a circle center error detection unit 3322.

[0104] The parallel line error detection unit 3321 is used to mark the intersection points of erroneous parallel lines when the angle of erroneous parallel lines in a geometric figure is greater than 5°. Preferably, the formula for calculating the included angle of parallel lines is:

[0105] ,

[0106] in: Let be the included angle between the two line segments. and These are the direction vectors of the two line segments, Represents the vector dot product. It represents the magnitude of the vector.

[0107] when When the line is drawn incorrectly, it is considered a parallel line error. The choice of 5° as the threshold is based on teaching experience; this value tolerates slight deviations in students' hand-drawn sketches while effectively identifying obvious geometric errors. In elementary school geometry teaching, this threshold can be appropriately relaxed to 8°, while in high school geometry teaching with high precision requirements, it can be tightened to 3°.

[0108] The center-of-circle error detection unit 3322 is used to detect when the centroid of all circles in a geometric figure lies on the outline of other circles and the offset is greater than 3 pixels, and then marks the outline of all circles. Preferably, the formula for calculating the center-of-circle offset is:

[0109] ,

[0110] in: This is the offset. and These are the radii of the two circles, respectively. and These are the coordinates of the centers of the two circles, This represents the Euclidean distance between the centers of the two circles.

[0111] when When the deviation exceeds 3 pixels, it is considered an error in the center of the circle. The choice of 3 pixels as the threshold balances the precision limitations of hand-drawn graphics with teaching requirements. For example, when teaching the concepts of inscribed and circumscribed circles, students often need to draw two circles so that they are exactly inscribed or circumscribed. If the center position deviates by more than 3 pixels, the system will identify this error, helping students understand the property of the line connecting the point of tangency and the center of the circle.

[0112] The error localization unit 333 is used to accurately locate the identified substructures that violate constraints, calculate error parameters, and generate error annotation data. Preferably, the error annotation data includes error type, location, parameters, and visualization attributes for subsequent visualization processing.

[0113] In teaching practice, the accuracy of error detection is crucial for students' understanding of geometric concepts. For example, in parallel line error detection, the system not only marks that two lines are not parallel, but also accurately calculates the location of their intersection and the size of their angle, helping students intuitively understand the concept of parallel lines. Similarly, in concentric circle error detection, the system marks the direction and distance of the center offset, helping students grasp the definition and properties of concentric circles.

[0114] like Figure 7 As shown, the graphic semantic visualization module 4 includes: a basic geometric figure drawing unit 41 and a labeled geometric figure drawing unit 42.

[0115] The basic geometric shape drawing unit 41 is used to label hand-drawn line segments with solid black lines, hand-drawn ellipses with thick solid black lines, and hand-drawn circles with thick solid black lines. Preferably, the line width of the line segments is set to 2 pixels, and the line width of the ellipses and circles is set to 3 pixels to ensure clear visibility.

[0116] The drawing and annotation geometric unit 42 is used to mark incorrect parallel lines (i.e., lines that should be parallel but are not) with red dashed lines, to mark the center of the intersection of incorrect parallel lines with red solid lines, and to mark the midpoints of two arcs with different radii but the same radii and two arcs with different radii with green solid lines. Preferably, the line width of the dashed lines is set to 2 pixels, the dashed line spacing is set to (4,2) pixels, the line width of the solid line annotation is set to 3 pixels, and the radius of the annotation point is set to 5 pixels.

[0117] In practical teaching applications, the intuitiveness of visualizations is crucial for students to understand geometric errors. For example, when a student attempts to draw two parallel lines, if the system detects that the two lines actually intersect, it will redraw the two lines on the original drawing with a red dashed line and mark their intersection with a red solid line. This intuitive visual feedback allows students to immediately understand where the error lies, promoting the correct formation of geometric concepts.

[0118] Similarly, in circle drawing exercises, if the system detects that two circles drawn by a student should be concentric but their centers are actually misaligned, it will mark the center position with a solid green line to help students understand the definition and characteristics of concentric circles. This differentiated visual presentation makes different types of geometric errors immediately apparent, greatly improving the efficiency of teaching feedback.

[0119] The graphic acquisition module 1 includes: a geometric primitive name recognition unit 11, a geometric primitive type recognition unit 12, and a geometric primitive quantity recognition unit 13.

[0120] The geometric primitive name recognition unit 11 is used to recognize the description of hand-drawn geometric images using a text classifier, thereby obtaining the text information describing the geometric images. Preferably, the text classifier uses a pre-trained language model such as BERT, fine-tuned for geometric terms, achieving a recognition accuracy of over 95%.

[0121] The geometric primitive type recognition unit 12 is used to recognize hand-drawn geometric images using an image classifier, obtaining the types of geometric primitives in the hand-drawn geometric images and the quantity of each type of geometric primitive. Preferably, the image classifier adopts mainstream network architectures such as ResNet-50 or EfficientNet-B3, and adapts to the geometric recognition task through transfer learning techniques.

[0122] The geometric primitive count recognition unit 13 is used to count the number of each type of geometric primitive to obtain the number of geometric primitives. Preferably, the position and number of geometric elements in the graphic are determined by combining object detection technology (such as Faster R-CNN or YOLO).

[0123] In teaching scenarios, students may simultaneously draw complex figures composed of multiple geometric primitives, such as geometric proofs consisting of lines, circles, and triangles. The image acquisition module can simultaneously identify various primitives and their quantities in the image, providing a foundation for subsequent semantic segmentation and error detection. For example, in an exercise where two sides and one angle of a triangle are given, the system can identify that the hand-drawn image contains a triangle, three line segments, and an angle marker, and perform subsequent error detection accordingly.

[0124] In one embodiment of the present invention, the geometric graphic image includes doodle image, straight line image, straight line + arc, straight line + circle and arc + circle image; the number of geometric primitives N∈[0,18], and the geometric primitives include straight line, arc, rectangle, line segment, triangle, parallelogram, equilateral triangle, ellipse, circle, square, trapezoid, rhombus, isosceles triangle, sector, hexagon and pentagon.

[0125] like Figure 8 As shown, this invention also provides a real-time error annotation method for geometric semantic segmentation, comprising the following steps:

[0126] The graphic acquisition process involves collecting images of students' hand-drawn geometric figures, and using an intelligent learning model to identify the geometric primitives in the hand-drawn geometric figures to obtain the names, quantities, and types of geometric primitives in the students' hand-drawn geometric figures.

[0127] The image preprocessing steps involve scaling, cropping, and data augmentation of the hand-drawn geometric images.

[0128] The graph semantic segmentation step performs pixel-level segmentation on the preprocessed hand-drawn geometric image, including: extracting the topological features of the hand-drawn geometric image through a topology-sensitive dilated convolutional network; enhancing key geometric features and suppressing background noise through a hierarchical multi-scale channel attention mechanism; and constructing the graph structure of the geometric image through a graph theory-driven geometric structure reasoning mechanism and detecting errors based on geometric constraint rules.

[0129] The graphic semantic visualization step visualizes each geometric primitive identified in the graphic semantic segmentation step using different colors, styles, and color gradient effects, and annotates the detected error areas using differentiated visual presentation methods.

[0130] In one embodiment of the present invention, the model training in the image semantic segmentation step employs a cross-entropy loss function, expressed as follows:

[0131] ,

[0132] in: This is the loss value. For the sample size, For the number of categories, For the sample Category The true label (0 or 1), Predict samples for the model Category The probability of.

[0133] Cross-entropy loss function is effective in handling multi-class problems in geometric figure segmentation. In geometry teaching scenarios, the system needs to simultaneously identify multiple primitives such as lines, circles, and triangles. Cross-entropy loss function improves the accuracy of multi-class recognition by minimizing the difference between the predicted distribution and the true distribution. For example, when recognizing a figure containing a parallelogram and an inscribed circle, cross-entropy loss ensures that the system can accurately distinguish between straight line boundaries and circular boundaries.

[0134] Preferably, the model training uses the Adam optimizer with an initial learning rate of 0.001 and a learning rate decay strategy, decreasing to 0.9 times the original rate every 10 epochs. The batch size is set to 16, and training is performed for at least 100 epochs until convergence. To prevent overfitting, L2 regularization (with a weight decay coefficient of 0.0001) and early stopping strategies are also employed.

[0135] In the image preprocessing step, scaling prioritizes maintaining the image's aspect ratio to avoid geometric distortion. For images with significant aspect ratio differences, padding is used to resize the image to a standard size, with the pixel value of the padding area set to 255 (white background).

[0136] Furthermore, the method of this invention can also set differentiated judgment thresholds for different types of geometric errors. For example, for beginners, the threshold for the angle between parallel lines can be relaxed to 8°, and the threshold for the center offset can be relaxed to 5 pixels; for advanced learners, these thresholds are set to 3° and 2 pixels respectively, to adapt to the accuracy requirements of different learning stages.

[0137] The working process of the present invention is illustrated below through specific embodiments:

[0138] Example 1: Parallel Line Error Detection

[0139] The student hand-drew two line segments that should be parallel. The system first acquires the hand-drawn image through the image acquisition module 1, and then performs standardization processing through the image preprocessing module 2. The topology-sensitive dilated convolutional network 31 of the image semantic segmentation module 3 extracts line features, the hierarchical multi-scale channel attention mechanism 32 enhances key features, and the graph theory-driven geometric structure reasoning mechanism 33 constructs the graph structure and identifies the relationship between the two line segments. The geometric constraint verification unit 332 calculates that the angle between the two line segments is 7.2°, which exceeds the 5° threshold, so it is judged as an incorrect parallel line. The error localization unit 333 calculates the intersection position of the two line segments and generates error annotation data. Finally, the image semantic visualization module 4 draws the basic line segments with black solid lines and marks the errors with red solid lines at the intersection positions.

[0140] This implementation is applicable to common parallel line drawing exercises in teaching. For example, when a teacher asks students to draw a parallelogram, the system can detect in real time whether the opposite sides drawn by the student are truly parallel and provide intuitive error prompts. This instant feedback helps students establish a correct concept of parallelism and improves their geometry learning outcomes.

[0141] Example 2: Center Error Detection

[0142] The student hand-drew two circles, one of which should have its center at the center of the other (concentric circles), but there was a deviation in the actual drawing. The system processed the image through the aforementioned process and calculated the distance between the centers of the two circles to be 4.5 pixels in the graph theory-driven geometric structure inference mechanism 33, which exceeded the 3-pixel threshold, thus determining that the center was incorrect. The graphic semantic visualization module 4 drew the basic circle with a thick black solid line and marked the center offset error with a green solid line.

[0143] This implementation is applicable to teaching exercises related to concepts such as concentric circles, inscribed circles, and circumscribed circles. For example, when teaching the topic of the relationship between two circles, the system can help students understand the defining characteristic of concentric circles, namely, that the two circles share a common center. When students attempt to draw concentric circles but the center position is off, the system's real-time annotation can help students correct errors and deepen their understanding of geometric concepts.

[0144] As can be seen from the above embodiments, the system of the present invention can accurately identify errors in geometric figures and provide intuitive error annotations, effectively improving learning outcomes and teaching efficiency.

[0145] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of protection of the present invention.

Claims

1. A real-time mislabeled annotation system for geometric semantic segmentation, characterized in that, The method comprises the steps of: a figure collection module is used for collecting student hand-drawn geometric figure images, identifying geometric primitives of the hand-drawn geometric figures through an intelligent learning model, and obtaining geometric primitive names, geometric primitive quantities and geometric primitive categories of the student hand-drawn geometric figures; an image preprocessing module connected with the figure collection module is used for receiving the hand-drawn geometric figure images sent by the figure collection module, and performing scaling, cropping and data enhancement processing on the hand-drawn geometric figure images; a figure semantic segmentation module connected with the image preprocessing module is used for receiving the preprocessed hand-drawn geometric figure images sent by the image preprocessing module, and performing pixel-level segmentation on the preprocessed hand-drawn geometric figure images; the figure semantic segmentation module comprises a topologically sensitive hollow convolutional network, a hierarchical multi-scale channel attention mechanism and a graph theory driven geometric structure reasoning mechanism; the topologically sensitive hollow convolutional network is used for extracting topological features of the hand-drawn geometric figures; the hierarchical multi-scale channel attention mechanism is used for enhancing key geometric features and suppressing background noise; and the graph theory driven geometric structure reasoning mechanism is used for constructing a graph structure of the geometric figures and detecting errors based on geometric constraint rules; a figure semantic visualization module connected with the figure semantic segmentation module is used for receiving segmentation results and error detection results of the figure semantic segmentation module, visualizing and drawing each geometric primitive identified by the figure semantic segmentation module by using different colors, styles and color gradient effects, and labeling error areas detected by using a differential visual presentation method; the topologically sensitive hollow convolutional network comprises: a hollow convolutional layer, which changes the last two layers of convolution in a DeepLabv3+ model into hollow convolution, keeps the number of convolution kernels unchanged, and sets the hollow rate to 2 and the step length to 2; a feature extraction layer, which is used for extracting geometric features of the hand-drawn geometric figures based on the hollow convolutional layer, expanding the receptive field and enhancing the perception ability of line continuity and intersection relationship; the graph theory driven geometric structure reasoning mechanism comprises: a graph structure construction unit, which is used for identifying geometric elements based on the pixel-level segmentation results, extracting key attributes of the geometric elements, and constructing a graph structure representing the geometric elements and their relationships; a geometric constraint verification unit, which is used for defining a geometric constraint rule set, applying the geometric constraint rule set to the graph structure, and identifying substructures that violate the constraints; an error positioning unit, which is used for accurately positioning the identified substructures that violate the constraints, calculating error parameters, and generating error labeling data.

2. The real-time mislabeled annotation system for geometric semantic segmentation of claim 1, wherein, the image preprocessing module comprises: a scaling unit, which is used for scaling a hand-drawn geometric figure image with a length greater than 256 pixels and a width greater than 192 pixels into an image with a pixel length not greater than 256 pixels and a width not greater than 192 pixels; a cropping unit, which is used for proportionally cutting a part of the hand-drawn geometric figure image according to the length or width of the hand-drawn geometric figure image; a data enhancement unit, which is used for performing horizontal mirroring, contrast enhancement and random color jittering operations on the hand-drawn geometric figure image.

3. The real-time mislabeled annotation system for geometric semantic segmentation of claim 1, wherein, the hierarchical multi-scale channel attention mechanism comprises: The feature layer is configured to connect results of the first layer and the second layer of the DeepLabv3+ model together, and process the results by using global pooling and a maximum value function to generate a feature representation map; The scale layer is configured to process input features by using global pooling, a maximum value function and an average value function respectively to generate three features with different scales; The channel attention layer is configured to input outputs of the feature layer and the three outputs of the scale layer into a SoftMax function to generate channel weights, and multiply the channel weights with the input features to enhance key geometric features and suppress background noise.

4. The real-time mislabeled annotation system for geometric semantic segmentation of claim 1, wherein, The geometric constraint checking unit comprises: A parallel line error detection unit configured to detect an angle between two parallel lines in a geometric figure and label a position of an intersection of the two parallel lines when the angle is greater than 5°; A circle center error detection unit configured to detect a center of a circle in a geometric figure and label an outline of all circles when the center is located on an outline of another circle and a deviation is greater than 3 pixels.

5. The real-time mislabeled annotation system for geometric semantic segmentation of claim 1, wherein, The graphic semantic visualization module comprises: A basic geometric figure drawing unit configured to label a hand-drawn line segment with a black solid line, label a hand-drawn ellipse with a black thick solid line, and label a hand-drawn circle with a black thick solid line; A labeled geometric figure drawing unit configured to label an error parallel line with a red dashed line, label a center of an intersection of the error parallel line with a red solid line, and label a midpoint of two different arcs with the same radius and a midpoint of two arcs with different radii with a green solid line.

6. The real-time mislabeled annotation system for geometric semantic segmentation of claim 1, wherein, The graphic acquisition module comprises: A geometric primitive name recognition unit configured to recognize a description of a hand-drawn geometric figure image by using a text classifier to obtain text information of the description of the geometric figure image; A geometric primitive type recognition unit configured to recognize the hand-drawn geometric figure image by using an image classifier to obtain a geometric primitive type of the hand-drawn geometric figure image and a number of each geometric primitive; A geometric primitive number recognition unit configured to count a number of each geometric primitive to obtain the number of the geometric primitive.

7. The real-time mislabeled annotation system for geometric semantic segmentation of claim 1, wherein, The geometric figure image comprises a scribble image, a straight line image, a straight line + arc line image, a straight line + circle image, and an arc line + circle image; the number N of the geometric primitive is in a range of [0, 18]; and the geometric primitive comprises a straight line, an arc, a rectangle, a line segment, a triangle, a parallelogram, an equilateral triangle, an ellipse, a circle, a square, a trapezoid, a diamond, an isosceles triangle, a sector, a hexagon, and a pentagon.

8. The real-time mislabeled annotation method of geometric semantic segmentation, using the real-time mislabeled annotation system of geometric semantic segmentation according to any one of claims 1-7, characterized in that, The method comprises the following steps: A graphic acquisition step of acquiring a hand-drawn geometric figure image of a student, recognizing geometric primitives of the hand-drawn geometric figure by using an intelligent learning model, and obtaining a geometric primitive name, a geometric primitive number, and a geometric primitive type of the hand-drawn geometric figure of the student; An image preprocessing step of performing scaling, cropping, and data enhancement on the hand-drawn geometric figure image; A graphic semantic segmentation step of performing pixel-level segmentation on the preprocessed hand-drawn geometric figure image, comprising: extracting topological features of the hand-drawn geometric figure by using a topologically sensitive hollow convolutional network; enhancing key geometric features and suppressing background noise by using a hierarchical multi-scale channel attention mechanism; and constructing a graph structure of the geometric figure by using a graph theory driven geometric structure reasoning mechanism and detecting errors based on geometric constraint rules. The graphic semantic visualization step visualizes and draws each geometric primitive identified by the graphic semantic segmentation step by using different colors, styles and color gradient effects, and labels the detected error area by using a differentiated visual presentation method.

Images