Intelligent recognition method and system for Chinese writing errors based on deep neural network

Abstract

The application relates to the technical field of writing error recognition, and discloses a Chinese writing error intelligent recognition method and system based on a deep neural network, which comprises the following steps: step 1, obtaining a target Chinese character single character image subjected to binarization processing to obtain a normalized binarization character image; step 2, carrying out skeleton extraction and node marking on the normalized image to construct a character topological graph; step 3, determining a structure paradigm label and dividing components based on the topological graph and image projection to obtain outer surrounding area, inner part area, left and right components and intersection point information; step 4, calculating surrounding, left and right and intersection structure consistency measurement; step 5, combining a target Chinese character standard character shape to obtain corresponding standard measurement, setting a fixed tolerance to form a standard parameter interval and generating a structural feature vector; and step 6, outputting a writing error recognition result according to a deep neural network model. The application realizes automatic recognition and error type determination of Chinese writing structural errors.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of writing error recognition, and particularly relates to a Chinese writing error intelligent recognition method and system based on a deep neural network. BACKGROUND

[0002] With the development of informatization and digitization, Chinese writing recognition has wide application in intelligent education, electronic marking, online learning and human-computer interaction. Existing Chinese character recognition technology mainly focuses on the overall matching and classification of characters, taking optical character recognition or handwriting input recognition as the core. However, for structural errors in the actual writing process, such as surrounding relationship misplacement, left-right component proportion imbalance or crossing stroke deviation, traditional methods often fail to effectively recognize them. Such errors are not simply missing strokes or noise interference, but involve the spatial layout and geometric relationship of the internal structure of Chinese characters. For such cases, traditional methods based on template matching or statistical learning often fail to make accurate judgments, and the interpretability of the recognition results is also limited.

[0003] In recent years, deep neural networks have made significant progress in image recognition and pattern classification, and can improve the processing capacity of complex tasks through end-to-end feature learning. However, existing deep learning-based Chinese character recognition mainly focuses on character classification, and lacks research on fine-grained judgment of writing structural errors. At the same time, relying solely on image convolution features makes it difficult to fully utilize the topological relationship and geometric measurement features of Chinese strokes, and there are still certain limitations in distinguishing standard character shapes and structural error character shapes. SUMMARY

[0004] The application provides a Chinese writing error intelligent recognition method and system based on a deep neural network, which solves the technical problems of being difficult to effectively recognize internal structural errors of Chinese characters, insufficient recognition accuracy and interpretability in related technologies.

[0005] The application provides a Chinese writing error intelligent recognition method based on a deep neural network, which includes the following steps:

[0006] Step 1, obtaining a Chinese single character image of a target Chinese character after binaryzation processing, determining an outer frame, a center, a height and a width, and performing coordinate normalization to obtain a normalized binary character shape image;

[0007] Step 2, performing skeleton extraction and node labeling on the normalized binary character shape image to construct a character topology graph describing the stroke connection relationship;

[0008] Step 3, determining the structure paradigm label based on the glyph topology graph and image projection and completing component division to obtain the outer surrounding area and inner part area, left component and right component, and intersection point information; wherein the structure paradigm label comprises: surrounding structure paradigm label, left and right structure paradigm label, and intersection structure paradigm label;

[0009] Step 4, calculating the surrounding structure consistency measure, left and right structure consistency measure, and intersection structure consistency measure according to the structure paradigm label to obtain a structure measure set;

[0010] Step 5, obtaining the standard measure set corresponding to step 4 based on the standard glyph of the target Chinese character, setting a fixed tolerance to form a standard parameter interval, aligning and encoding the structure measure set and the standard parameter interval item by item to generate a structure feature vector;

[0011] Step 6, inputting the normalized binary glyph image, glyph topology graph, structure paradigm label, and structure feature vector as input into the deep neural network model trained to output the error recognition result of the Chinese character image by the deep neural network model, wherein the error recognition result comprises: whether there is a structural writing error and the corresponding error type.

[0012] Further, a binary processed Chinese character image is obtained, the bounding box, center, height, and width are determined, and coordinate normalization is performed to obtain a normalized binary glyph image, comprising:

[0013] Step 11, extracting a foreground pixel point set, determining a bounding box containing the foreground pixel point set, and calculating the left boundary, right boundary, upper boundary, and lower boundary of the bounding box;

[0014] Step 12, calculating the bounding box width as the difference between the right boundary and the left boundary, the height as the difference between the lower boundary and the upper boundary, and the geometric center coordinates as the midpoint of the left boundary and the right boundary and the midpoint of the upper boundary and the lower boundary based on the bounding box;

[0015] Step 13, performing translation and scale normalization processing on the foreground pixel point coordinates according to the geometric center coordinates and the bounding box height, and detecting the offset angle of the main direction of the foreground pixel points in the bounding box by the least square method, and when the offset angle is greater than a preset offset threshold, performing rotation correction on all foreground pixel points to generate a normalized binary glyph image.

[0016] Further, skeleton extraction and node labeling are performed on the normalized binary glyph image to construct a glyph topology graph describing the stroke connection relationship, comprising:

[0017] Step 21, performing an iterative thinning operation on the normalized binary glyph image, keeping the connectivity and the number of end points of the original foreground pixels unchanged during the thinning process, and continuing the iteration until the strokes are all converged to single-pixel width, obtaining a skeleton image;

[0018] Step 22, in the skeleton image, calculating the neighborhood degree of each skeleton pixel with an eight-neighborhood as the judgment range, marking the end points as the neighborhood degree of 1, marking the transition points as the neighborhood degree of 2, and marking the intersection points as the neighborhood degree of 3 or more, forming a set of junctions; for the skeleton pixels located at the boundary, calculating the degree based on the actual number of neighborhood pixels;

[0019] Step 23, constructing a glyph topology graph with the end points and intersection points as vertices and the shortest connected path composed of transition points therebetween as edges.

[0020] Further, the determination of the outer enclosing region and the inner part region comprises:

[0021] Step 31, enumerating closed paths without self-intersection in the glyph topology graph and forming a set of closed loops;

[0022] Step 32, judging whether the inside of each closed loop contains foreground pixels by using the ray method, obtaining a set of candidate closed loops meeting the conditions;

[0023] Step 33, when there are multiple candidate closed loops, selecting the closed loop with the largest geometric area as the outer enclosing region, and defining the union of the connected foreground pixels inside the closed loop as the inner part region; when there are nested closed loops, only taking the outermost layer as the outer enclosing region; after completing the division of the outer enclosing region and the inner part region, generating the enclosing structure paradigm label.

[0024] Further, the component division comprises:

[0025] Step 41, counting the number of foreground pixels in each column of the normalized binary glyph image in the horizontal direction, obtaining a vertical projection sequence;

[0026] Step 42, detecting the index interval of adjacent columns with a projection value of zero, and determining that the glyph has a left-right structure when the width of the interval accounts for more than a first preset proportion of the entire bounding box width and the height in the vertical direction exceeds a second preset proportion;

[0027] Step 43, selecting the column with the minimum difference in the number of foreground pixels on the left and right sides in the index interval of adjacent columns as the division line, defining the set of foreground pixels on the left side of the division line as the left component, and defining the set of foreground pixels on the right side as the right component; after completing the division of the left component and the right component, generating the left-right structure paradigm label.

[0028] Further, the acquisition of the intersection point information comprises:

[0029] Step 51, calling the intersection set in the labeled node set, extracting the adjacent edge direction vector of each intersection, the direction vector being the displacement vector from the intersection to the adjacent skeleton pixel;

[0030] Step 52, calculating the included angle of any two direction vectors, and judging according to the preset angle deviation threshold and the preset vertical tolerance threshold;

[0031] Step 53, when there are two pairs of approximately opposite and perpendicular directions, the intersection is determined to be X type; when there is a pair of approximately opposite and another perpendicular direction, it is determined to be T type; when there are two approximately perpendicular directions without opposite pairs, it is determined to be L type;

[0032] Step 54, recording the intersection and its corresponding type as intersection information, and generating the intersection structure paradigm label.

[0033] Further, according to the structure paradigm label, the surrounding structure consistency measure, the left-right structure consistency measure and the intersection structure consistency measure are calculated respectively to obtain a structure measure set, including:

[0034] Step 61, normalizing the geometric quantities related to the length according to the height of the bounding box, normalizing the geometric quantities related to the area according to the area of the bounding box, and taking the left upper corner of the bounding box as the origin of the position coordinates, respectively dividing by the height and width of the bounding box to form a unified measure reference;

[0035] Step 62, when the structure paradigm label contains the surrounding structure paradigm label, calculating the surrounding structure consistency measure, including: taking the ratio of the area of the inner part to the area of the outer surrounding as the area ratio; taking the ratio of the minimum distance difference between the inner part and the upper and lower boundaries of the outer surrounding to the sum as the upper and lower white area asymmetry; taking the ratio of the minimum distance difference between the inner part and the left and right boundaries of the outer surrounding to the sum as the left and right white area asymmetry; taking the relative offset between the centroids of the inner part and the outer surrounding as the alignment deviation;

[0036] Step 63, when the structure paradigm label contains the left-right structure, calculating the left-right structure consistency measure, including: taking the proportion of the difference between the centroids of the left and right parts to the width of the bounding box as the centroid horizontal difference; taking the proportion of the overlapping length of the vertical projection intervals of the inner and outer parts to the longer interval length as the vertical height overlap ratio; for the width sequence obtained by scanning the white area channel row by row, taking the minimum value as the proportion of the minimum width of the channel to the width of the bounding box, taking the variance of the width sequence as the channel width variance, and taking the proportion of the number of times the width sequence decreases between adjacent rows to the total number of rows as the channel narrowing frequency;

[0037] Step 64, when the structural paradigm label contains a cross structure, calculating a cross structure consistency measure for each intersection point in the intersection point information, including: taking the included angle between the two main stroke direction vectors adjacent to the intersection point as the main stroke angle; taking the normalized coordinates of the intersection point coordinates relative to the upper left corner of the bounding box as the relative position;

[0038] Step 65, assembling the various measurement indicators obtained in steps 62 to 64 into a structural measurement set in a fixed order, and establishing an index mapping.

[0039] Further, the generation process of the structural feature vector includes:

[0040] Step 71, calculating a standard measurement set corresponding to the structural measurement set in step 4 based on the standard form of the target Chinese character;

[0041] Step 72, for each item of the standard measurement set, setting a fixed tolerance to form a standard parameter space, the fixed tolerance being a preset percentage of the standard measurement value;

[0042] Step 73, aligning the structural measurement set with the standard parameter interval item by item, calculating the signed deviation of each item from the center of the standard parameter interval, the interval out-of-bound proportion, and the out-of-bound indication, wherein the signed deviation is the ratio of the actual measurement minus the center of the interval to the interval half-width; the interval out-of-bound proportion is the proportion of the actual measurement exceeding the standard parameter interval part to the width of the standard parameter interval; the out-of-bound indication is a Boolean variable indicating whether the actual measurement is outside the standard parameter interval;

[0043] Step 74, assembling the alignment results obtained in step 73 into a structural feature vector according to the fixed order of the index mapping.

[0044] Further, the deep neural network model includes: a multi-branch input module, a gated fusion module, an attention aggregation module, and a multi-label classification output module; the multi-branch input module includes a convolutional neural network branch, a graph convolutional network branch, and a fully connected network branch;

[0045] The error recognition result of the Chinese character image output by the deep neural network model includes:

[0046] Step 81, inputting the normalized binary glyph image into the convolutional neural network branch to extract the regionalized appearance features of the outer bounding region, the inner part region, and the left and right components; inputting the glyph topology graph into the graph convolutional network branch to extract the structural features of the stroke connection relationship and the intersection complexity; inputting the structural feature vector into the fully connected network branch to extract the out-of-bound proportion sequence of each consistency measure and the standard parameter interval;

[0047] Step 82, the gating fusion module generates a gating vector based on the out-of-bound proportion sequence, and acts on each branch feature representation through element-by-element multiplication to weight the features;

[0048] Step 83, the attention aggregation module calculates attention weights in combination with the structure paradigm label and the out-of-bound proportion sequence, wherein the structure paradigm label is input as a condition of attention after one-hot encoding, and is used for weighting and converging between regional features and structural features;

[0049] Step 84, the fused features are input into a multi-label classification output layer to output an error recognition result, wherein the error recognition result includes: whether there is a structural writing error, and an error type; wherein the error type includes: surrounding structure error, left-right structure error, and cross structure error.

[0050] The application provides a Chinese writing error intelligent recognition system based on a deep neural network, which comprises:

[0051] An image normalization module acquires a Chinese single character image of a target Chinese character after binaryzation processing, determines a bounding box, a center, a height and a width, and performs coordinate normalization to obtain a normalized binary glyph image;

[0052] A skeleton topology construction module performs skeleton extraction and node labeling on the normalized binary glyph image to construct a glyph topology graph describing stroke connection relationships;

[0053] A paradigm division module determines a structure paradigm label and completes component division based on the glyph topology graph and image projection to obtain an outer surrounding area and an inner component area, a left component and a right component, and intersection point information; wherein the structure paradigm label includes: a surrounding structure paradigm label, a left-right structure paradigm label and a cross structure paradigm label;

[0054] A consistency measurement module calculates a surrounding structure consistency measurement, a left-right structure consistency measurement and a cross structure consistency measurement according to the structure paradigm label to obtain a structural measurement set;

[0055] A feature vector generation module acquires a standard measurement set corresponding to the consistency measurement module based on a standard glyph of a target Chinese character, sets a fixed tolerance to form a standard parameter interval, aligns and encodes the structural measurement set and the standard parameter interval item by item to generate a structural feature vector;

[0056] A deep recognition module inputs the normalized binary glyph image, the glyph topology graph, the structure paradigm label and the structural feature vector as inputs into a deep neural network model trained to output an error recognition result of the Chinese single character image, wherein the error recognition result includes: whether there is a structural writing error and a corresponding error type.

[0057] The application has the advantages that: the application can realize unified processing in position, scale and inclination by normalizing, skeleton extraction and topology construction of the character image, and ensure the stability of input data under different writing conditions; by introducing structure paradigm labels and component division, the surrounding structure, left-right structure and cross structure are explicitly modeled, and the structural feature vector with physical meaning and interpretability is generated by aligning each item with the standard character through consistency measurement, thereby overcoming the defect that the traditional method is difficult to identify fine-grained structure errors due to the dependence on overall similarity determination; by jointly inputting the image features, topology features and structural feature vectors into the deep neural network model, and assisting with the gating fusion and attention aggregation mechanism, the local geometric details and complex relationships across components can be effectively captured, and the robustness and discrimination ability to diversified writing errors are improved. Overall, the application can not only output whether there is a writing error, but also explicitly identify the error type, and realize high accuracy and interpretability. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 is a flowchart of the Chinese writing error intelligent recognition method based on deep neural network of the application. DETAILED DESCRIPTION

[0059] The subject matter described herein will now be discussed with reference to example implementations. It should be understood that the discussion of these implementations is merely meant to provide a better understanding of the subject matter described herein and can include changes, modifications, additions or omissions of the functions and arrangements of the elements discussed without departing from the scope of the content of this specification. Various processes or components can be omitted, replaced or added according to needs of various examples. In addition, features described with respect to some examples can also be combined in other examples.

[0060] As shown in Figure 1 the Chinese writing error intelligent recognition method based on deep neural network includes the following steps:

[0061] Step 1, obtaining a Chinese single character image of a target Chinese character after binaryzation processing, determining the bounding box, center, height and width, and performing coordinate normalization to obtain a normalized binary character image;

[0062] Step 2, skeleton extraction and node labeling are performed on the normalized binary character image to construct a character topology graph describing stroke connection relationship;

[0063] Step 3, based on the character topology graph and image projection, the structure paradigm label is determined and the component division is completed to obtain the outer surrounding area and inner part area, left component and right component, and intersection point information; wherein the structure paradigm label includes: surrounding structure paradigm label, left-right structure paradigm label and cross structure paradigm label;

[0064] Step 4, respectively calculating the surrounding structure consistency measure, the left-right structure consistency measure and the cross structure consistency measure according to the structure paradigm label, obtaining a structure measure set;

[0065] Step 5, obtaining the standard measure set corresponding to step 4 based on the standard form of the target Chinese character, and setting a fixed tolerance to form a standard parameter interval, aligning and encoding the structure measure set and the standard parameter interval item by item to generate a structure feature vector;

[0066] Step 6, inputting the normalized binary character image, the character topological graph, the structure paradigm label and the structure feature vector as inputs into the deep neural network model trained, and outputting the error recognition result of the Chinese character image by the deep neural network model, wherein the error recognition result includes: whether there is a structural writing error and the corresponding error type.

[0067] In an embodiment of the present application, the image of the target Chinese character is obtained by scanning, electronic handwriting board input, camera shooting and the like, and is binarized to obtain a Chinese character image containing foreground and background, wherein the foreground pixels are used to represent the stroke area, and the background pixels are used to represent the blank area; the Chinese character image can highlight the geometric structure of the strokes and simplify the data expression.

[0068] In an embodiment of the present application, the Chinese character image of the target Chinese character after binarization processing is obtained, the bounding box, the center, the height and the width are determined, and the coordinate normalization is performed to obtain a normalized binary character image, including:

[0069] Step 11, separating the foreground and the background of the input Chinese character image, taking the pixel points corresponding to the strokes as a foreground pixel point set, and determining a bounding box capable of completely containing the foreground pixel point set. The bounding box is defined by four boundary lines of left boundary, right boundary, upper boundary and lower boundary, thereby limiting the smallest rectangular area containing the strokes;

[0070] Step 12, calculating the bounding box width as the difference between the right boundary and the left boundary, the height as the difference between the lower boundary and the upper boundary, and the geometric center coordinates as the midpoint of the left boundary and the right boundary and the midpoint of the upper boundary and the lower boundary based on the bounding box;

[0071] Step 13, according to the geometric center coordinates and the height of the circumscribed frame, the translation and scale normalization processing is performed on the foreground pixel point coordinates, the overall position of the stroke is adjusted to a unified coordinate system, and the strokes of different sizes are kept consistent in scale; and the offset angle is detected by least square fitting of the main direction of the foreground pixel points in the circumscribed frame, when the offset angle is greater than a preset offset threshold, it is determined that the character has obvious inclination, and the rotation correction is performed on all foreground pixel points to generate a normalized binary character image; the step makes the character image keep standard alignment in the horizontal direction and the vertical direction.

[0072] Through the above steps, the stroke structure of Chinese single characters can be effectively standardized in position, scale and rotation direction, so as to eliminate the geometric differences brought by different writers and different writing conditions, and ensure that the input data of subsequent skeleton extraction, structure analysis and deep neural network recognition is under a unified standard.

[0073] In an embodiment of the present application, skeleton extraction and node labeling are performed on the normalized binary character image, and a character topology graph describing the connection relationship of strokes is constructed, including:

[0074] Step 21, an iterative thinning operation is performed on the normalized binary character image, and during the thinning process, the foreground pixels of the stroke edge are gradually removed in each iteration step, and the connectivity and the number of end points of the original stroke are kept from being damaged, so as to avoid the breakage of the stroke structure or the loss of the end points; the thinning process continues until all strokes in the image converge to a single-pixel width, and a skeleton image that preserves the overall morphology of the stroke is obtained;

[0075] Step 22, in the skeleton image, the neighborhood degree of each skeleton pixel is calculated within an eight-neighborhood judgment range, the end points are labeled as the neighborhood degree of 1, the transition points are labeled as the neighborhood degree of 2, and the intersection points are labeled as the neighborhood degree of 3 or more, and a node set is formed; for the skeleton pixels located at the boundary, the actual neighborhood pixel number is taken as the basis for degree calculation;

[0076] Step 23, the character topology graph is constructed by taking the end points and the intersection points as vertices and the shortest connected path composed of transition points therebetween as edges; the character topology graph can accurately represent the connection relationship and interaction morphology of Chinese character strokes.

[0077] Through the above steps, the present application can convert the stroke image from the pixel level to the topology graph structure, which not only preserves the global connectivity of the written stroke, but also simplifies the data expression, and further improves the accuracy and interpretability of the overall error recognition method.

[0078] In an embodiment of the present application, the determination of the outer enclosing region and the inner part region includes:

[0079] Step 31, enumerate the closed path without intersection in the glyph topology graph and form a closed loop set; the closed path represents that the path starting point and the ending point coincide, and no intersection is generated in the path interior, so that the geometric shape of the closed loop is complete;

[0080] Step 32, adopt the ray method to judge whether the interior of each closed loop contains foreground pixels, and obtain a candidate closed loop set meeting the condition; specifically, a ray is emitted from an arbitrary point in the closed loop interior, and the intersection point number of the ray and the closed loop boundary is counted; when the intersection point number meets the odd number judgment principle, it is indicated that the foreground pixel region exists in the closed loop interior;

[0081] Step 33, when there are multiple candidate closed loops, the closed loop with the largest geometric area is selected as the outer enclosing region, so that the outer enclosing region can cover the stroke structure to the maximum extent; and the connected foreground pixel union in the closed loop interior is defined as the inner part region; when there are nested closed loops, only the outermost layer is taken as the outer enclosing region, so as to avoid repeated division; after the outer enclosing region and the inner part region are divided, the surrounding structure paradigm label is generated.

[0082] Through the above steps, the application can accurately identify and separate the surrounding relationship in the complex stroke structure, ensure that the inner and outer region division has geometric rationality and topological consistency, so as to improve the extraction precision of the surrounding structure related features, and provide stable structure reference for the error type judgment.

[0083] In an embodiment of the application, the component division includes:

[0084] Step 41, the foreground pixel number of the normalized binary glyph image is counted column by column in the horizontal direction, and a vertical projection sequence is obtained; the vertical projection sequence can reflect the distribution of the stroke in the horizontal direction; when the projection value of a column is zero, it indicates that there is no foreground pixel in the column, which belongs to the background column;

[0085] Step 42, detect the adjacent column index interval with zero projection value; when the width of the interval accounts for more than a first preset proportion of the entire bounding box width and the height in the vertical direction exceeds a second preset proportion, it is determined that the glyph has a left-right structure; the above conditions ensure that the detected zero value interval is not only wide enough, but also continuous in the vertical direction, which can effectively serve as the separation area of the left and right components;

[0086] Step 43, select the column with the minimum difference value of the foreground pixel number on the left and right sides in the adjacent column index interval as the division line; the set of foreground pixels on the left side of the division line is defined as the left component, and the set of foreground pixels on the right side is defined as the right component; after the left component and the right component are divided, the left-right structure paradigm label is generated.

[0087] In an embodiment of the application, the acquisition of the intersection point information includes:

[0088] Step 51, calling a set of intersection points in the labeled node set, extracting the adjacent edge direction vector of each intersection point, the direction vector being the displacement vector of the intersection point to the adjacent skeleton pixel; the vector can represent the directional characteristics of the stroke extension in the neighborhood of the intersection point;

[0089] Step 52, calculating the included angle of any two direction vectors, and judging according to the preset angle deviation threshold and the preset vertical tolerance threshold, whether the different direction vectors belong to approximately opposite relationship or approximately vertical relationship;

[0090] Step 53, when there are two pairs of approximately opposite and perpendicular directions, the intersection point is determined to be X type; when there is a pair of approximately opposite and another perpendicular direction, it is determined to be T type; when there are two approximately perpendicular directions without opposite pairs, it is determined to be L type;

[0091] Step 54, recording the intersection point and its corresponding type as intersection point information, and generating an intersection structure paradigm label, which can accurately describe the intersection structure characteristics in the character shape.

[0092] Through the above steps, the present application can classify the intersection points on the basis of the topological graph, ensure that different types of intersection structures are distinguished in the recognition process, thereby improving the recognition accuracy of intersection type writing errors, and enhancing the explainability of the overall method.

[0093] In an embodiment of the present application, the structure consistency measure, the left-right structure consistency measure and the intersection structure consistency measure are calculated according to the structure paradigm label respectively, and a structure measure set is obtained, including:

[0094] Step 61, normalizing the geometric quantities related to length according to the height of the circumscribed frame, normalizing the geometric quantities related to area according to the area of the circumscribed frame, and taking the upper left corner of the circumscribed frame as the origin of the position coordinates, respectively dividing by the height and width of the circumscribed frame to form a unified dimension reference; wherein the length-related geometric quantities include the width and height of the circumscribed frame, the minimum distance from the inner part to the outer enclosing boundary, the horizontal distance between the left and right parts, the white area channel width, and the distance from the intersection point to the boundary, etc.; the area-related geometric quantities include the area of the outer enclosing region, the area of the inner part region, and the area of the left and right part regions; this step can eliminate the influence of the size and position difference of the written font, so that different character shapes have comparability under the same dimension;

[0095] Step 62, when the structure paradigm label contains a surrounding structure, calculate the surrounding structure consistency measure, including: taking the ratio of the inner part area to the outer surrounding area as the area ratio to measure the scale relationship of the inner part in the outer surrounding; taking the ratio of the minimum distance difference of the inner part to the upper and lower boundaries of the outer surrounding to the sum as the upper and lower white area asymmetry to reflect the offset of the inner part in the vertical direction; taking the ratio of the minimum distance difference of the inner part to the left and right boundaries of the outer surrounding to the sum as the left and right white area asymmetry to reflect the horizontal balance; taking the relative offset between the center of mass of the inner part and the center of mass of the outer surrounding as the alignment deviation to measure the overall alignment of the inner and outer parts;

[0096] Step 63, when the structure paradigm label contains a left-right structure, calculate the left-right structure consistency measure, including: taking the proportion of the center of mass difference of the left and right components to the width of the bounding box as the center of mass horizontal difference to measure the symmetry of the left and right components; taking the proportion of the overlapping length of the vertical projection intervals of the inner and outer components to the longer interval length as the vertical height overlap ratio to reflect the consistency of the left and right components in the vertical direction; taking the minimum width of the white area channel obtained by scanning each row as the minimum channel width to determine whether there is a narrow channel, taking the variance of the width sequence as the channel width variance to measure the volatility of the channel width, and taking the proportion of the number of times the width sequence decreases between adjacent rows to the total number of rows as the channel narrowing frequency to reflect whether the channel continuity is abnormally contracted;

[0097] Step 64, when the structure paradigm label contains a crossing structure, calculate the crossing structure consistency measure for each intersection point in the intersection point information, including: taking the included angle between the two adjacent main stroke direction vectors of the intersection point as the main stroke included angle to measure the angle characteristics of the crossing stroke; taking the normalized coordinates of the intersection point coordinates relative to the top left corner of the bounding box as the relative position to represent the relative position of the intersection point in the overall glyph;

[0098] Step 65, assemble the various measurement indicators obtained in steps 62 to 64 into a structure measurement set in a fixed order, and establish an index mapping; the various measurement indicators include: area ratio, upper and lower white area asymmetry, left and right white area asymmetry, alignment deviation, center of mass horizontal difference, vertical height overlap ratio, minimum channel width, channel width variance, channel narrowing frequency, main stroke included angle, and relative position; the index mapping is used to clearly correspond each measurement indicator to the structure paradigm and physical meaning to which it belongs, ensuring that each measurement indicator can be called and traced back in the subsequent comparison and feature coding process.

[0099] Through the above steps, the present application can convert complex glyph structures into normalized numerical measures, both preserving the spatial geometric characteristics of the stroke structure and the component relationship, and achieving comparability between different glyphs, thereby providing a complete, stable and interpretable feature basis for subsequent error recognition.

[0100] In an embodiment of the present application, the generation process of the structural feature vector comprises:

[0101] In step 71, based on the standard form of the target Chinese character, the standard metric set corresponding to the structural metric set in step 4 is calculated in the same processing manner as the previous steps; wherein the standard form comes from a pre-set standard character library and is one-to-one corresponding to the target Chinese character;

[0102] In step 72, for each item of the standard metric set, a fixed tolerance is set to form a standard parameter space, and the fixed tolerance is a pre-set percentage of the standard metric value, which is used to reflect the reasonable deviation range allowed under normal writing conditions. By introducing the fixed tolerance, it can avoid misjudgment caused by individual writing differences or slight noise, thereby enhancing the overall adaptability;

[0103] In step 73, the structural metric set and the standard parameter interval are aligned item by item, and the signed deviation, the interval out-of-bound proportion and the out-of-bound indication of each item relative to the center of the standard parameter interval are calculated, wherein the signed deviation is the ratio of the actual metric minus the center of the interval to the half width of the interval, which is used to reflect the direction and amplitude of the actual value deviating from the standard center; the interval out-of-bound proportion is the proportion of the actual metric exceeding the standard parameter interval to the width of the standard parameter interval, which is used to quantify the severity of the out-of-bound; the out-of-bound indication is a Boolean variable marking whether the actual metric is outside the standard parameter interval; through the combination of the above three types of indicators, the deviation between the actual metric and the standard metric can be comprehensively described from different dimensions;

[0104] In step 74, the alignment results obtained in step 73 are assembled into a structural feature vector in a fixed order according to the index mapping.

[0105] In an embodiment of the present application, the deep neural network model comprises a multi-branch input module, a gated fusion module, an attention aggregation module, and a multi-label classification output module;

[0106] The error recognition result of the Chinese single character image output by the deep neural network model comprises:

[0107] Step 81, the multi-branch input module includes a convolutional neural network branch, a graph convolutional network branch and a fully connected network branch; the normalized binary glyph image is input into the convolutional neural network branch to extract the regionalized appearance features of the outer enclosing region, the inner part region and the left and right component regions; the glyph topology graph is input into the graph convolutional network branch to extract the structural features of stroke connection relationship and cross complexity; the structural feature vector is input into the fully connected network branch to extract the out-of-bound proportion sequence of each consistency measure and specification parameter interval; through the multi-branch input module, different source feature representations can be obtained in parallel, and it is ensured that the three types of information of glyph appearance, topology structure and specification measure are all encoded;

[0108] Step 82, the gating fusion module generates a gating vector based on the out-of-bound proportion sequence, and applies element-wise multiplication to each branch feature representation to weight the features; when the deviation of a certain structural paradigm is large, the weight of the corresponding feature branch is enhanced, so as to highlight the feature contribution related to the abnormality. The gating mechanism can introduce the structural consistency measure result as an explicit control signal, so that the deep neural network model has higher sensitivity to potential abnormal areas in the fusion process;

[0109] Step 83, the attention aggregation module calculates attention weights by combining the structural paradigm label and the out-of-bound proportion sequence; the structural paradigm label is one-hot encoded and used as the conditional input of attention, for weighted convergence between regional features and structural features; through this process, the model can apply higher weights to key regions according to the paradigm label, so as to realize differentiated focusing and dynamic fusion of features;

[0110] Step 84, the fused features are input into a multi-label classification output layer to output an error recognition result, the error recognition result including: whether there is a structural writing error, and an error type; wherein the error type includes: surrounding structure error, left and right structure error, and cross structure error.

[0111] Through the above implementation process, the image appearance features, topology structure features and specification deviation features are uniformly input into the deep neural network, and the dynamic adjustment of feature contribution is realized through the gating fusion and attention aggregation, so as to ensure the model expression while enhancing the interpretability and robustness of the recognition result, and improve the accuracy and practicability of Chinese writing error recognition.

[0112] It should be noted that the structural feature vector generated in step 5 of the present application is based on the alignment result of the target Chinese character and the standard character form item by item measurement, and its main role is to express the structural information such as geometric proportion, symmetry and intersection position in a quantitative way, thereby providing a feature set with clear physical meaning. The feature set has strong interpretability and can directly reflect the specific structural deviation situation. However, relying only on these manually defined features, it is difficult to fully cover the complex detail information in the character form, such as stroke thickness variation, local distortion pattern and nonlinear interaction between different structural domains, so it is not enough to ensure high robustness in diversified writing scenarios. Therefore, the present application introduces a deep neural network model in step 6, taking the normalized binary character form image, the character form topology graph, the structural paradigm label and the structural feature vector obtained in step 5 as joint input. On the one hand, the deep neural network can automatically capture the stroke texture, local geometric pattern and high-dimensional correlation features across components through end-to-end learning, making up for the details that cannot be covered by manual measurement; on the other hand, the structural feature vector is used as an explicit input for training and reasoning, providing interpretable prior constraints for the network, thereby guiding the model to pay more attention to the key areas of structural errors during identification.

[0113] The present embodiment also provides a Chinese writing error intelligent recognition system based on a deep neural network, comprising:

[0114] An image normalization module acquires a Chinese single character image of a target Chinese character after binaryzation processing, determines a bounding box, a center, a height and a width, and performs coordinate normalization to obtain a normalized binary character form image;

[0115] A skeleton topology construction module performs skeleton extraction and node labeling on the normalized binary character form image to construct a character form topology graph describing stroke connection relationship;

[0116] A paradigm division module determines a structural paradigm label and completes component division based on the character form topology graph and image projection to obtain outer enclosing area and inner part area, left component and right component and intersection point information; wherein the structural paradigm label comprises: enclosing structural paradigm label, left-right structural paradigm label and intersection structural paradigm label;

[0117] A consistency measurement module calculates enclosing structure consistency measurement, left-right structure consistency measurement and intersection structure consistency measurement respectively according to the structural paradigm label to obtain a structural measurement set;

[0118] A feature vector generation module acquires a standard measurement set corresponding to the consistency measurement module based on the standard character form of the target Chinese character, sets a fixed tolerance to form a standard parameter interval, aligns and encodes the structural measurement set and the standard parameter interval item by item to generate a structural feature vector;

[0119] The deep recognition module takes the normalized binary glyph image, the glyph topology graph, the structural paradigm label and the structural feature vector as input, inputs them into the trained deep neural network model, and outputs the error recognition result of the Chinese character image through the deep neural network model, wherein the error recognition result includes whether there is a structural writing error and a corresponding error type.

[0120] It should be noted that the setting of the interval and the threshold size is for the convenience of comparison, wherein the size of the threshold depends on the amount of sample data and the base number set by the person skilled in the art for each group of sample data, as long as it does not affect the proportional relationship between the parameters and the quantized values. And the above formula is a calculation of the value without dimension. The formula is obtained by software simulation of a large amount of data to obtain a formula of the nearest true situation. The preset parameters in the formula are set by the person skilled in the art according to the actual situation.

[0121] The embodiments of the present application are described above, but the present application is not limited to the above specific embodiments. The above specific embodiments are only illustrative and not limiting. Those skilled in the art can make many forms under the inspiration of the present embodiment, which are all within the protection of the present embodiment.

Claims

1. A method for intelligent recognition of Chinese handwriting errors based on deep neural networks, characterized in that, Includes the following steps: Step 1: Obtain the Chinese character image of the target Chinese character after binarization, determine the bounding box, center, height and width, and perform coordinate normalization to obtain the normalized binary character image; Step 2: Extract the skeleton and label the nodes of the normalized binary glyph image to construct a glyph topology map describing the stroke connection relationship; Step 3: Determine the structural paradigm labels based on the glyph topology map and image projection, and complete the component division to obtain information on the outer enclosing region and inner component region, left component and right component, and intersection point; among which, the structural paradigm labels include: enclosing structural paradigm labels, left and right structural paradigm labels, and intersection structural paradigm labels. Step 4: Calculate the enclosing structure consistency metric, the left-right structure consistency metric, and the intersection structure consistency metric based on the structural paradigm labels to obtain a set of structural metrics, specifically including: Step 61: Normalize the geometric quantities involving length according to the height of the bounding box, normalize the geometric quantities involving area according to the area of ​​the bounding box, and normalize the position coordinates by dividing the bounding box height and width respectively with the upper left corner of the bounding box as the origin, to form a unified measurement benchmark. Step 62: When the structural paradigm label includes the enclosing structural paradigm label, calculate the enclosing structural consistency metric, including: using the ratio of the area of ​​the inner region to the area of ​​the outer enclosing region as the area ratio; using the ratio of the minimum spacing difference between the inner and outer enclosing upper and lower boundaries to the sum of the two as the upper and lower white area asymmetry; using the ratio of the minimum spacing difference between the inner and outer enclosing left and right boundaries to the sum of the two as the left and right white area asymmetry; and using the relative offset between the centroid of the inner and the centroid of the outer enclosing region as the alignment deviation. Step 63: When the structural paradigm label includes left and right structures, calculate the consistency metric of the left and right structures, including: using the ratio of the difference in the center of gravity of the left and right components to the width of the bounding box as the horizontal difference in the center of gravity; using the ratio of the overlap length of the projection interval of the inner and outer components in the vertical direction to the length of the longer interval as the vertical height overlap ratio; taking the minimum value of the width sequence obtained by the white area channel in line-by-line scanning as the minimum width of the channel, taking the variance of the width sequence as the channel width variance, and using the ratio of the number of times the width sequence decreases between adjacent lines to the total number of lines as the channel narrowing frequency. Step 64: When the structural paradigm label contains a cross structure, calculate the cross structure consistency metric for each cross point in the cross point information, including: using the angle between the direction vectors of the two main strokes adjacent to the cross point as the angle of the main strokes; and using the normalized coordinates of the cross point relative to the upper left corner of the bounding box as the relative position. Step 65: Assemble the various metrics obtained in steps 62 to 64 into a structured metric set in a fixed order, and establish an index mapping; Step 5: Based on the standardized glyphs of the target Chinese characters, obtain the standardized measurement set corresponding to Step 4, and set a fixed tolerance to form a standardized parameter range. Align the structural measurement set with the standardized parameter range item by item and encode them to generate a structural feature vector. Step 6: Input the normalized binary glyph image, glyph topology graph, structural paradigm label and structural feature vector into the trained deep neural network model, and output the error recognition result of the Chinese single character image through the deep neural network model. The error recognition result includes: whether there is a structural writing error and the corresponding error type.

2. The intelligent Chinese handwriting error recognition method based on deep neural networks according to claim 1, characterized in that, Obtain the binarized Chinese character image, determine the bounding box, center, height, and width, and perform coordinate normalization to obtain a normalized binary character image, including: Step 11: Extract the set of foreground pixels, determine the bounding box containing the set of foreground pixels, and calculate the left, right, top, and bottom boundaries of the bounding box; Step 12: Based on the bounding box, calculate the width of the bounding box as the difference between the right and left boundaries, the height as the difference between the bottom and top boundaries, and the geometric center coordinates as the midpoint between the left and right boundaries and the midpoint between the top and bottom boundaries. Step 13: Based on the geometric center coordinates and the bounding box height, perform translation and scale normalization on the coordinates of the foreground pixels, and use the least squares method to fit the main direction of the foreground pixels in the bounding box to detect the offset angle. When the offset angle is greater than the preset offset threshold, perform rotation correction on all foreground pixels to generate a normalized binary glyph image.

3. The intelligent Chinese handwriting error recognition method based on deep neural networks according to claim 2, characterized in that, The normalized binary glyph image is subjected to skeleton extraction and node annotation to construct a glyph topology graph describing the stroke connection relationships, including: Step 21: Perform an iterative thinning operation on the normalized binary glyph image. During the thinning process, maintain the connectivity of the original foreground pixels and the number of endpoints. Continue iterating until all strokes converge to a single pixel width to obtain the skeleton image. Step 22: In the skeleton image, using eight neighborhoods as the determination range, calculate the neighborhood degree of each skeleton pixel. The neighboring degree of 1 is marked as the endpoint, the neighboring degree of 2 is marked as the transition point, and the neighboring degree of 3 or more is marked as the intersection point, forming a set of nodes; for skeleton pixels located on the boundary, the degree is calculated based on the actual number of neighboring pixels. Step 23: Construct a glyph topology graph with endpoints and intersections as vertices and the shortest connected path between them via transition points as edges.

4. The intelligent Chinese handwriting error recognition method based on deep neural networks according to claim 1, characterized in that, The determination of the outer and inner regions includes: Step 31: Enumerate closed paths without self-intersections in the glyph topology graph and form a closed loop set; Step 32: For each closed loop, use the ray tracing method to determine whether it contains foreground pixels, and obtain a set of candidate closed loops that meet the conditions. Step 33: When there are multiple candidate closed loops, select the closed loop with the largest geometric area as the outer bounding region; and define the union of the connected foreground pixels inside the closed loop as the inner region; when there are nested closed loops, only the outermost layer is taken as the outer bounding region; after completing the division of the outer bounding region and the inner region, generate the bounding structure paradigm label.

5. The intelligent Chinese handwriting error recognition method based on deep neural networks according to claim 1, characterized in that, The component division includes: Step 41: Count the number of foreground pixels column by column in the horizontal direction of the normalized binary glyph image to obtain the vertical projection sequence; Step 42: Detect the adjacent column index interval with a projection value of zero. When the width of this interval accounts for a proportion greater than the width of the entire outer frame and the height proportion in the vertical direction exceeds the second preset proportion, it is determined that the character has a left-right structure. Step 43: Select the column that minimizes the difference in the number of foreground pixels on the left and right sides within the adjacent column index interval as the dividing line. Define the set of foreground pixels on the left side of the dividing line as the left component and the set of foreground pixels on the right side as the right component. After completing the division of the left and right components, generate left and right structure paradigm labels.

6. The intelligent Chinese handwriting error recognition method based on deep neural networks according to claim 1, characterized in that, The acquisition of the intersection information includes: Step 51: In the set of labeled nodes, call the set of intersection points and extract the direction vector of the adjacent side of each intersection point. The direction vector is the displacement vector from the intersection point to the adjacent skeleton pixel. Step 52: Calculate the included angle between any two direction vectors and make a judgment based on the preset included angle deviation threshold and the preset vertical tolerance threshold; Step 53: When there are two pairs of approximately opposite and mutually perpendicular directions, the intersection point is determined to be X-shaped; when there is a pair of approximately opposite directions and another perpendicular direction, it is determined to be T-shaped; when there are two approximately perpendicular directions without opposite pairing, it is determined to be L-shaped. Step 54: Record the intersection points and their corresponding types as intersection point information, and generate intersection structure paradigm labels.

7. The intelligent Chinese handwriting error recognition method based on deep neural networks according to claim 1, characterized in that, The process of generating structural feature vectors includes: Step 71: Calculate the standardized metric set corresponding to the structural metric set in Step 4 based on the standardized glyphs of the target Chinese characters; Step 72: For each item in the standardized metric set, a fixed tolerance is set to form a standardized parameter space, wherein the fixed tolerance is a preset percentage of the standardized metric value; Step 73: Align the set of structured metrics with the standard parameter interval item by item, and calculate the signed deviation, interval out-of-bounds ratio, and out-of-bounds indicator for each item relative to the center of the standard parameter interval. The signed deviation is the ratio of the actual metric minus the center of the standard parameter interval to the half-width of the interval; the interval out-of-bounds ratio is the proportion of the actual metric that exceeds the width of the standard parameter interval; and the out-of-bounds indicator is a Boolean variable that indicates whether the actual metric is outside the standard parameter interval. Step 74: Assemble the alignment results obtained in step 73 into a structural feature vector according to the fixed order of the index mapping.

8. The intelligent Chinese handwriting error recognition method based on deep neural networks according to claim 1, characterized in that, The deep neural network model includes: a multi-branch input module, a gated fusion module, an attention aggregation module, and a multi-label classification output module; the multi-branch input module includes a convolutional neural network branch, a graph convolutional network branch, and a fully connected network branch. The error recognition results of Chinese single-character images output by the deep neural network model include: Step 81: Input the normalized binary glyph image into the convolutional neural network branch to extract the regional appearance features of the outer region, inner region and left and right components; input the glyph topology graph into the graph convolutional network branch to extract the structural features of stroke connection relationship and cross complexity; input the structural feature vector into the fully connected network branch to extract the out-of-bounds ratio sequence of each consistency measure and normalization parameter interval. Step 82: The gated fusion module generates a gated vector based on the out-of-bounds ratio sequence, and applies element-wise multiplication to the feature representation of each branch to weight the features; Step 83: The attention aggregation module calculates the attention weight by combining the structural paradigm label and the out-of-bounds ratio sequence. The structural paradigm label is used as the conditional input for attention after one-hot encoding, and is used to perform weighted aggregation between regional features and structural features. Step 84: Input the fused features into the multi-label classification output layer and output the error identification results. The error identification results include: whether there are structural writing errors and the error type; wherein, the error type includes: enclosing structure error, left and right structure error, and intersection structure error.

9. A Chinese handwriting error intelligent recognition system based on deep neural networks, characterized in that, The method for intelligent recognition of Chinese handwriting errors based on deep neural networks as described in any one of claims 1-8 includes: The image normalization module acquires the Chinese character image of the target Chinese character after binarization, determines the bounding box, center, height and width, and performs coordinate normalization to obtain a normalized binary character image. The skeleton topology construction module performs skeleton extraction and node annotation on the normalized binary glyph image to construct a glyph topology map describing the stroke connection relationship; The paradigm division module determines the structural paradigm labels and completes the component division based on the glyph topology map and image projection, obtaining information on the outer enclosing region and inner component region, left component and right component, and intersection point; among them, the structural paradigm labels include: enclosing structural paradigm labels, left and right structural paradigm labels, and intersection structural paradigm labels. The consistency measurement module calculates the consistency measurement of the enclosing structure, the consistency measurement of the left and right structures, and the consistency measurement of the intersection structure based on the structural paradigm label, to obtain a set of structural measurements. The feature vector generation module obtains the standardized glyphs of the target Chinese character and the standardized measurement set corresponding to the consistency measurement module, and sets a fixed tolerance to form a standardized parameter range. The structural measurement set and the standardized parameter range are aligned and encoded item by item to generate a structural feature vector. The deep recognition module takes normalized binary glyph images, glyph topology graphs, structural paradigm labels, and structural feature vectors as inputs to a trained deep neural network model. The deep neural network model outputs error recognition results for Chinese single-character images, including whether there are structural writing errors and the corresponding error types.

Images