Automatic scoring system and method for writing normativity of handwritten Chinese character image

Abstract

The invention discloses an automatic scoring system and method for writing normativity of a handwritten Chinese character image, and relates to the technical field of image data processing, and the system comprises a dictionary construction module which is used for constructing a dictionary containing a standard handwritten Chinese character template; the feature extraction module is used for acquiring handwritten Chinese character pictures and extracting feature information from the handwritten Chinese character pictures; the feature matching module is used for performing feature matching with the dictionary construction module according to the feature information of the feature extraction module, and judging which handwritten Chinese character belongs to the handwritten Chinese character picture in a standard handwritten Chinese character template dictionary; a template-based method and a machine learning-based method are combined, and a machine learning model is trained by taking similarity indexes of Chinese character features of works and Chinese character features of standard copybooks as training data, so that the interpretability of function realization is ensured, and the problem that different machine learning models need to be trained for different styles of Chinese characters is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and more specifically, to a system and method for automatically grading the writing standardization of handwritten Chinese character images. Background Art

[0002] Handwritten Chinese character standardization scoring technology refers to an algorithm in which a computer uses an algorithm to give a score to the standardization of writing of each Chinese character in a handwritten calligraphy work. The more standardized the writing, the higher the score.

[0003] Existing Chinese character scoring technologies are mainly divided into rule-based, template-based, and deep learning-based methods;

[0004] The rule-based method uses low integer value encoding to represent the stroke features and character features of Chinese characters, and manually selects thresholds to judge the correctness of strokes, and verifies the stroke actions input by users online. By formulating corresponding Chinese character feature extraction rules and various stroke evaluation rules, the Chinese character writing level is obtained by judging the classification area to which the features belong, and the decision tree is used to train the microscopic features of the strokes, so as to achieve the standardization of strokes from both macro and micro aspects;

[0005] The template-based method starts with imitation, uses a set of standardized templates as a reference, and performs similarity analysis with the Chinese characters to be scored, so as to achieve the purpose of scoring the quality of students' handwritten Chinese characters. The line spacing, character spacing, and horizontal and vertical projections of calligraphy works are combined to enrich the characteristics of the layout to achieve a more accurate evaluation of the works; at the same time, it is divided into two methods: similarity evaluation and difference evaluation;

[0006] The deep learning method is a method that uses computers to simulate the human learning process. This method continuously optimizes the algorithm performance by learning a large amount of handwritten work data and corresponding scores. It has strong learning ability and adaptability.

[0007] However, in actual use, the rule-based Chinese character quality evaluation method is simple to operate and easy to implement. However, due to the huge number and variety of Chinese characters, it is difficult to meet the evaluation needs of all Chinese characters by simply relying on the restrictions of rules. At the same time, as the number of rules increases, conflicts between rules will appear, which will lead to the resetting of rules. In addition, the styles of handwritten Chinese characters are changeable and it is difficult to constrain them with rules. Therefore, the results of this type of method are often not ideal.

[0008] The template-based method relies on the analysis and extraction of Chinese character features. Currently, most studies mainly use calligraphy experts with many years of practical experience to determine the features of Chinese characters and the importance of the features, which lacks objectivity. At the same time, there is correlation between Chinese character features, and manual weight allocation of features is prone to repeated evaluation problems. Summary of the invention

[0009] In order to solve the above problems, the present invention provides a system and method for automatically scoring the writing standardization of handwritten Chinese character images.

[0010] The present invention provides a system for automatically scoring the writing standardization of handwritten Chinese character images, comprising:

[0011] A dictionary building module, wherein the dictionary building module is used to build a dictionary containing standard handwritten Chinese character templates;

[0012] A feature extraction module, wherein the feature extraction module is used to obtain a handwritten Chinese character image and extract feature information from the handwritten Chinese character image;

[0013] A feature matching module, the feature matching module is used to perform feature matching based on the feature information of the feature extraction module and the dictionary construction module, and determine which feature of the corresponding Chinese character in the standard handwritten Chinese character template dictionary the feature in the handwritten Chinese character picture belongs to;

[0014] A feature similarity calculation module; the feature similarity calculation module is used to calculate the similarity between the feature information of the feature extraction module and the handwritten Chinese characters corresponding to the standard handwritten Chinese character template dictionary in the dictionary construction module according to the judgment result of the feature matching module;

[0015] The scoring module is used to give a writing standardization score of the handwritten Chinese characters in the handwritten Chinese character picture according to the result of the feature similarity calculation module.

[0016] Preferably, the specific working mode of the feature extraction module is as follows:

[0017] The feature extraction module includes a macroscopic level extraction unit, a mesoscopic level extraction unit and a microscopic level extraction unit;

[0018] The macro-level extraction unit is used to extract the whole character features in the handwritten Chinese character image, specifically:

[0019] Calculate the centroid coordinates of the handwritten Chinese character image, use the binarization result of the image, count the coordinates of each pixel, calculate the centroid position, and complete the Chinese character spatial position extraction;

[0020] By calculating the circumscribed rectangle of the Chinese characters in the handwritten Chinese character image, the width and height of the Chinese characters are obtained, and the size of the Chinese characters is extracted;

[0021] Calculate the aspect ratio of the Chinese characters in the handwritten Chinese character image, that is, the ratio of the width to the height of the Chinese characters, and complete the extraction of the aspect ratio of the Chinese characters;

[0022] Perform refinement on the handwritten Chinese character image, extract the skeleton structure of the Chinese character, and complete the Chinese character skeleton extraction;

[0023] Divide the handwritten Chinese character image into nine-square grids, count the pixel distribution in each small grid, and complete the pixel distribution extraction in the nine-square grid;

[0024] By scanning the outer contour of the Chinese characters in the handwritten Chinese character image, the area of ​​the outer contour of the Chinese characters is counted to complete the outer contour feature extraction;

[0025] Scan the skeleton image of the Chinese character in the handwritten Chinese character image, count the area of ​​the internal structure, and complete the inner contour feature extraction;

[0026] The meso-level extraction unit is used to extract component features in the handwritten Chinese character image;

[0027] The microscopic level extraction unit is used to extract stroke features in a handwritten Chinese character image.

[0028] Preferably, the specific working steps of the meso-level extraction unit are as follows:

[0029] Segment the Chinese characters in the handwritten Chinese character image into components, calculate the coordinates of the center of gravity of each component, determine the relative position of the center of gravity of the component and the center of gravity of the Chinese character, and complete the spatial position extraction of the component;

[0030] Calculate the circumscribed rectangle of the Chinese character in the handwritten Chinese character image to obtain the bounding box of the Chinese character, calculate the circumscribed rectangle of each component to obtain the bounding box of the component, calculate the ratio of the circumscribed rectangle area of ​​the component to the circumscribed rectangle area of ​​the Chinese character, and complete the component size extraction;

[0031] The width and height of the circumscribed rectangle of each component are calculated to extract the component aspect ratio. The aspect ratio of the component is calculated to evaluate whether the shape of the component is appropriate.

[0032] Preferably, the specific working steps of the microscopic level extraction unit are as follows:

[0033] Calculate the coordinates of the center of gravity of the Chinese character in the handwritten Chinese character image, use it as a reference point, segment the Chinese character into strokes, calculate the coordinates of the center of gravity of each stroke, determine the relative position of the stroke center of gravity and the center of gravity of the Chinese character, and complete the stroke spatial position extraction;

[0034] Thin the handwritten Chinese character image, extract the skeleton of the strokes, determine the starting and ending points of each stroke, calculate the relative position of the stroke endpoints and the center of gravity of the Chinese character, complete the stroke endpoint extraction, and evaluate the position of the stroke endpoints;

[0035] The length of the stroke is determined by the number of pixels of the stroke skeleton of the Chinese character in the handwritten Chinese character image. The skeleton of each stroke is traversed, the number of pixels is counted, and the stroke length is extracted:

[0036] Connect the starting point and ending point of each stroke of the Chinese character in the handwritten Chinese character image, calculate the slope of the connecting line, and complete the stroke slope extraction.

[0037] Preferably, the specific working mode of the feature matching module is as follows:

[0038] Start: Start the process of feature matching algorithm;

[0039] Construct a bipartite graph: Create a bipartite graph for feature matching in the model;

[0040] Initialize top labels: set initial labels for nodes in the bipartite graph;

[0041] Hungarian Algorithm: Apply the Hungarian algorithm to find the maximum matching in a bipartite graph;

[0042] Yes / No Decision: Checks whether a perfect match was found;

[0043] If yes, the process ends;

[0044] If no, go to the next step;

[0045] Modify the feasible top mark: adjust the top mark to find a better match;

[0046] End: Complete the process of feature matching algorithm.

[0047] Preferably, the working steps of the feature similarity calculation module include a multivariate linear regression unit and a BP neural network unit, both of which are used to calculate the similarity between the feature information of the feature extraction module and the handwritten Chinese characters corresponding to the standard handwritten Chinese character template dictionary in the dictionary construction module:

[0048] The multiple linear regression unit is used to perform similarity calculation through a multiple linear regression model, specifically:

[0049] Normalizing the historical feature information of the feature extraction module;

[0050] For each handwritten Chinese character image, a historical feature vector is constructed, which contains all the extracted feature values;

[0051] Taking the historical feature vector as the independent variable and the historical similarity score between the handwritten Chinese character image and the standard handwritten Chinese character template dictionary as the dependent variable, a multivariate linear regression model is constructed as a similarity scoring model.

[0052] Using the previous handwritten Chinese character images and their similarity scores, train a multivariate linear regression model;

[0053] For a new handwritten Chinese character image, using the trained model, constructing a new feature vector from the feature information of the feature extraction module, inputting the new feature vector into the multivariate linear regression model, and calculating the multivariate linear regression unit similarity score W with the standard handwritten Chinese character template dictionary;

[0054] Output the calculated multivariate linear regression unit similarity score W.

[0055] Preferably, the working steps of the BP neural network unit include:

[0056] For new handwritten Chinese character images, use the trained model to perform similarity scoring;

[0057] Feedback loop: Compare the forecast results with expert evaluations to establish a feedback loop.

[0058] Preferably, the working steps of the BP neural network unit include:

[0059] Design BP neural network model, including input layer, hidden layer and output layer;

[0060] Select activation functions for hidden and output layers;

[0061] Determine the loss function and optimizer;

[0062] The previous handwritten Chinese character images and their similarity scores are used as a dataset, and the dataset is divided into a training set, a validation set, and a test set;

[0063] Train the model, using the training set data, and monitor performance on the validation set;

[0064] The feature information of the feature extraction module and the similarity of the handwritten Chinese characters corresponding to the standard handwritten Chinese character template dictionary in the dictionary construction module are input as feature vectors into the BP neural network model, and the similarity score E of the BP neural network unit is output.

[0065] Preferably, the specific working mode of the scoring module is as follows:

[0066] Obtaining the similarity score W of the multivariate linear regression unit and the similarity score E of the BP neural network unit;

[0067] Obtaining the variance R of all similarity scores before the current time period output by the multivariate linear regression unit;

[0068] Obtaining the variance T of all similarity scores output by the BP neural network unit before the current time period;

[0069] According to the formula Calculate and obtain the confidence C of the multivariate linear regression unit1 ;

[0070] According to the formula Calculate and obtain the confidence C of the BP neural network unit 2 ;

[0071] According to the formula The final similarity score S is obtained by calculation.

[0072] The present invention also proposes a method for automatically scoring the writing standardization of handwritten Chinese character images, comprising the following steps:

[0073] Step 1: Build a dictionary containing standard handwritten Chinese character templates, obtain handwritten Chinese character images, and extract feature information from the handwritten Chinese character images;

[0074] Step 2: Perform feature matching based on the feature information of the feature extraction module and the dictionary construction module to determine which feature of the corresponding Chinese character in the standard handwritten Chinese character template dictionary the feature in the handwritten Chinese character image belongs to;

[0075] Step three: Based on the judgment result of the feature matching module, calculate the similarity between the feature information of the feature extraction module and the handwritten Chinese characters corresponding to the standard handwritten Chinese character template dictionary in the dictionary construction module, and give the writing standardization score of the handwritten Chinese characters in the handwritten Chinese character picture based on the result of the feature similarity calculation module.

[0076] Beneficial effects: The template-based and machine learning-based methods are combined, and the similarity index between the Chinese character features of the work and the Chinese character features of the standard copybook is used as training data to train the machine learning model, which not only ensures the interpretability of its function implementation, but also gets rid of the problem that different styles of Chinese characters need to train different machine learning models;

[0077] It is possible to input a picture of a handwritten Chinese character work, obtain the scoring result, and quantify the feature information of the Chinese characters. This can be used as the data basis for other downstream tasks and can be used to simulate the distribution range of real teaching evaluation scores. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 is a schematic diagram of a feature extraction module of the present invention;

[0079] Figure 2 is a schematic diagram of a macroscopic level extraction unit of the present invention;

[0080] Figure 3 is a schematic diagram of a mesoscopic level extraction unit of the present invention;

[0081] Figure 4 is a schematic diagram of a microscopic level extraction unit of the present invention;

[0082] Figure 5 It is a flow chart of the present invention as a whole;

[0083] Figure 6 It is a flow chart of the feature matching module of the present invention. DETAILED DESCRIPTION

[0084] Application scenarios: In actual use, the rule-based Chinese character quality evaluation method is simple to operate and easy to implement. However, due to the huge number and variety of Chinese characters, it is difficult to meet the evaluation needs of all Chinese characters by simply relying on the restrictions of rules. At the same time, as the number of rules increases, conflicts between rules will appear, which will lead to the resetting of rules. In addition, the styles of handwritten Chinese characters are varied and it is difficult to constrain them with rules. Therefore, the results of this type of method are often not ideal.

[0085] The template-based method relies on the analysis and extraction of Chinese character features. Currently, most studies mainly use calligraphy experts with many years of practical experience to determine the features of Chinese characters and the importance of the features, which lacks objectivity. At the same time, there is correlation between Chinese character features, and manual weight allocation of features is prone to repeated evaluation problems.

[0086] like Figures 1 to 6 Shown: A writing standardization automatic scoring system for handwritten Chinese character images, comprising:

[0087] A dictionary construction module, the dictionary construction module is used to construct a dictionary containing standard handwritten Chinese character templates; it should be noted that in this embodiment, standard handwritten Chinese character samples disclosed on the Internet or calligraphy experts can be used to collect writing samples that they believe are standard, and a dictionary of standard handwritten Chinese character templates can be constructed based on these writing samples; it should be noted that the main core features of the standard handwritten Chinese character template dictionary are feature information at three levels: macro level, meso level and micro level of Chinese characters, and the feature information at three levels of the standard handwritten Chinese character template dictionary is close to the standards of workers in this field;

[0088] A feature extraction module, wherein the feature extraction module is used to obtain a handwritten Chinese character image and extract feature information from the handwritten Chinese character image; it should be noted that, in this embodiment, the feature information includes feature information at three levels: macro level, meso level and micro level;

[0089] Specifically, at the macro level: including the spatial position, size, aspect ratio, skeleton, pixel distribution in the nine-square grid, outer contour features and inner contour features of Chinese characters;

[0090] Meso-level: includes the spatial position, size and aspect ratio of components;

[0091] Microscopic level: including the spatial position, endpoints, length and slope of the strokes;

[0092] A feature matching module, the feature matching module is used to perform feature matching based on the feature information of the feature extraction module and the dictionary construction module, and determine which handwritten Chinese character in the standard handwritten Chinese character template dictionary the handwritten Chinese character picture belongs to;

[0093] A feature similarity calculation module; the feature similarity calculation module is used to calculate the similarity between the feature information of the feature extraction module and the handwritten Chinese characters corresponding to the standard handwritten Chinese character template dictionary in the dictionary construction module according to the judgment result of the feature matching module;

[0094] The scoring module is used to give a writing standardization score of the handwritten Chinese characters in the handwritten Chinese character picture according to the result of the feature similarity calculation module.

[0095] Through the above modules, it is possible to handle a large number and types of Chinese characters, adapt to the changing handwriting styles, avoid conflicts and resetting of rules due to the increasing complexity of rules, and through automated feature extraction and matching, reduce the reliance on the personal experience and subjective judgment of calligraphy experts, improve the objectivity and accuracy of scoring, enhance the understanding and evaluation of the correlation between Chinese character features, and reduce the problem of repeated evaluation that may occur in the allocation of manual feature weights;

[0096] It should be noted that the handwritten Chinese character standardization scoring technology refers to an algorithm that uses a computer to give a score to the standardization of each Chinese character in a handwritten calligraphy work. The more standardized the writing, the higher the score. The main methods are divided into rule-based Chinese character quality evaluation methods and template-based methods;

[0097] However, in actual use, the rule-based Chinese character quality evaluation method is simple to operate and easy to implement. However, due to the huge number and variety of Chinese characters, it is difficult to meet the evaluation needs of all Chinese characters by simply relying on the restrictions of rules. At the same time, as the number of rules increases, conflicts between rules will appear, which will lead to the resetting of rules. In addition, the styles of handwritten Chinese characters are changeable and it is difficult to constrain them with rules. Therefore, the results of this type of method are often not ideal.

[0098] The template-based method relies on the analysis and extraction of Chinese character features. Currently, most studies mainly use calligraphy experts with many years of practical experience to determine the features of Chinese characters and the importance of the features, which lacks objectivity. At the same time, there is correlation between Chinese character features, and manual weight allocation of features is prone to repeated evaluation problems.

[0099] As an optional embodiment: the specific working mode of the feature extraction module is as follows:

[0100] The feature extraction module includes a macroscopic level extraction unit, a mesoscopic level extraction unit and a microscopic level extraction unit;

[0101] The macro-level extraction unit is used to extract the whole character features in the handwritten Chinese character image, specifically:

[0102] Calculate the coordinates of the center of gravity of the handwritten Chinese character image, use the binarization result of the image, count the coordinates of each pixel, calculate the center of gravity position, and complete the extraction of the spatial position of the Chinese character; it should be noted that the center of gravity position can be used to describe whether the position of the Chinese character in the Tianzi grid is offset;

[0103] By calculating the circumscribed rectangle of the Chinese character in the handwritten Chinese character image, the width and height of the Chinese character are obtained to complete the extraction of the size of the Chinese character; according to the ratio of the area of ​​the circumscribed rectangle to the area of ​​the grid, it is judged whether the writing area of ​​the Chinese character is reasonable;

[0104] Calculate the aspect ratio of the Chinese characters in the handwritten Chinese character image, that is, the ratio of the width to the height of the Chinese characters, and complete the extraction of the aspect ratio of the Chinese characters; this ratio can be used to determine the height, width and narrowness of the Chinese characters, affecting the beauty of the characters;

[0105] Perform thinning processing on the handwritten Chinese character image to extract the skeleton structure of the Chinese character and complete the Chinese character skeleton extraction; commonly used thinning algorithms include the Zhang-Suen thinning algorithm, which can effectively retain the structural characteristics of Chinese characters;

[0106] The handwritten Chinese character image is divided into nine-square grids, and the pixel distribution in each small grid is counted to complete the pixel distribution extraction in the nine-square grid; by calculating the proportion of white pixels (strokes) in each small grid to the total area, it is judged whether the distribution of each part of the Chinese character is symmetrical;

[0107] By scanning the outer contour of the Chinese characters in the handwritten Chinese character image and counting the area of ​​the outer contour of the Chinese characters, the outer contour feature extraction is completed; the elastic grid technology can be used to improve the robustness to character deformation and ensure the accuracy of the outer contour feature;

[0108] Scan the skeleton image of the Chinese character in the handwritten Chinese character image, count the area of ​​the internal structure, and complete the inner contour feature extraction; extract the inner contour feature of the Chinese character by comparing the area of ​​the first and second passes through the stroke;

[0109] The meso-level extraction unit is used to extract component features in the handwritten Chinese character image;

[0110] The microscopic level extraction unit is used to extract stroke features in a handwritten Chinese character image.

[0111] It should be noted that by extracting the coordinates of the center of gravity, circumscribed rectangle, aspect ratio, skeleton structure, pixel distribution within the nine-square grid, and outer contour features of Chinese characters, it is possible to evaluate whether the overall structure and layout of the Chinese characters are standardized. For example, if the center of gravity of a Chinese character is offset, it may mean that the structure of the character is unstable when written; if the aspect ratio of the Chinese character is unreasonable, it may affect the beauty and recognition of the character.

[0112] As an optional embodiment: the specific working steps of the meso-level extraction unit are as follows:

[0113] Segment the Chinese characters in the handwritten Chinese character image into components, calculate the coordinates of the center of gravity of each component, determine the relative position of the center of gravity of the component and the center of gravity of the Chinese character, and complete the component spatial position extraction to evaluate whether the component position is offset;

[0114] Calculate the circumscribed rectangle of the Chinese characters in the handwritten Chinese character image to obtain the bounding box of the Chinese characters, calculate the circumscribed rectangle of each component to obtain the bounding box of the component, calculate the ratio of the circumscribed rectangle area of ​​the component to the circumscribed rectangle area of ​​the Chinese character, and complete the component size extraction to evaluate whether the component size is reasonable;

[0115] The width and height of the circumscribed rectangle of each component are calculated to extract the component aspect ratio. The aspect ratio of the component is calculated to evaluate whether the shape of the component is appropriate.

[0116] It should be noted that by segmenting components, calculating component center of gravity coordinates, extracting component circumscribed rectangle area ratios and component aspect ratios, it is possible to evaluate whether the position, size and shape of each component in a Chinese character conforms to writing standards. For example, if the center of gravity of a component is offset relative to the center of gravity of the Chinese character, it may mean that the component is not written accurately; if the aspect ratio of the component does not conform to the standard ratio, it may mean that the component is written too wide or too narrow.

[0117] As an optional embodiment: the specific working steps of the microscopic level extraction unit are as follows:

[0118] Calculate the coordinates of the center of gravity of the Chinese character in the handwritten Chinese character image, use them as reference points, segment the Chinese character into strokes, calculate the coordinates of the center of gravity of each stroke, determine the relative position of the center of gravity of the stroke and the center of gravity of the Chinese character, and complete the extraction of the spatial position of the strokes; in order to evaluate whether the stroke position is offset;

[0119] Thin the handwritten Chinese character image, extract the skeleton of the strokes, determine the starting and ending points of each stroke, calculate the relative position of the stroke endpoints and the center of gravity of the Chinese character, complete the stroke endpoint extraction, and evaluate the position of the stroke endpoints;

[0120] The length of the stroke is determined by the number of pixels of the stroke skeleton of the Chinese character in the handwritten Chinese character image. The skeleton of each stroke is traversed, the number of pixels is counted, and the stroke length is extracted:

[0121] Connect the starting points and ending points of each stroke of the Chinese character in the handwritten Chinese character image, calculate the slope of the connection line, and complete the extraction of the stroke slope to determine the stroke direction;

[0122] It should be noted that through stroke segmentation, calculation of stroke center-of-gravity coordinates, extraction of stroke endpoints, extraction of stroke lengths, and extraction of stroke slopes, it is possible to evaluate whether the position, length, and direction of each stroke in a Chinese character are standard. For example, if the positions of the starting point and ending point of a stroke are inaccurate, it may mean that the starting or ending position of the stroke writing is incorrect; if the stroke slope is abnormal, it may mean that the inclination of the stroke does not conform to the writing standard.

[0123] As an optional embodiment: The specific working method of the above feature matching module is as follows:

[0124] Start: Initiate the process of the feature matching algorithm;

[0125] Construct a bipartite graph: Create a bipartite graph for feature matching in the model;

[0126] Initialize the vertex labels: Set the initial labels for the nodes in the bipartite graph;

[0127] Hungarian algorithm: Apply the Hungarian algorithm to find the maximum matching in the bipartite graph;

[0128] Yes / No judgment: Check whether a perfect matching has been found;

[0129] If yes, end the process;

[0130] If no, proceed to the next step;

[0131] Modify the feasible vertex labels: Adjust the vertex labels to find a better matching;

[0132] End: Complete the process of the feature matching algorithm.

[0133] It should be noted that, for example, we have an image of a handwritten Chinese character, and our goal is to use the feature matching module to determine whether this handwritten Chinese character matches the corresponding character in the standard template;

[0134] For example, the Chinese character in this handwritten Chinese character image is "秀", and the feature matching module compares the features of the handwritten "秀" with the standard "秀" character template in the dictionary;

[0135] Compare the spatial position features (center-of-gravity positions) of the "禾" component and the "乃" component of the handwritten "秀" with the corresponding components in the standard template to see if they are aligned; if the center of gravity of the handwritten "禾" component is biased to the left while it is centered in the standard template, then the matching degree will decrease;

[0136] The strokes in the handwritten "xiu", such as the horizontal stroke of "he" and the left-falling stroke of "nai", have their endpoints, lengths, and slopes matched with the strokes in the standard template. For example, if the starting point of the left-falling stroke of the handwritten "xiu" does not coincide with the starting point position in the standard template, then this feature matching will fail.

[0137] The feature matching module will output a matching result, indicating the degree of matching between the handwritten "xiu" and the standard template. If all features are highly consistent with the standard template, then the matching is successful, and we can determine that the handwritten Chinese character "xiu" matches the standard template.

[0138] Suppose the center of gravity of the "he" component in the handwritten "xiu" is offset, while the size of the "nai" component is appropriate. Then the feature matching module will identify the mismatch of the "he" component and may require the user to adjust the position of the "he" component to better match the standard template.

[0139] Through this example, we can see how the feature matching module works and how it helps to identify the degree of matching between the handwritten Chinese character and the standard template. This process is automated and can quickly match and identify a large number of handwritten Chinese characters.

[0140] As an optional embodiment: The working steps of the feature similarity calculation module include a multiple linear regression unit and a BP neural network unit, both of which are used to calculate the similarity between the feature information of the feature extraction module and the corresponding handwritten Chinese characters in the standard handwritten Chinese character template dictionary of the dictionary construction module.

[0141] The multiple linear regression unit is used to calculate the similarity through a multiple linear regression model, specifically as follows:

[0142] Normalize the historical feature information of the feature extraction module. It should be noted that in this embodiment, Min-MaxScaler or StandardScaler can be used.

[0143] For each handwritten Chinese character image, construct a historical feature vector containing all the extracted feature values. It should be noted that the historical feature vector is the historical feature vector of each handwritten Chinese character image collected previously, and these feature vectors are used to train the model.

[0144] Take the historical feature vector as the independent variable and the historical similarity score between the handwritten Chinese character image and the standard handwritten Chinese character template dictionary as the dependent variable, and construct a multiple linear regression model as the similarity score model. It should be noted that in this embodiment, the historical similarity score between the handwritten Chinese character image and the standard handwritten Chinese character template dictionary is obtained through the corresponding system and expert evaluation.

[0145] Using the previous handwritten Chinese character images and their similarity scores, train a multivariate linear regression model;

[0146] For a new handwritten Chinese character image, using the trained model, constructing a new feature vector from the feature information of the feature extraction module, inputting the new feature vector into the multivariate linear regression model, and calculating the multivariate linear regression unit similarity score W with the standard handwritten Chinese character template dictionary;

[0147] Output the calculated multivariate linear regression unit similarity score W.

[0148] As an optional embodiment: the working steps of the BP neural network unit include: It should be noted that although the multivariate linear regression unit can obtain a similarity score, this method is highly dependent on the quality of previous data. The quality of the data determines the quality of the model. In practical applications, due to the limitation of the amount of data, it is difficult to accurately predict the inherent laws of the data. This technical solution can solve the above defects;

[0149] For new handwritten Chinese character images, use the trained model to perform similarity scoring;

[0150] Feedback loop: Compare the forecast results with expert evaluations to establish a feedback loop.

[0151] As an optional embodiment: the working steps of the BP neural network unit include:

[0152] Design a BP neural network model, including input layer, hidden layer and output layer; it should be noted that the input layer: the number of nodes is equal to the number of features, the hidden layer: at least one hidden layer, the number of nodes can be determined according to the complexity of the problem and the amount of data, usually need to be adjusted through experiments, the output layer: a single node, output similarity score;

[0153] Select the activation function of the hidden layer and the output layer; it should be noted that, in this embodiment, the hidden layer: usually uses ReLU or tanh as the activation function, and the output layer: uses a linear activation function because the similarity score is a continuous value;

[0154] Determine a loss function and an optimizer; in this embodiment, the loss function is: mean square error (MSE) or root mean square error (RMSE), and the optimizer is: Adam or SGD (stochastic gradient descent);

[0155] The previous handwritten Chinese character images and their similarity scores are used as a dataset, and the dataset is divided into a training set, a validation set, and a test set; the usual ratio is 70% training set, 15% validation set, and 15% test set;

[0156] Train the model, using the training set data, and monitor performance on the validation set;

[0157] The feature information newly extracted by the feature extraction module is input into the BP neural network model as a feature vector, and the similarity score E of the BP neural network unit is output.

[0158] For example, we have an image of handwritten Chinese characters, and the following are the feature data extracted from the macro, meso, and micro levels:

[0159] Macro-level characteristics:

[0160] Spatial position (center coordinates): (0.3, 0.5);

[0161] size(width, height): (0.8, 0.6);

[0162] Aspect ratio: 1.33;

[0163] Skeleton: The ratio of skeleton pixels to total area is 0.15;

[0164] Pixel distribution in the nine-square grid: [0.1, 0.2, 0.3, 0.4, 0.5, 0.4, 0.3, 0.2, 0.1];

[0165] Outer contour features: The ratio of outer contour area to total area is 0.2;

[0166] Inner contour feature: The ratio of inner contour area to total area is 0.1;

[0167] Meso-level characteristics:

[0168] Component space position (center coordinates): (0.4, 0.6);

[0169] Component size (circumscribed rectangle area): 0.7;

[0170] Width ratio of components: 1.2;

[0171] Microscopic characteristics:

[0172] Stroke space position (center of gravity coordinates): (0.5, 0.7);

[0173] Stroke endpoints: starting point (0.2, 0.3), ending point (0.8, 0.9);

[0174] Stroke length: 1.5;

[0175] Stroke slope: The slope of the line connecting the starting point and the ending point is calculated to be 0.75;

[0176] Input layer: The number of nodes in the input layer is 15, corresponding to the above 15 features;

[0177] Hidden layer: Suppose we set two hidden layers, the first hidden layer has 20 nodes and the second hidden layer has 10 nodes;

[0178] Output layer: The number of nodes in the output layer is 1, and the similarity score is output;

[0179] Activation function: The hidden layer uses the ReLU activation function, and the output layer uses the linear activation function;

[0180] Loss function: mean square error (MSE);

[0181] Optimizer: Adam optimizer;

[0182] Assume that we have 100 annotated images of handwritten Chinese characters and their feature data, as well as similarity scores with standard templates; we divide these data into a training set (70 samples), a validation set (15 samples), and a test set (15 samples);

[0183] Use the training set data to train the BP neural network model and monitor the model performance on the validation set to prevent overfitting. During the training process, we adjust parameters such as learning rate and batch size to obtain the best model performance.

[0184] For new images of handwritten Chinese characters, we extract the same 15 features, construct feature vectors, and input them into the trained BP neural network model to calculate the similarity score with the standard handwritten Chinese character template;

[0185] The model outputs a predicted similarity score, for example 0.85, which means that the handwritten Chinese character image is 85% similar to the standard template.

[0186] As an optional embodiment: the specific working mode of the scoring module is as follows:

[0187] Obtaining the similarity score W of the multivariate linear regression unit and the similarity score E of the BP neural network unit;

[0188] Obtaining the variance R of all similarity scores before the current time period output by the multivariate linear regression unit;

[0189] Obtaining the variance T of all similarity scores output by the BP neural network unit before the current time period;

[0190] According to the formula Calculate and obtain the confidence C of the multivariate linear regression unit 1 ;

[0191] According to the formula Calculate and obtain the confidence C of the BP neural network unit 2 ;

[0192] According to the formula The final similarity score S is calculated and obtained. It should be noted that the confidence can dynamically reflect the latest performance of the model, increase the diversity of model fusion, comprehensively consider the confidence and the correlation between models, adjust the weight of the model in the final score, and make the final similarity score more accurate and robust.

[0193] The present invention also proposes a method for automatically scoring the writing standardization of handwritten Chinese character images, comprising the following steps:

[0194] Step 1: Build a dictionary containing standard handwritten Chinese character templates, obtain handwritten Chinese character images, and extract feature information from the handwritten Chinese character images;

[0195] Step 2: Perform feature matching based on the feature information of the feature extraction module and the dictionary construction module to determine which feature of the corresponding Chinese character in the standard handwritten Chinese character template dictionary the feature in the handwritten Chinese character image belongs to;

[0196] Step three: Based on the judgment result of the feature matching module, calculate the similarity between the feature information of the feature extraction module and the handwritten Chinese characters corresponding to the standard handwritten Chinese character template dictionary in the dictionary construction module, and give the writing standardization score of the handwritten Chinese characters in the handwritten Chinese character picture based on the result of the feature similarity calculation module.

[0197] How it works

[0198] Construct a dictionary containing standard handwritten Chinese character templates, obtain handwritten Chinese character images, and extract feature information from the handwritten Chinese character images; perform feature matching based on the feature information of the feature extraction module and the dictionary construction module to determine which handwritten Chinese character in the standard handwritten Chinese character template dictionary the handwritten Chinese character image belongs to; calculate the similarity between the feature information of the feature extraction module and the handwritten Chinese character corresponding to the standard handwritten Chinese character template dictionary in the dictionary construction module based on the judgment result of the feature matching module, and give the writing standardization score of the handwritten Chinese characters in the handwritten Chinese character image based on the result of the feature similarity calculation module. The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technical staff in this technical field, several improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of this template.

Claims

1. An automatic scoring system for the writing standardization of handwritten Chinese character images, characterized in that: include: A dictionary building module, wherein the dictionary building module is used to build a dictionary containing standard handwritten Chinese character templates; A feature extraction module, wherein the feature extraction module is used to obtain a handwritten Chinese character image and extract feature information from the handwritten Chinese character image; A feature matching module, the feature matching module is used to perform feature matching based on the feature information of the feature extraction module and the dictionary construction module, and determine which feature of the corresponding Chinese character in the standard handwritten Chinese character template dictionary the feature in the handwritten Chinese character picture belongs to; Feature similarity calculation module; The feature similarity calculation module is used to calculate the similarity between the feature information of the feature extraction module and the handwritten Chinese characters corresponding to the standard handwritten Chinese character template dictionary in the dictionary construction module according to the judgment result of the feature matching module; The scoring module is used to give a writing standardization score of the handwritten Chinese characters in the handwritten Chinese character picture according to the result of the feature similarity calculation module.

2. The automatic scoring system for writing standardization of handwritten Chinese character images according to claim 1 is characterized in that: The specific working mode of the feature extraction module is as follows: The feature extraction module includes a macroscopic level extraction unit, a mesoscopic level extraction unit and a microscopic level extraction unit; The macro-level extraction unit is used to extract the whole character features in the handwritten Chinese character image, specifically: Calculate the centroid coordinates of the handwritten Chinese character image, use the binarization result of the image, count the coordinates of each pixel, calculate the centroid position, and complete the Chinese character spatial position extraction; By calculating the circumscribed rectangle of the Chinese characters in the handwritten Chinese character image, the width and height of the Chinese characters are obtained, and the size of the Chinese characters is extracted; Calculate the aspect ratio of the Chinese characters in the handwritten Chinese character image, that is, the ratio of the width to the height of the Chinese characters, and complete the extraction of the aspect ratio of the Chinese characters; Perform refinement on the handwritten Chinese character image, extract the skeleton structure of the Chinese character, and complete the Chinese character skeleton extraction; Divide the handwritten Chinese character image into nine-square grids, count the pixel distribution in each small grid, and complete the pixel distribution extraction in the nine-square grid; By scanning the outer contour of the Chinese characters in the handwritten Chinese character image, the area of ​​the outer contour of the Chinese characters is counted to complete the outer contour feature extraction; Scan the skeleton image of the Chinese character in the handwritten Chinese character image, count the area of ​​the internal structure, and complete the inner contour feature extraction; The meso-level extraction unit is used to extract component features in the handwritten Chinese character image; The microscopic level extraction unit is used to extract stroke features in a handwritten Chinese character image.

3. The automatic scoring system for handwritten Chinese character images according to claim 2 is characterized in that: The specific working steps of the meso-level extraction unit are as follows: Segment the Chinese characters in the handwritten Chinese character image into components, calculate the coordinates of the center of gravity of each component, determine the relative position of the center of gravity of the component and the center of gravity of the Chinese character, and complete the spatial position extraction of the component; Calculate the circumscribed rectangle of the Chinese character in the handwritten Chinese character image to obtain the bounding box of the Chinese character, calculate the circumscribed rectangle of each component to obtain the bounding box of the component, calculate the ratio of the circumscribed rectangle area of ​​the component to the circumscribed rectangle area of ​​the Chinese character, and complete the component size extraction; The width and height of the circumscribed rectangle of each component are calculated to extract the component aspect ratio. The aspect ratio of the component is calculated to evaluate whether the shape of the component is appropriate.

4. The automatic scoring system for writing standardization of handwritten Chinese character images according to claim 2 is characterized in that: The specific working steps of the microscopic level extraction unit are as follows: Calculate the coordinates of the center of gravity of the Chinese character in the handwritten Chinese character image, use it as a reference point, segment the Chinese character into strokes, calculate the coordinates of the center of gravity of each stroke, determine the relative position of the stroke center of gravity and the center of gravity of the Chinese character, and complete the stroke spatial position extraction; Thin the handwritten Chinese character image, extract the skeleton of the strokes, determine the starting and ending points of each stroke, calculate the relative position of the stroke endpoints and the center of gravity of the Chinese character, complete the stroke endpoint extraction, and evaluate the position of the stroke endpoints; The length of the stroke is determined by the number of pixels of the stroke skeleton of the Chinese character in the handwritten Chinese character image. The skeleton of each stroke is traversed, the number of pixels is counted, and the stroke length is extracted: Connect the starting point and ending point of each stroke of the Chinese character in the handwritten Chinese character image, calculate the slope of the connecting line, and complete the stroke slope extraction.

5. The automatic scoring system for writing standardization of handwritten Chinese character images according to claim 1 is characterized in that: The specific working mode of the feature matching module is as follows: Start: Start the process of feature matching algorithm; Construct a bipartite graph: Create a bipartite graph for feature matching in the model; Initialize top labels: set initial labels for nodes in the bipartite graph; Hungarian Algorithm: Apply the Hungarian algorithm to find the maximum matching in a bipartite graph; Yes / No Decision: Checks whether a perfect match was found; If yes, the process ends; If no, go to the next step; Modify the feasible top mark: adjust the top mark to find a better match; End: Complete the process of feature matching algorithm.

6. The automatic scoring system for handwritten Chinese character images according to claim 5, characterized in that: The working steps of the feature similarity calculation module include a multivariate linear regression unit and a BP neural network unit, both of which are used to calculate the similarity between the feature information of the feature extraction module and the handwritten Chinese characters corresponding to the standard handwritten Chinese character template dictionary in the dictionary construction module: The multiple linear regression unit is used to perform similarity calculation through a multiple linear regression model, specifically: Normalizing the historical feature information of the feature extraction module; For each handwritten Chinese character image, a historical feature vector is constructed, which contains all the extracted feature values; Taking the historical feature vector as the independent variable and the historical similarity score between the handwritten Chinese character image and the standard handwritten Chinese character template dictionary as the dependent variable, a multivariate linear regression model is constructed as a similarity scoring model. Using the previous handwritten Chinese character images and their similarity scores, train a multivariate linear regression model; For a new handwritten Chinese character image, using the trained model, constructing a new feature vector from the feature information of the feature extraction module, inputting the new feature vector into the multivariate linear regression model, and calculating the multivariate linear regression unit similarity score W with the standard handwritten Chinese character template dictionary; Output the calculated multivariate linear regression unit similarity score W.

7. The automatic scoring system for handwritten Chinese character images according to claim 6, characterized in that: The working steps of the BP neural network unit include: For new handwritten Chinese character images, use the trained model to perform similarity scoring; Feedback loop: Compare the forecast results with expert evaluations to establish a feedback loop.

8. The automatic scoring system for handwritten Chinese character images according to claim 6, characterized in that: The working steps of the BP neural network unit include: Design BP neural network model, including input layer, hidden layer and output layer; Select activation functions for hidden and output layers; Determine the loss function and optimizer; The previous handwritten Chinese character images and their similarity scores are used as a dataset, and the dataset is divided into a training set, a validation set, and a test set; Train the model, using the training set data, and monitor performance on the validation set; The feature information of the feature extraction module and the similarity of the handwritten Chinese characters corresponding to the standard handwritten Chinese character template dictionary in the dictionary construction module are input as feature vectors into the BP neural network model, and the similarity score E of the BP neural network unit is output.

9. The automatic scoring system for writing standardization of handwritten Chinese character images according to claim 8, characterized in that: The specific working method of the scoring module is as follows: Obtaining the similarity score W of the multivariate linear regression unit and the similarity score E of the BP neural network unit; Obtaining the variance R of all similarity scores before the current time period output by the multivariate linear regression unit; Obtaining the variance T of all similarity scores output by the BP neural network unit before the current time period; According to the formula Calculate and obtain the confidence C1 of the multivariate linear regression unit; According to the formula Calculate and obtain the confidence C2 of the BP neural network unit; According to the formula The final similarity score S is obtained by calculation.

10. A method for automatically scoring the writing standardization of handwritten Chinese character images, characterized in that: The following steps are involved: Step 1: Build a dictionary containing standard handwritten Chinese character templates, obtain handwritten Chinese character images, and extract feature information from the handwritten Chinese character images; Step 2: Perform feature matching based on the feature information of the feature extraction module and the dictionary construction module to determine which feature of the corresponding Chinese character in the standard handwritten Chinese character template dictionary the feature in the handwritten Chinese character image belongs to; Step three: Based on the judgment result of the feature matching module, calculate the similarity between the feature information of the feature extraction module and the handwritten Chinese characters corresponding to the standard handwritten Chinese character template dictionary in the dictionary construction module, and give the writing standardization score of the handwritten Chinese characters in the handwritten Chinese character picture based on the result of the feature similarity calculation module.

Images