An offline handwritten Chinese character scoring method and device and a storage medium

Abstract

The application discloses a kind of offline handwritten Chinese character scoring method, device and storage medium, wherein method includes: obtaining handwritten Chinese character image to be scored;Handwritten Chinese character image is input into trained Chinese character scoring model, the characteristic structure of handwritten Chinese character is extracted, the corresponding Chinese character in standard library is compared and analyzed with the characteristic structure extracted, and the comprehensive score of handwritten Chinese character is obtained;And / or, based on the distribution law of Chinese character pixel points, the two-dimensional distribution of Chinese character pixel points in handwritten Chinese character image is analyzed, a plurality of two-dimensional features are obtained, and the corresponding Chinese character in standard library is compared and analyzed according to the plurality of two-dimensional features obtained, and the score on different dimensions of handwritten Chinese character is obtained.The application processes handwritten character image by model, or processes handwritten character image according to Chinese character pixel points, can score handwritten Chinese character as a whole, and provides improvement reference for writer.The application can be widely applied in image pattern recognition technical field.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence, deep learning and image pattern recognition technology, and in particular to an offline scoring method, device and storage medium for handwritten Chinese characters. Background Technology

[0002] Offline handwritten Chinese character recognition and scoring is a subfield of image pattern recognition. Offline refers to the processing of handwritten text, specifically two-dimensional images of handwritten Chinese characters captured by image capture devices such as scanners or cameras; hereinafter, this will be referred to simply as handwritten Chinese characters. Currently, feedback and guidance for student practice mainly rely on human intervention. However, different people have different aesthetic preferences, making it difficult to standardize criteria. This leads to low efficiency and unreliable results for human scoring. Therefore, the ideal approach is to teach computers how to score.

[0003] Although there are already some deep learning-based handwritten Chinese character recognition methods in use, and their recognition accuracy has reached over 90%, there is still a lack of research on their scoring.

[0004] Contrastive learning is a rapidly developing subfield of deep learning in recent years, and it is a form of self-supervised learning. Its basic idea is to bring the representations of the same image from different perspectives closer together in the feature space, also known as the individual discrimination task, in order to obtain a better feature representation of an image. The extracted features can then be used for various downstream tasks, such as image classification, object segmentation, etc. This idea has been widely applied in the field of computer vision.

[0005] For scoring handwritten Chinese characters, traditional machine learning methods rely on experts to design and select indicators that influence the aesthetic quality of handwritten characters, such as line thickness, smoothness, and font regularity. These indicators are scored using manually set thresholds, and the final score is obtained by combining all scores. However, relying solely on expert experience is insufficient to address the vast differences in handwritten Chinese characters. The main drawbacks are: 1. Since the indicators are manually selected, it's unclear whether other indicators should be selected, their importance to the final score, or whether different characters require different scoring indicators; 2. Individual aesthetic preferences vary, making the manual scoring of the scoring dataset extremely labor-intensive, and errors are difficult to assess; 3. It's impossible to analyze the correlations between the indicators or quantitatively determine their contribution to the final score. Summary of the Invention

[0006] In order to at least partially solve one of the technical problems existing in the prior art, the purpose of this invention is to provide an offline scoring method, device and storage medium for handwritten Chinese characters.

[0007] The technical solution adopted in this invention is:

[0008] An offline scoring method for handwritten Chinese characters includes the following steps:

[0009] Obtain images of the handwritten Chinese characters to be scored;

[0010] The handwritten Chinese character image is input into the trained Chinese character scoring model, the feature structure of the handwritten Chinese character is extracted, and the extracted feature structure is compared and analyzed with the corresponding Chinese characters in the standard library to obtain the comprehensive score of the handwritten Chinese character.

[0011] And / or,

[0012] Based on the distribution pattern of Chinese character pixels, the two-dimensional distribution of Chinese character pixels in handwritten Chinese character images is analyzed to obtain various two-dimensional features. The obtained two-dimensional features are compared and analyzed with the corresponding Chinese characters in the standard library to obtain scores of handwritten Chinese characters in different dimensions.

[0013] Furthermore, the Chinese character scoring model is trained in the following way:

[0014] Collect an offline handwritten Chinese character dataset with category information, and obtain an evaluation dataset based on the offline handwritten Chinese character dataset;

[0015] A convolutional neural network was used as the backbone network for extracting features of handwritten Chinese characters, and it was pre-trained using an offline handwritten Chinese character dataset.

[0016] After pre-training, a classification and recognition network is added after the backbone network, trained using an offline handwritten Chinese character dataset, and the parameters of the backbone network are fixed after training.

[0017] A score regression prediction network is added in parallel after the skeleton network, and trained using the evaluation dataset and a preset loss function to obtain the final Chinese character scoring model.

[0018] Furthermore, the method of using a convolutional neural network as the backbone network for extracting handwritten Chinese character features, and pre-training it using an offline handwritten Chinese character dataset, includes:

[0019] Data augmentation was performed on the offline handwritten Chinese character dataset so that each image could be viewed from two different perspectives; the two different perspectives of the same image were used as positive sample pairs, and the rest were used as negative sample pairs.

[0020] The backbone network was trained using the enhanced dataset and the InfoNCE loss function;

[0021] The expression for the InfoNCE loss function is as follows:

[0022]

[0023] In the formula, x represents the image data after data augmentation, x+ represents the positive sample of x, and x- represents the negative sample of x; f_q represents the skeleton network, a mapping MLP layer and a prediction MLP layer, f_k represents the skeleton network and a mapping MLP layer, and f_k shares the parameters of f_q in a certain proportion, which is called the momentum encoder; the symbol · represents the similarity calculation method, and τ is the temperature hyperparameter.

[0024] Furthermore, a classification and recognition network is added after the backbone network, trained using an offline handwritten Chinese character dataset. After training, the parameters of the backbone network are fixed, including:

[0025] Random transformation processing is performed on the data in the offline handwritten Chinese character dataset;

[0026] The classification results are constrained using cross-entropy loss and center distance loss functions. The purpose of using the center distance loss function is to address the issue of having many types of Chinese characters, allowing the feature representations of different types of characters to differ more significantly after passing through the network model.

[0027] Loss = Loss crossEntropy +λLoss Center

[0028]

[0029]

[0030] Where λ is a hyperparameter; W and b are trainable model parameters in the classification prediction network; x i y represents the features extracted from image data through a convolutional neural network. i is the category label of image i; m represents the number of images in a batch, and n represents the total number of categories; It is class y i The feature center.

[0031] Furthermore, the step of adding a score regression prediction network in parallel after the skeleton network, and training it using an evaluation dataset and a preset loss function, includes:

[0032] The activation function of the score regression prediction network is the softmax function, which is used to classify handwritten Chinese characters into three categories: excellent, good, and poor. The score is calculated based on the classification results. Optionally, each group of data in the evaluation dataset contains one standard character and five handwritten Chinese characters.

[0033] Specifically, the triplet loss function and the absolute value loss function are used to constrain the scoring results of the score regression prediction network:

[0034]

[0035]

[0036] Loss Triplet =max(d(a,p)-d(a,n)+margin,0)

[0037] In the formula, λ is a hyperparameter; y i Let be the score of image i obtained according to statistical rules, and score(i) be the score of image i obtained through a neural network; 'a' represents the anchor image, which is the standard character in a set of data; 'p' represents the positive sample image, which is the top-ranked handwritten Chinese character in a set of data; 'n' represents the negative sample image, which is the bottom-ranked handwritten Chinese character in a set of data; 'margin' is a hyperparameter, and 'd' represents the difference between the scores obtained by the two images after passing through the scoring network.

[0038] Furthermore, the various two-dimensional features include the fullness feature, symmetry feature, center of gravity position feature, and compactness feature of handwritten Chinese characters; specifically defined as follows:

[0039] Fullness feature: It is represented by the ratio of the perimeter of the convex hull of the handwritten Chinese character to its minimum bounding box, and the ratio of the number of black pixels in the convex hull to the area of ​​the convex hull. The score reflects whether the fullness of the handwritten Chinese character is consistent with the standard character.

[0040] Uniformity feature: It is represented by the distribution of black pixels in different regions of handwritten Chinese characters. The score reflects whether the overall uniformity of the handwritten Chinese characters is consistent with the standard characters.

[0041] Center of gravity position characteristics: The score reflects whether the center of gravity position of the handwritten Chinese character is consistent with the standard character;

[0042] Compactness feature: The score reflects whether the overall compactness of the handwritten Chinese characters is consistent with the standard characters.

[0043] Furthermore, the fullness feature is represented by a two-dimensional vector, defined as the perimeter ratio f1 of the convex hull and the minimum bounding box, and the convexity f2. The specific calculation formula is as follows:

[0044]

[0045] In the formula, P convex P represents the perimeter of the convex hull of the handwritten Chinese character. bounding_box C represents the perimeter of the smallest bounding box of the handwritten Chinese character; convex Indicates the number of black pixels in the convex hull of the handwritten Chinese character; A convex This represents the area of ​​the convex hull of the handwritten Chinese character;

[0046] The symmetry feature is represented by an eight-dimensional vector f3-f 10 This is used to represent and reflect the distribution of pixels. The specific calculation formula is as follows:

[0047]

[0048]

[0049] In the formula, f 2+i During calculation, the handwritten Chinese character is divided into four equal regions according to the constraint box, which is defined as the smallest square that exactly encloses the entire handwritten Chinese character; S i represents the number of black pixels in region i; n represents the number of black pixels constituting the entire Chinese character; f 5+j In the calculation, the convex hull of the handwritten Chinese character is divided into four regions based on its center point; C convex(j) A represents the number of black pixels in the convex hull region j; convex(j) Let j represent the area of ​​the convex hull region.

[0050] The centroid position feature is represented by a two-dimensional vector f. 11 -f 12 The specific calculation formula is as follows:

[0051]

[0052]

[0053] In the formula, W and H represent the width and height of the handwritten Chinese character image; n represents the number of black pixels constituting the handwritten Chinese character; I i，j It is an indicator function, when the point (x) on the image... i y j A value of 1 is assigned when a pixel is black, and 0 otherwise.

[0054] The compactness feature is represented by a seven-dimensional vector f. 13 -f 19 This represents the normalized average distance between all n black pixels constituting a Chinese character and the center point, as well as the overall distribution of the pixels. The pixel distribution is first represented by dividing all black pixels into 16 regions evenly using three horizontal and three vertical lines, and then using the position coordinates of these six lines. The specific calculation formula is as follows:

[0055]

[0056]

[0057]

[0058] In the formula, W and H represent the width and height of the handwritten Chinese character image; x center y center Indicates the coordinates of the center point; I i，j It is an indicator function, when the point (x) on the image... i y j When a pixel is black, the value is 1; otherwise, it is 0. n represents the number of black pixels constituting the handwritten Chinese character; a represents the side length of the handwritten Chinese character image; x v(i) Represents the row coordinate of the i-th vertical line; y h(j) This represents the column coordinate of the j-th vertical line.

[0059] Furthermore, the calculation formulas for the scores on different dimensions are as follows:

[0060]

[0061] Where f and f standard This represents the characteristic features of handwritten Chinese characters and their corresponding standard characters from a certain angle; ||ff standard || represents the difference between the two, calculated using the L2 norm; bound represents the range of variation of this feature compared to the standard character.

[0062] Another technical solution adopted in this invention is:

[0063] An offline scoring device for handwritten Chinese characters includes:

[0064] At least one processor;

[0065] At least one memory for storing at least one program;

[0066] When the at least one program is executed by the at least one processor, the at least one processor implements the method described above.

[0067] Another technical solution adopted in this invention is:

[0068] A computer-readable storage medium storing a processor-executable program, which, when executed by a processor, performs the method described above.

[0069] The beneficial effects of this invention are: by processing handwritten text images through models or by processing handwritten text images based on Chinese character pixels, this invention can provide an overall score for handwritten Chinese characters, offering writers a reference for improvement. Attached Figure Description

[0070] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of the present invention or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0071] Figure 1 This is an overall flowchart of a deep learning-based handwritten Chinese character recognition and scoring method in an embodiment of the present invention;

[0072] Figure 2 This is a flowchart of an offline handwritten Chinese character scoring method based on the two-dimensional distribution of handwritten Chinese character pixels in an embodiment of the present invention. Detailed Implementation

[0073] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0074] In the description of this invention, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0075] In the description of this invention, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0076] In the description of this invention, unless otherwise explicitly defined, terms such as "set up," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0077] like Figure 1 As shown, this embodiment provides an offline handwritten Chinese character recognition and scoring method based on deep learning. By constructing a handwritten Chinese character dataset and designing a reasonable and effective training method, the algorithm can accurately identify the type of handwritten Chinese characters and provide an overall score that is more in line with human subjective aesthetics. The method specifically includes the following steps:

[0078] S1. Collect an offline handwritten Chinese character dataset with category information, select a portion of it and label it according to human subjective aesthetics, and at the same time construct an evaluation dataset by combining it with the Chinese character standard library data.

[0079] Specifically, step S1 includes steps S11-S13:

[0080] S11. The collection of handwritten Chinese character datasets can utilize the open-source CASIA-HWDB1.0-1.2 (hereinafter referred to as HWDB). This dataset consists of offline handwritten Chinese character samples collected by the Institute of Automation, Chinese Academy of Sciences, written by 1020 writers, containing a total of 7185 characters and 3,721,874 images. It is one of the most comprehensive and authoritative datasets currently used for handwritten Chinese character research. The training and test sets are divided in an 8:2 ratio when training the classification and recognition network.

[0081] S12. Preprocessing the dataset: The original dataset has different resolutions, which is not conducive to training the classification and scoring network. Therefore, it is necessary to unify all images to a resolution of 64*64 through operations such as padding and scaling, while preserving their original shape.

[0082] S13. Constructing the Evaluation Dataset: The HWDB dataset mentioned above only contains category information for Chinese characters and can only train the classification and recognition network. Therefore, manual scoring is required to train the scoring network. To reduce errors caused by meticulous manual scoring, this invention labels each set of data as 5 randomly selected images of the same type from HWDB, sorted according to human subjective aesthetics, resulting in a total of 8586 labeled sets. Simultaneously, a standard font for each Chinese character is constructed, which can be standard KaiTi or multiple different fonts. By comparing the character with the standard font, the model learns the scoring differences between different characters, thereby training the score regression prediction network.

[0083] S2. A convolutional neural network is used as the backbone network for extracting features of handwritten Chinese characters, and pre-training is performed using unsupervised contrastive learning techniques.

[0084] Specifically, step S2 includes steps S21-S23:

[0085] S21. Select ResNet18 convolutional neural network to extract features of handwritten Chinese characters. The feature dimension after extraction is 512.

[0086] S22. Perform data augmentation on the HWDB dataset to provide two different perspectives for each image. Specific data augmentation methods include randomly cropping a portion of the original image and resizing it to its original size, randomly applying Gaussian blur to the original image, randomly flipping the original image horizontally with a probability of 0.5, and randomly adjusting the brightness, contrast, and hue of the original image.

[0087] S23. Pre-training: Data augmentation is performed on n images from the same batch to obtain 2n sample data points. Two different viewpoints of the same image are considered positive sample pairs, and the rest are negative sample pairs. These 2n sample data points are input into the skeleton network model, and then mapped using a specific MLP network to obtain their vector representations. Training is then performed using the InfoNCE loss function, expressed as follows:

[0088]

[0089] Where x represents the image data after data augmentation, x + x represents a positive sample. - represents the negative samples of x; f_q represents the skeleton network, a mapping MLP layer, and a prediction MLP layer; f_k represents the skeleton network and a mapping MLP layer. f_k shares the parameters of f_q in a certain proportion and is called the momentum encoder; · represents the similarity calculation method, which is cosine similarity here; τ is the temperature hyperparameter, which is 0.2.

[0090] S3. After the skeleton network, add a classification and recognition network (using the form of MLP), train it using a handwritten Chinese character dataset to obtain a better feature representation of handwritten Chinese characters, and then fix the network parameters.

[0091] Specifically, step S3 includes steps S31-S32:

[0092] S31. To make the classification and recognition results more robust, the input data needs to be randomly transformed, including randomly adding points, interfering with lines, random Gaussian blurring, random binarization, and random transmission transformation.

[0093] S32. The classification results are constrained using the cross-entropy loss function and the center distance loss function. The specific loss function expressions are as follows:

[0094] Loss = Loss CrossEntropy +λLoss Center

[0095]

[0096]

[0097] Where λ is a hyperparameter used to balance the two loss functions, and is set to 0.1; W and b are the trainable model parameters in the classification prediction network; x i y represents the features extracted from image data through a convolutional neural network. i is the category label of image i; m represents the number of images in a batch, and n represents the total number of categories; It is class y i The feature centers are updated as training progresses.

[0098] S4. After the skeleton network, add a score regression prediction network (in the form of MLP) in parallel. Train it using the evaluation dataset and a reasonably effective loss function to obtain an overall score that better reflects human subjective aesthetics for handwritten Chinese characters.

[0099] Specifically, step S4 includes steps S41-S44:

[0100] S41. The activation function of the score regression prediction network is softmax, which classifies handwritten Chinese characters into three categories: excellent, good, and poor. The final scoring formula is:

[0101] score = p(Excellent) * 100 + p(Good) * 50 + p(Poor) * 0

[0102] Where p represents the confidence level of the category to which the handwritten Chinese character belongs.

[0103] S42. Construct the target score required for the regression task: Calculate the L2 distance (extracted from the skeleton network model above) between each handwritten Chinese character and the standard character in HWDB, divide the distance into intervals, and assign a score range to each interval. For example, the first 15% of the distance scores 75-90 points, the middle 70% scores 60-75 points, and the last 15% scores 45-60 points. The closer the distance between the handwritten Chinese characters and the standard characters, the higher the score; the standard characters receive a default full score.

[0104] S43. The input to the network is the feature difference between the handwritten Chinese character and its standard character extracted by the skeleton network, along with its category information. Directly inputting the difference between the two is beneficial for training, while concatenating the category information enables differentiated scoring standards, allowing the network to learn to use different scoring standards for each Chinese character, resulting in more robust performance.

[0105] S44. Using the evaluation dataset, the network's scoring results are constrained by using the triplet loss function and the absolute value loss function, allowing the network to learn the difference between handwritten Chinese characters and their standard characters.

[0106] As an optional approach, step S44 is as follows: The six Chinese characters (including one standard character and five handwritten characters) in each group of the evaluation dataset are denoted as A12345, and triples are constructed as follows: A13, A14, A15, A24, A25, and A35. This interval construction can reduce errors generated during manual annotation. These triples are used to train the network's scoring ability, and the specific loss function expression is as follows:

[0107]

[0108]

[0109] Loss Triplet =max(d(a,p)-d(a,n)+margin,0)

[0110] Where λ is a hyperparameter used to balance the two loss functions, and is set to 0.5; y i Let be the score of image i obtained according to statistical rules, and score(i) be the score of image i obtained through a neural network; 'a' represents the anchor image, which is the standard character in a set of data; 'p' represents the positive sample image, which is the top-ranked handwritten Chinese character in a set of data; 'n' represents the negative sample image, which is the bottom-ranked handwritten Chinese character in a set of data; 'margin' is a hyperparameter used to adjust the score difference, which is set to 15; 'd' represents the difference between the scores obtained by the two images after passing through the scoring network.

[0111] In summary, the method of this embodiment has the following advantages and beneficial effects compared with the prior art:

[0112] This embodiment employs a ResNet18 convolutional neural network as the backbone to extract features of handwritten Chinese characters. It pre-trains the backbone network using appropriate contrastive learning techniques, and then fine-tunes it while training the classification and recognition network. Experiments show that the classification accuracy on the HWDB test set reaches 96.24%, indicating a good feature representation of handwritten Chinese characters. Furthermore, by constructing ranked data from the evaluation dataset and combining triplet loss and absolute value loss functions, the score regression prediction network learns the differences between different handwritten Chinese characters. This allows the algorithm to provide an overall score that better aligns with human subjective aesthetics; specifically, the designed loss function effectively decreases on both the training and test sets.

[0113] like Figure 2As shown, this embodiment also provides an offline handwritten Chinese character scoring method based on the two-dimensional distribution of handwritten Chinese character pixels. It analyzes the two-dimensional distribution of handwritten Chinese characters from different perspectives and compares it with data in a Chinese character standard library to obtain scores for the fullness, symmetry, center of gravity, and compactness of handwritten Chinese characters, providing writers with improvement references. The method specifically includes the following steps:

[0114] A1. To unify the resolution and size of handwritten Chinese characters in the HWDB dataset, cropping, padding, and resizing operations are used to standardize them to 64*64 pixels while preserving their original shape. Then, binarization is performed. This is achieved by calling functions from the OpenCV library in Python to speed up the computation process.

[0115] A2. The fullness feature is represented by a two-dimensional vector, defined as the ratio of the perimeter of the convex hull to the minimum bounding box, f1, and the convexity, f2. The specific calculation formula is as follows:

[0116]

[0117] Among them, P convex P represents the perimeter of the convex hull of the handwritten Chinese character. bounding_box This represents the perimeter of the smallest bounding box of the handwritten Chinese character; the smallest bounding box is a rectangle; C convex Indicates the number of black pixels in the convex hull of the handwritten Chinese character; A convex This represents the area of ​​the convex hull of the handwritten Chinese character.

[0118] A3. The symmetry feature is represented by an eight-dimensional vector f3-f 10 This is used to represent and reflect the distribution of pixels. The specific calculation formula is as follows:

[0119]

[0120]

[0121] Among them, f 2+i During calculation, the handwritten Chinese character is divided into four equal regions according to the constraint box, which is defined as the smallest square that exactly encloses the entire handwritten Chinese character; S i represents the number of black pixels in region i; n represents the number of black pixels constituting the entire Chinese character; f 5+j In the calculation, the convex hull of the handwritten Chinese character is divided into four regions based on its center point; C convex(j) A represents the number of black pixels in the convex hull region j; convex(j) This represents the area of ​​the convex hull region j.

[0122] A4. The centroid location feature is represented by a two-dimensional vector f. 11 -f 12The specific calculation formula is as follows:

[0123]

[0124]

[0125] Where W and H represent the width and height of the handwritten Chinese character image; n represents the number of black pixels that make up the handwritten Chinese character; I i，j It is an indicator function, when the point (x) on the image... i y j When a pixel is black, the value is 1; otherwise, it is 0.

[0126] A5. The compactness feature is represented by a seven-dimensional vector f. 13 -f 19 This represents the normalized average distance between all n black pixels constituting a Chinese character and the center point, as well as the overall distribution of the pixels. The pixel distribution is first represented by dividing all black pixels into 16 regions evenly using three horizontal and three vertical lines, and then using the position coordinates of these six lines. The specific calculation formula is as follows:

[0127]

[0128]

[0129]

[0130] Where W and H represent the width and height of the handwritten Chinese character image; x center y center Indicates the coordinates of the center point; I i，j It is an indicator function, when the point (x) on the image... i y j When a pixel is black, the value is 1; otherwise, it is 0. n represents the number of black pixels constituting the handwritten Chinese character; a represents the side length of the handwritten Chinese character image; x v(i) Represents the row coordinate of the i-th vertical line; y h(j) This represents the column coordinate of the j-th vertical line.

[0131] A6. Based on the formula, handwritten Chinese characters are scored from four perspectives: fullness, symmetry, center of gravity, and compactness. The characteristics of each perspective are represented by a multi-dimensional vector, and the specific scoring formula for each perspective is as follows:

[0132]

[0133] Where f and f standard The feature representation of a handwritten Chinese character and its corresponding standard character from a certain angle is a multi-dimensional vector; ||ffstandard || represents the difference between the two, calculated using the L2 norm; bound represents the range of variation of this feature compared to the standard character, obtained by statistical analysis of data in HWDB, and different Chinese characters will also have differences.

[0134] In summary, the method of this embodiment has the following advantages and beneficial effects compared with the prior art:

[0135] This embodiment of the method extracts features from four aspects of handwritten Chinese characters: fullness, symmetry, center of gravity, and compactness through manual design, and compares them with corresponding standard characters. At the same time, it combines a large amount of data comparison to make the scoring of these four aspects more scientific, and can effectively provide certain improvement guidance for people who need to practice calligraphy.

[0136] This embodiment also provides an offline handwritten Chinese character scoring device, including:

[0137] At least one processor;

[0138] At least one memory for storing at least one program;

[0139] When the at least one program is executed by the at least one processor, the at least one processor implements Figure 1 and / or Figure 2 The method shown.

[0140] This embodiment of the device for scoring offline handwritten Chinese characters can execute the scoring method for offline handwritten Chinese characters provided in the method embodiment of the present invention. It can execute any combination of the implementation steps of the method embodiment and has the corresponding functions and beneficial effects of the method.

[0141] This application also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform... Figure 1 and / or Figure 2 The method shown.

[0142] This embodiment also provides a storage medium storing instructions or programs that can execute the offline handwritten Chinese character scoring method provided in the method embodiment of the present invention. When the instructions or programs are run, any combination of implementation steps of the method embodiment can be executed, and the method has the corresponding functions and beneficial effects.

[0143] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0144] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0145] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0146] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0147] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0148] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0149] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0150] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0151] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A scoring method for offline handwritten Chinese characters, characterized in that, Includes the following steps: Obtain images of the handwritten Chinese characters to be scored; Based on the distribution pattern of Chinese character pixels, the two-dimensional distribution of Chinese character pixels in handwritten Chinese character images is analyzed to obtain various multi-dimensional features. The obtained multi-dimensional features are compared and analyzed with the corresponding Chinese characters in the standard library to obtain scores of handwritten Chinese characters in different dimensions. The various multidimensional features include the fullness, symmetry, center of gravity position, and compactness of handwritten Chinese characters; The specific definitions are as follows: Fullness feature: Represented by a two-dimensional vector, it is the ratio of the perimeter of the convex hull of a handwritten Chinese character to the perimeter of its minimum bounding box. The ratio of the number of black pixels in the convex hull to the area of ​​the convex hull. The score reflects whether the fullness of the handwritten Chinese characters matches the standard characters. The formula is as follows: In the formula, This indicates the perimeter of the convex hull of the handwritten Chinese character; This represents the perimeter of the smallest bounding box of the handwritten Chinese character; This indicates the number of black pixels in the convex hull of the handwritten Chinese character; This represents the area of ​​the convex hull of the handwritten Chinese character; Symmetry characteristic: using an eight-dimensional vector The score is represented by the distribution of black pixels in different regions of a handwritten Chinese character. The score reflects whether the overall uniformity of the handwritten Chinese character is consistent with the standard character. The formula is as follows: In the formula, During the calculation, the handwritten Chinese character is divided into four regions equally according to the constraint box. The constraint box is defined as the smallest square that exactly encloses the entire handwritten Chinese character. Indicates the area The number of black pixels in the image; This indicates the number of black pixels that make up the entire Chinese character; These represent the proportion of black pixels to the total number of pixels in each of the four regions; During the calculation, the convex hull of the handwritten Chinese character is divided into four regions based on the center point; Represents the convex hull region The number of black pixels in the middle; Represents the convex hull region The area; These represent the ratios of the number of black pixels to the area of ​​each of the four regions of the convex hull; Centroid position feature: using a two-dimensional vector The score indicates whether the center of gravity of the handwritten Chinese character is consistent with the standard character. The formula is as follows: In the formula, , This indicates the width and height of the handwritten Chinese character image; This indicates the number of black pixels that make up the handwritten Chinese character; It is an indicator function, when a point on the image... When a pixel is black, the value is 1; otherwise, it is 0. This represents the ratio of the centroid's x-coordinate to the image width. This represents the ratio of the centroid's y-coordinate to the image height. Compactness feature: The score reflects whether the overall compactness of the handwritten Chinese characters is consistent with the standard characters; the compactness feature is represented by a seven-dimensional vector. It means that all the components of Chinese characters are represented. The normalized average distance between each black pixel and the center point, and the overall distribution of the pixels; the pixel distribution is first divided into 16 regions evenly using three horizontal lines and three vertical lines, and then represented by the position coordinates of these six lines; the specific calculation formula is as follows: In the formula, , Represents the coordinates of the center point; It is an indicator function, when a point on the image... When a pixel is black, the value is 1; otherwise, it is 0. This indicates the number of black pixels that make up the handwritten Chinese character; This indicates the side length of the handwritten Chinese character image; Indicates the first The row coordinates of the vertical line; Indicates the first The column coordinates of the vertical lines; This represents the average distance from all black pixels to the center point of the Chinese character. These represent the normalized x-coordinates of the three perpendicular dividing lines. These represent the normalized y-coordinates of the three horizontal dividing lines.

2. The scoring method for offline handwritten Chinese characters according to claim 1, characterized in that, The Chinese character scoring model was trained using the following methods: Collect an offline handwritten Chinese character dataset with category information, and obtain an evaluation dataset based on the offline handwritten Chinese character dataset; A convolutional neural network was used as the backbone network for extracting features of handwritten Chinese characters, and it was pre-trained using an offline handwritten Chinese character dataset. After pre-training, a classification and recognition network is added after the backbone network, trained using an offline handwritten Chinese character dataset, and the parameters of the backbone network are fixed after training. A score regression prediction network is added in parallel after the backbone network, and trained using the evaluation dataset and a preset loss function to obtain the final Chinese character scoring model.

3. The scoring method for offline handwritten Chinese characters according to claim 2, characterized in that, The method of using a convolutional neural network as the backbone network for extracting handwritten Chinese character features, and pre-training using an offline handwritten Chinese character dataset, includes: Data augmentation was performed on the offline handwritten Chinese character dataset so that each image could be viewed from two different perspectives; the two different perspectives of the same image were used as positive sample pairs, and the rest were used as negative sample pairs. The backbone network was trained using the enhanced dataset and the InfoNCE loss function; The expression for the InfoNCE loss function is as follows: In the formula, This represents the image data after data augmentation. + represent Positive samples, - represent Negative samples; This represents a backbone network, a mapping MLP layer, and a prediction MLP layer. Represents a backbone network and a mapping MLP layer; symbol Representative similarity calculation methods, It is a temperature hyperparameter.

4. The scoring method for offline handwritten Chinese characters according to claim 2, characterized in that, After pre-training, a classification and recognition network is added after the backbone network, trained using an offline handwritten Chinese character dataset. After training, the parameters of the backbone network are fixed, including: Random transformation processing is performed on the data in the offline handwritten Chinese character dataset; The classification results are constrained using cross-entropy loss and center distance loss functions: in, It's a hyperparameter; These are the trainable model parameters in a classification and prediction network. It is the first The weight of the actual Chinese character category in the image. It is the first The bias of the actual Chinese character category in the image. It is the first The weight of each Chinese character category, It is the first The bias of each Chinese character category; This represents the features extracted from image data using a convolutional neural network. It's a picture. Category tags; This indicates the number of images in a batch. Indicates the total number of categories; Is class The feature center.

5. The scoring method for offline handwritten Chinese characters according to claim 2, characterized in that, The step of adding a score regression prediction network in parallel after the backbone network, and training it using an evaluation dataset and a preset loss function, includes: The activation function of the score regression prediction network is the softmax function, used to classify handwritten Chinese characters into three categories: excellent, good, and poor; a score is calculated based on the classification results. ; Specifically, the triplet loss function and the absolute value loss function are used to constrain the scoring results of the score regression prediction network: In the formula, It's a hyperparameter; For image i The score derived from statistical patterns For image i The score obtained through a neural network; The anchor image represents a set of standard characters within a dataset. p This represents a positive sample image, which is the top-ranking handwritten Chinese character in a set of data. The negative sample image represents the handwritten Chinese character that ranks last in a set of data. For hyperparameters, It is the difference in scores obtained by two images after they have been processed by a scoring network.

6. The scoring method for offline handwritten Chinese characters according to claim 1, characterized in that, The formulas for calculating scores on different dimensions are as follows: in, and A representation of the characteristics of handwritten Chinese characters and their corresponding standard characters from a certain angle; The difference between the two is expressed using the L2 norm. This indicates the range of variation of the feature compared to the standard character.

7. A scoring device for offline handwritten Chinese characters, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1-6.

8. A computer-readable storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to perform the method as described in any one of claims 1-6.

Images