An automatic method, system, device and medium for reviewing the quality of English writing

Abstract

The application discloses an automatic reviewing method, system and device for English writing quality and a medium, and relates to the technical field of writing quality reviewing. The method comprises the following steps: obtaining a target calligraphy image; the target calligraphy image is a scanned image of a calligraphy work to be reviewed; a writing quality reviewing model is used to review the target calligraphy image, so that a score and an award of the calligraphy work to be reviewed are obtained; the score comprises an index score and a total score of the work; the index score comprises a horizontal angle consistency score, an inclination angle consistency score, a letter size consistency score, a character spacing consistency score and a word spacing consistency score; wherein the writing quality reviewing model comprises a scoring mode selection module, a scoring module and a post-processing module; the scoring module comprises a rough scoring network and a fine scoring network. The application can realize objective and accurate evaluation of the overall appearance degree of an English handwritten work.

Description

Technical Field

[0001] This invention relates to the field of handwriting quality assessment technology, and in particular to an automatic assessment method, system, device and medium for English handwriting quality. Background Technology

[0002] Handwriting is an essential skill for students, significantly impacting the aesthetics of their work and reflecting their cultural literacy. The beauty of English handwriting is influenced not only by the aesthetics of individual letters but, more importantly, by overall neatness. Many schools offer dedicated handwriting courses, and some organizations hold English calligraphy competitions to encourage students to improve their skills. Improving handwriting requires extensive practice and guidance from professional teachers. However, teachers often cannot provide timely feedback to a large number of students simultaneously, significantly compromising teaching quality. Utilizing artificial intelligence to assess handwriting quality and provide timely feedback can greatly improve teaching efficiency and facilitate self-improvement for users.

[0003] Artificial intelligence has made significant progress and been successfully applied in the field of education, and some methods for assessing handwriting quality have also developed to a certain extent. However, most of these methods are focused on assessing individual characters and assisting teaching, and there is still no good way to evaluate the overall aesthetics.

[0004] Patent application number 202210799266.1 discloses a method and system for intelligent scoring of Chinese character writing based on structural attributes and deep neural networks. It segments Chinese characters based on their structural attributes, constructs a dataset, and has professionals score ten attributes to generate a dataset. A deep network is then trained based on this dataset to score the aesthetics of individual Chinese characters. However, this method only evaluates individual characters, ignoring the overall aesthetics of the writing and providing no guidance.

[0005] Patent application number 202110100196.1 discloses a method for intelligent evaluation and comment generation of calligraphy copying based on machine learning. First, the image is preprocessed. Then, scores are generated for character shape matching overlap, structural matching overlap, projection similarity, and key point similarity of the calligraphy font. Finally, a comprehensive score is obtained through machine learning regression, and guidance is provided based on the regression score. This method relies on the calligraphy copybook, evaluating the aesthetics of the writing based on the degree of overlap with the copybook. This dependence on the copybook severely limits its application scenarios, and it can only evaluate individual characters. Furthermore, the overall neatness of the written work has a greater impact on the overall aesthetics. Summary of the Invention

[0006] The purpose of this invention is to provide an automatic evaluation method, system, device, and medium for English handwriting quality, so as to evaluate the overall aesthetics of English handwritten works.

[0007] To achieve the above objectives, the present invention provides the following solution:

[0008] An automatic method for evaluating the quality of English handwriting includes:

[0009] Acquire the target calligraphy image; the target calligraphy image is a scanned image of the calligraphy work to be reviewed;

[0010] The target calligraphy image is evaluated using a writing quality assessment model to obtain a score and award for the calligraphy work to be evaluated; the score includes: indicator score and total score; the indicator score includes: horizontal angle consistency score, tilt angle consistency score, letter size consistency score, character spacing consistency score, and word spacing consistency score.

[0011] The handwriting quality assessment model includes: a scoring mode selection module, a scoring module, and a post-processing module; the scoring module includes: a coarse scoring network and a fine scoring network.

[0012] The scoring mode selection module is used to perform binary classification on the target calligraphy image to obtain a classification result; the classification result includes: the calligraphy work to be evaluated is a work that is definitely not eligible for an award or the calligraphy work to be evaluated is a candidate for an award;

[0013] The scoring module is used to score the target calligraphy image based on the classification results, and obtain the index score of each line of the calligraphy work to be evaluated; wherein, the coarse scoring network is used to score the works that are definitely not eligible for awards, and the fine scoring network is used to score the works that are candidate for awards.

[0014] The post-processing module is used to determine the score and award of the calligraphy work to be reviewed based on the index score of each line of the calligraphy work to be reviewed.

[0015] Optionally, the method for determining the scoring mode selection module and the scoring module in the writing quality assessment model specifically includes:

[0016] Obtain a calligraphy work dataset; the calligraphy work dataset includes a strongly labeled sample dataset and a weakly labeled sample dataset; the strongly labeled sample dataset includes several sample calligraphy images and their corresponding scores and awards; the weakly labeled sample dataset includes several sample calligraphy images and their corresponding awards; the sample calligraphy images are scanned images of the reviewed calligraphy works;

[0017] Based on the score cutoff, the strongly labeled sample dataset is divided into a dataset of works that do not win awards and a dataset of works that are candidates for awards.

[0018] A first convolutional neural network is trained using the dataset of works that are definitely not eligible for awards and the dataset of works that are candidate for awards to obtain a scoring mode selection module.

[0019] A second convolutional neural network was trained using the dataset of works that were definitely not eligible for awards to obtain a coarse scoring network.

[0020] A second convolutional neural network was trained using the dataset of award-winning candidate works to obtain the initial scoring network;

[0021] The weakly labeled sample dataset is supplemented with pseudo-labels using the initial scoring network to obtain a supplemented labeled sample dataset.

[0022] The initial scoring network is trained using the strongly labeled sample dataset and the supplementary labeled sample dataset to obtain the refined scoring network.

[0023] Optionally, the weakly labeled sample dataset is supplemented with pseudo-labels using the initial scoring network to obtain a supplemented labeled sample dataset, specifically including:

[0024] Based on the range of the total score of the works, the dataset of award-winning candidate works is divided into a first training set, a second training set, and a third training set. The total score of the works corresponding to the sample calligraphy images in the first training set is greater than the first score value and less than or equal to the full score value. The total score of the works corresponding to the sample calligraphy images in the second training set is greater than the second score value and less than or equal to the third score value. The total score of the works corresponding to the sample calligraphy images in the third training set is greater than the score threshold and less than or equal to the first score value. Among them, the score threshold value is less than the second score value, the second score value is less than the first score value, the first score value is less than the third score value, and the third score value is less than the full score value.

[0025] The initial scoring network is trained using the first training set to obtain a first teacher model; the initial scoring network is trained using the second training set to obtain a second teacher model; and the initial scoring network is trained using the third training set to obtain a third teacher model.

[0026] Based on the award level, the weakly labeled sample dataset is divided into a high-award sample dataset and a low-award sample dataset;

[0027] The first teacher model is used to perform a first round of pseudo-label supplementation and confidence calculation on the sample calligraphy images in the high award sample dataset to obtain the first supplemented dataset and the first candidate dataset.

[0028] The third teacher model is used to perform a first round of pseudo-label supplementation and confidence calculation on the sample calligraphy images in the low award sample dataset to obtain a second supplemented dataset and a second candidate dataset.

[0029] The second teacher model is used to perform a second round of pseudo-label supplementation and confidence calculation on the sample calligraphy images in the first candidate dataset and the second candidate dataset to obtain the third supplemented dataset and the abandoned dataset.

[0030] The first supplementary dataset, the second supplementary dataset, and the third supplementary dataset are determined as the supplementary label sample dataset.

[0031] Optionally, the initial scoring network is trained using the strongly labeled sample dataset and the supplementary labeled sample dataset to obtain a refined scoring network, specifically including:

[0032] The sample calligraphy images from the strongly labeled sample dataset are input into the initial scoring network to obtain the prediction results of the strongly labeled data; the strongly labeled data are the sample calligraphy images from the strongly labeled sample dataset.

[0033] The sample calligraphy images from the supplementary label sample dataset are input into the initial scoring network to obtain the prediction results of the weak label data; the weak label data are the sample calligraphy images from the supplementary label sample dataset.

[0034] The target loss function is determined based on the prediction results and corresponding real labels of the strongly labeled data, the prediction results and corresponding pseudo labels of the weakly labeled data, and the confidence level; wherein, the real label is the score corresponding to the sample calligraphy image in the strongly labeled sample dataset; the pseudo label is the score corresponding to the sample calligraphy image in the supplementary labeled sample dataset; and the confidence level is determined based on the probability distribution of the correspondence between score intervals and weak labels and the probability distribution of scores within each interval.

[0035] With the convergence of the target loss function as the objective, the initial scoring network is trained to obtain a refined scoring network.

[0036] Optionally, the expression for the target loss function is:

[0037]

[0038] Among them, L m The third loss value, Ps p Gs is the prediction result for the p-th strongly labeled data. p Let Pw be the true label of the p-th strongly labeled data. q For the prediction result of the q-th weak label data, Gwq Let p be the pseudo-label of the q-th weakly labeled data, p be the index of the sample calligraphy image in the strong-labeled sample dataset, P be the total number of sample calligraphy images in the strong-labeled sample dataset, q be the index of the sample calligraphy image in the supplementary-labeled sample dataset, and Q be the total number of sample calligraphy images in the supplementary-labeled sample dataset. conf2 Y n θ m ,X n For pseudo-tag Gw q confidence level, Y n θ represents the score of the nth weakly labeled data. m Let X represent the m-th teacher model. n Let L1 represent the nth weakly labeled data, and L1() represent the L1 loss function.

[0039] Optionally, the confidence level of the pseudo-label is calculated using the following formula:

[0040] p conf2 Y n θ m ,X n =p conf1 δ l θ m ,X n *p d Y n δ l ;

[0041]

[0042] Where, p conf1 δ l θ m ,X n Let p be the probability distribution of the correspondence between the score intervals of the pseudo-labels and the weak labels. d Y n δ l Let δ be the probability distribution of scores within the score interval where the pseudo-label is located. l Let d represent the l-th fraction interval. l This represents the length of the l-th fractional interval.

[0043] Optionally, it also includes:

[0044] The comments on the calligraphy works to be reviewed are determined based on the comment database and the scores of the calligraphy works to be reviewed; the comment database includes comments corresponding to different indicators in different score ranges; the indicators include: horizontal angle consistency, tilt angle consistency, letter size consistency, character spacing consistency, and word spacing consistency.

[0045] An automatic evaluation system for English handwriting quality includes:

[0046] The acquisition module is used to acquire a target calligraphy image; the target calligraphy image is a scanned image of the calligraphy work to be reviewed.

[0047] The evaluation module uses a writing quality evaluation model to evaluate the target calligraphy image and obtain the score and award of the calligraphy work to be evaluated. The score includes: indicator score and total score of the work. The indicator score includes: horizontal angle consistency score, tilt angle consistency score, letter size consistency score, character spacing consistency score and word spacing consistency score.

[0048] The handwriting quality assessment model includes: a scoring mode selection module, a scoring module, and a post-processing module; the scoring module includes: a coarse scoring network and a fine scoring network.

[0049] The scoring mode selection module is used to perform binary classification on the target calligraphy image to obtain a classification result; the classification result includes: the calligraphy work to be evaluated is a work that is definitely not eligible for an award or the calligraphy work to be evaluated is a candidate for an award;

[0050] The scoring module is used to score the target calligraphy image based on the classification results, and obtain the index score of each line of the calligraphy work to be evaluated; wherein, the coarse scoring network is used to score the works that are definitely not eligible for awards, and the fine scoring network is used to score the works that are candidate for awards.

[0051] The post-processing module is used to determine the score and award of the calligraphy work to be reviewed based on the index score of each line of the calligraphy work to be reviewed.

[0052] An electronic device includes a memory and a processor, the memory storing a computer program, and the processor running the computer program to enable the electronic device to perform the above-described automatic English handwriting quality assessment method.

[0053] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for automatically evaluating the quality of English handwriting.

[0054] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0055] The automatic evaluation method for English handwriting quality provided by this invention uses a handwriting quality evaluation model to evaluate target calligraphy images. First, a scoring mode selection module performs binary classification on the target calligraphy image to obtain classification results. Then, based on the classification results, a coarse scoring network or a fine scoring network is selected to score the target calligraphy image, obtaining an index score for each line of the calligraphy work to be evaluated. Finally, a post-processing module determines the score and award of the calligraphy work to be evaluated. This method can evaluate each line of the calligraphy work separately from five aspects: horizontal angle consistency, tilt angle consistency, letter size consistency, character spacing consistency, and word spacing consistency, and give a total score reflecting the overall aesthetics of the work. This achieves an objective and accurate evaluation of the overall aesthetics of English handwritten works. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A flowchart of the automatic evaluation method for English handwriting quality provided by the present invention;

[0058] Figure 2 A flowchart illustrating the automatic evaluation method for English handwriting quality provided by this invention;

[0059] Figure 3 This is a schematic diagram of the preprocessing process for scanned images provided by the present invention;

[0060] Figure 4 This is a schematic diagram of the structure of the first convolutional neural network provided by the present invention;

[0061] Figure 5 This is a schematic diagram of the process for overlaying and cropping preprocessed scanned images provided by the present invention.

[0062] Figure 6 This is a schematic diagram of the structure of the second convolutional neural network provided by the present invention;

[0063] Figure 7 The training flowchart for the multi-teacher model provided by this invention. Detailed Implementation

[0064] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0065] The purpose of this invention is to provide an automatic evaluation method, system, device, and medium for English handwriting quality, so as to evaluate the overall aesthetics of English handwritten works.

[0066] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0067] Example 1

[0068] This invention provides an automatic method for evaluating the quality of English handwriting. For example... Figure 1 As shown, the automatic review method includes:

[0069] Step 110: Obtain the target calligraphy image; the target calligraphy image is a scanned image of the calligraphy work to be reviewed.

[0070] Step 120: Use a writing quality evaluation model to evaluate the target calligraphy image and obtain the score and award of the calligraphy work to be evaluated; the score includes: indicator score and total score of the work; the indicator score includes: horizontal angle consistency score, tilt angle consistency score, letter size consistency score, character spacing consistency score and word spacing consistency score (that is, the indicators include: horizontal angle consistency, tilt angle consistency, letter size consistency, character spacing consistency and word spacing consistency).

[0071] The writing quality evaluation model includes a scoring mode selection module, a scoring module, and a post-processing module. The scoring module includes a coarse scoring network and a fine scoring network. The scoring mode selection module performs binary classification on the target calligraphy image to obtain a classification result. The classification result includes either the calligraphy work to be evaluated being a work that is definitely not eligible for an award, or the calligraphy work to be evaluated being a candidate for an award. The scoring module scores the target calligraphy image based on the classification result, obtaining an index score for each line of the calligraphy work to be evaluated. The coarse scoring network is used to score works that are definitely not eligible for an award, and the fine scoring network is used to score candidate works for an award. The post-processing module determines the score and award of the calligraphy work to be evaluated based on the index scores of each line of the calligraphy work.

[0072] Furthermore, this automated grading method also includes:

[0073] Step 130: Determine the comments for the calligraphy work to be reviewed based on the comment database and the score of the calligraphy work to be reviewed; the comment database includes comments corresponding to different indicators in different score ranges.

[0074] The specific methods for determining the scoring mode selection module and the scoring module in the writing quality assessment model include:

[0075] Obtain a calligraphy work dataset; the calligraphy work dataset includes a strongly labeled sample dataset and a weakly labeled sample dataset; the strongly labeled sample dataset includes several sample calligraphy images and their corresponding scores and awards; the weakly labeled sample dataset includes several sample calligraphy images and their corresponding awards; the sample calligraphy images are scanned images of calligraphy works that have been reviewed.

[0076] Based on the score cutoff, the strongly labeled sample dataset is divided into a dataset of works that do not win awards and a dataset of works that are candidates for awards.

[0077] A first convolutional neural network is trained using the dataset of works that are certain to not win awards and the dataset of candidate works that will win awards to obtain a scoring mode selection module. Preferably, the first convolutional neural network is a binary classification convolutional neural network; the binary classification convolutional neural network includes 6 convolutional layers and 3 fully connected layers connected in sequence.

[0078] A second convolutional neural network is trained using the dataset of works that are certain to not win awards to obtain a coarse scoring network. An initial scoring network is then trained using the dataset of candidate winning works. Preferably, the second convolutional neural network includes a front-end network and five branch networks; each branch network is connected to the front-end network; the front-end network includes four sequentially connected convolutional layers; and each branch network includes eight sequentially connected convolutional layers and three fully connected layers.

[0079] The weakly labeled sample dataset is supplemented with pseudo-labels using the initial scoring network to obtain a supplemented labeled sample dataset.

[0080] The initial scoring network is trained using the strongly labeled sample dataset and the supplementary labeled sample dataset to obtain the refined scoring network.

[0081] Furthermore, the method for determining the scoring mode selection module and the scoring module in the writing quality evaluation model also includes: performing three-line overlapping cropping on the sample calligraphy images in the calligraphy work dataset, and processing the scores and awards corresponding to the sample calligraphy images.

[0082] In the above steps, the weakly labeled sample dataset is supplemented with pseudo-labels using the initial scoring network to obtain a supplemented labeled sample dataset, specifically including:

[0083] Based on the range of the total score of the works, the dataset of award-winning candidate works is divided into a first training set, a second training set, and a third training set. The total score of the works corresponding to the sample calligraphy images in the first training set is greater than the first score value and less than or equal to the full score value. The total score of the works corresponding to the sample calligraphy images in the second training set is greater than the second score value and less than or equal to the third score value. The total score of the works corresponding to the sample calligraphy images in the third training set is greater than the score threshold and less than or equal to the first score value. Among them, the score threshold value is less than the second score value, the second score value is less than the first score value, the first score value is less than the third score value, and the third score value is less than the full score value.

[0084] The initial scoring network is trained using the first training set to obtain a first teacher model; the initial scoring network is trained using the second training set to obtain a second teacher model; and the initial scoring network is trained using the third training set to obtain a third teacher model.

[0085] Based on the award level, the weakly labeled sample dataset is divided into a high-award sample dataset and a low-award sample dataset.

[0086] The first teacher model is used to perform a first round of pseudo-label supplementation and confidence calculation on the sample calligraphy images in the high-award sample dataset to obtain the first supplemented dataset and the first candidate dataset.

[0087] The third teacher model is used to perform the first round of pseudo-label supplementation and confidence calculation on the sample calligraphy images in the low-award sample dataset to obtain the second supplemented dataset and the second candidate dataset.

[0088] The second teacher model is used to perform a second round of pseudo-label supplementation and confidence calculation on the sample calligraphy images in the first candidate dataset and the second candidate dataset to obtain the third supplemented dataset and the abandoned dataset.

[0089] The first supplementary dataset, the second supplementary dataset, and the third supplementary dataset are determined as the supplementary label sample dataset.

[0090] Furthermore, the initial scoring network is trained using the strongly labeled sample dataset and the supplementary labeled sample dataset to obtain a refined scoring network, specifically including:

[0091] The sample calligraphy images from the strongly labeled sample dataset are input into the initial scoring network to obtain the prediction results of the strongly labeled data; the strongly labeled data are the sample calligraphy images from the strongly labeled sample dataset.

[0092] The sample calligraphy images from the supplementary label sample dataset are input into the initial scoring network to obtain the prediction results of the weak label data; the weak label data are the sample calligraphy images from the supplementary label sample dataset.

[0093] The target loss function is determined based on the prediction results and corresponding real labels of the strong-label data, the prediction results and corresponding pseudo labels of the weak-label data, and the confidence level; wherein, the real label is the score corresponding to the sample calligraphy image in the strong-label sample dataset; the pseudo label is the score corresponding to the sample calligraphy image in the supplementary label sample dataset; and the confidence level is determined based on the probability distribution of the correspondence between score intervals and weak labels and the probability distribution of scores within each interval.

[0094] With the convergence of the target loss function as the objective, the initial scoring network is trained to obtain a refined scoring network.

[0095] The following is combined Figure 2 The above-mentioned automatic evaluation method will be explained in detail.

[0096] Step S1: Obtain scanned images of calligraphy works and preprocess them. The image preprocessing process includes correcting the text and cropping out the handwritten calligraphy portion. A professional calligraphy teacher then scores each line of the calligraphy work according to five indicators: horizontal angle consistency, tilt angle consistency, letter size consistency, letter spacing consistency, and word spacing consistency. These scores serve as labels for the dataset.

[0097] Preferably, step S1 specifically includes:

[0098] Step S101: Use Hough transform to determine the four endpoints of the handwritten calligraphy portion, then use the coordinates of these four endpoints for correction and cropping to obtain a corrected image containing only the handwritten calligraphy portion. See [link to image]. Figure 3 .

[0099] Step S102: The corrected image is given to a professional calligraphy teacher who scores the consistency of horizontal angle, tilt angle, letter size, letter spacing, and word spacing for each line. In this embodiment, each indicator has a maximum score of 10 points, and the total maximum score for all indicators is 50 points. The scores of the five indicators for each line serve as the original labels for the dataset. After the above processing, the calligraphy work dataset is obtained.

[0100] In practical applications, in step S102, 585 data sets with scores (strong labels) and original award categories (weak labels) and 1862 data sets with only awards (weak labels) are obtained. Since the data used in this embodiment are entries from the 2019 calligraphy competition, and there was no unified scoring standard in 2019, the awards were all given based on the judges' first impressions. Therefore, the award information is not completely accurate, but it can still provide a general idea of ​​the actual scores.

[0101] Step S2: Construct a binary classification network for the scoring mode selection module to classify calligraphy works. Specifically, a binary classification convolutional neural network consisting of 6 convolutional layers and 3 fully connected layers is constructed as the scoring mode selection module. The preprocessed images and processed labels from step S1 are used as the dataset and divided into training and test sets. The network is trained using the cross-entropy loss of binary classification as the loss function. This module classifies works into those that are definitely not eligible for awards and those that are candidates for awards.

[0102] Preferably, step S2 specifically includes:

[0103] Step S201: First, a first convolutional neural network consisting of 6 convolutional layers and 3 fully connected layers is constructed to perform 128x downsampling; this first convolutional neural network is specifically a binary classification convolutional neural network, the structure of which is described in [link to relevant documentation]. Figure 4 .

[0104] Step S202: For the dataset obtained in step S1, first calculate the score of the entire work. Let the image have N rows, and the score of the entire image be S. f s i,j Let be the score of the i-th indicator in the j-th row. Then the score of the entire work satisfies the following formula.

[0105]

[0106] Step S203: After step S202, the total score for each image has been obtained. Let the score line be S. d If S f >S d Mark this image as 1. If S f d The images were labeled as 0, thus turning continuous scores into binary labels. The 585 strongly labeled data points were then divided into 470 training sets and 115 test sets.

[0107] Step S204: Train the binary classification convolutional neural network built in step S201 using the training set. The loss function satisfies the following formula, where L1 is the first loss value, y i For real-world labeling, ​The prediction labels are used. After the loss function converges, the scoring mode selection module is obtained. In this embodiment, the scoring mode selection module can be used to divide the data in the test set into 79 award-winning candidate works and 36 works that are definitely not going to win an award.

[0108]

[0109] Step S3: The cropping module crops the entire image into three overlapping rows of images, and processes the labels from step S1 (specifically, it averages the labels of each row) to generate the dataset for the scoring module.

[0110] Specifically, for the image and label obtained in step S1, a three-row overlay cropping is performed to obtain the input for the scoring module, such that each cropped image contains the three rows of the original image. Therefore, for an image, each row appears three times. See [link to relevant documentation]. Figure 5 The cropped label G satisfies the following formula, where S i The score is for the i-th indicator.

[0111]

[0112] G = [S1, S2, S3, S4, S5]

[0113] In step S4, the scoring module is divided into a fine-grained scoring network and a coarse-grained scoring network. The two networks have the same structure and are trained on different datasets. They are used to regress scores on the works classified as award candidates and those definitely not deserving of an award in step S2, respectively. After cropping in step S3, the three rows of images are used as the input dataset for the scoring module, and the scores of the labels are regressed.

[0114] Furthermore, step S4 specifically includes:

[0115] Step S401: Construct a second convolutional neural network consisting of a front-end network and five branch networks. The front-end network has four convolutional layers to perform 8x downsampling. The branch networks have the same structure, each consisting of eight convolutional layers to perform 16x downsampling. Each branch network is finally connected to three fully connected layers. The structure of the second convolutional neural network is shown below. Figure 6 The five branch networks are used to simultaneously regress the scores of five indicators, with each branch network responsible for the regression of one indicator.

[0116] In step S402, the training set divided in step S203 is grouped into two groups: one for data with total scores above the cutoff score and another for data with total scores below (including) the cutoff score. A fine-grained scoring network and a coarse-grained scoring network are trained on each group. The 79 works predicted as award candidates and the 36 works predicted not to win in step S2 are used as the test sets for the two networks. The datasets are then cropped as in step S3, and labels are calculated. In this embodiment, 38 points is used as the cutoff score.

[0117] Step S403: Using the cropped data with scores above 38 as the training set, and the 79 works predicted as award candidates in step S2 as the test set, train the fine-grained scoring network. Using the cropped data with scores below 39 as the training set, and the 36 works predicted as definitely not awardable in step S2 as the test set, train the coarse-grained scoring network. The loss function is as follows, where L2 represents the second loss value, P... d G represents the predicted score of the d-th sample. d Let d represent the true score of the d-th sample, L1loss represent the L1 loss function, and D be the total number of samples.

[0118]

[0119] In step S404, after training on different training sets, two networks with the same network structure (but different parameters) are obtained. The fine scoring network is used to regress the scores of the five indicators of the works predicted as award candidates in step S2, and the coarse scoring network is used to regress the scores of the five indicators of the works predicted as definitely not to win an award in step S2.

[0120] Step S5: Due to the limited availability of strongly labeled data, the fine-grained scoring network is trained using a combination of strongly labeled and weakly labeled data. In this embodiment, the strongly labeled data is divided to train three teacher models separately. The weakly labeled data undergoes data cleaning and pseudo-label addition. The confidence level of the pseudo-labels is calculated using sampling statistics, and this confidence level is multiplied as a coefficient in the loss function. The model is then trained together with the strongly labeled data. After training, the model is fine-tuned using the strongly labeled data. This training method improves the regression accuracy of the fine-grained scoring network.

[0121] like Figure 7 As shown, step S5 specifically includes:

[0122] Step S501: Divide the strongly labeled data into three sub-training sets: data with a total score of 43-50 points as sub-training set 1, data with a total score of 41-47 points as sub-training set 2, and data with a total score of 38-43 points as sub-training set 3. Train the initial fine scoring network (specifically the initial scoring network) to obtain teacher model 1, teacher model 2, and teacher model 3.

[0123] Step S502: Sample 585 strongly labeled data points and divide the score intervals into five intervals: (0,40], (40,42], (42,44], (44,46], and (46,50]. These intervals correspond to five weak labels (i.e., award level): no prize (i.e., fifth prize), fourth prize, third prize, second prize, and first prize. The three teacher models trained in step S501 are used to predict the weakly labeled data, and the prediction results are used as pseudo-labels. If the score interval corresponding to the pseudo-label is adjacent to the original weak label, it is considered an acceptable pseudo-label, and the frequency of its correspondence is used as the confidence level; otherwise, it is considered an unacceptable pseudo-label. The probability distribution of the correspondence between score intervals and weak labels is shown in Table 1.

[0124] Table 1. Probability distribution of the correspondence between score intervals and weak labels.

[0125]

[0126] Step S503: Use Teacher Model 1 and Teacher Model 3 to supplement pseudo-labels for the data with weak labels of first and second prizes and third and fourth prizes, respectively. The pseudo-label supplementation method satisfies Gw k =f(θ) m ,X n ), where Gw k Let p represent the pseudo-label of the k-th weakly labeled data. If an acceptable pseudo-label is obtained as described in step S502, then this data is retained (i.e., the first supplementary dataset and the second supplementary dataset are obtained), and the frequency of its corresponding relationship is used as the confidence level (i.e., the first confidence level) of the correspondence between score and award. conf1 δ l θ m ,X n , where δ l Let θ represent the l-th fraction interval. m Let X represent the m-th teacher model. n Let n represent the nth weakly labeled data. For weakly labeled data that did not receive a pseudo-label in the first round of pseudo-label supplementation (i.e., the first and second candidate datasets), pseudo-labels are supplemented again using Teacher Model 2. If an acceptable pseudo-label can be obtained, the data is retained (i.e., the third supplemented dataset is obtained) and the pseudo-label is used as its label. For data that still cannot receive an acceptable pseudo-label, it is discarded (i.e., it is discarded dataset).

[0127] Step S504, further calculate the confidence level of the pseudo-label, p conf1 δ l θ m ,X nThis reflects the probability that the false label falls within the interval. Within each interval, the distribution of scores is approximately uniform, which is assumed to conform to a uniform distribution. The probability distribution of scores within each interval is p. d Y n δ l p d Y n δ l The score confidence level (i.e., second confidence level) of the pseudo-label p conf2 Y n θ m ,X n If the following equation is satisfied, Y n d represents the score of the nth weakly labeled data. l This represents the length of the l-th fractional interval.

[0128]

[0129] p conf2 Y n θ m ,X n =p conf1 δ l θ m ,X n *p d Y n δ l

[0130] Step S505: The confidence scores of the pseudo-labels obtained in step S504 are considered to be 1. This confidence score is then multiplied by the loss function. Strong and weak labels are mixed together to train the accurate scoring network, improving its regression accuracy. The test set is still set with 79 award-winning candidate works. The loss function satisfies the following formula: Let there be P strong-labeled data points and Q weak-labeled data points, where L... m The third loss value, Ps p Gs is the prediction result for the p-th strongly labeled data. p Let Pw be the true label of the p-th strongly labeled data. q For the prediction result of the q-th weak label data, Gw q It is the pseudo-label of the q-th weakly labeled data.

[0131]

[0132] Step S506: Once the loss function converges, the final refined scoring network is obtained.

[0133] Step S6: After the input image passes through the scoring mode selection module and the scoring module, classification information and score regression information are obtained. These two pieces of information are input into the post-processing module to output the award of the work, the score of each indicator in each row and the total score of the work.

[0134] Step S6 specifically includes:

[0135] Step S601: First, determine whether the data was determined to be definitely not eligible for a prize in step S2. If it is determined to be definitely not eligible for a prize, output it as "not eligible for a prize" directly, and use a coarse scoring network to determine the score P for each indicator in each row. u,v , represents the score of the vth indicator in the uth row of the calligraphy work to be reviewed.

[0136] Step S602: For the award-winning candidate works, the score SP of the v-th indicator is calculated using a fine-grained scoring network. v The overall image prediction score Sp total The following formula must be satisfied, where U is the number of lines in the calligraphy work to be evaluated, and there are a total of V indicators. In this embodiment, the value of V is 5.

[0137]

[0138]

[0139] Step S603, Sp of the works predicted as score candidates total The rankings are determined, and the awards each award should receive are calculated based on the winning rate of each award.

[0140] Step S7: Based on the score of each indicator, the final comments are generated and fed back to the user after corresponding with the comments.

[0141] Step S7 specifically includes:

[0142] Step S701: For each calligraphy work to be reviewed, the scores SP for each indicator were obtained in steps S601 and S602. v Since 100 points is generally used as the maximum score in actual scoring, but in this embodiment, the maximum score for each indicator is 10 points, and the maximum score for the five indicators is 50 points, it is necessary to multiply the score of each indicator by two so that the maximum score for each indicator reaches 20 points and the maximum score for the five indicators reaches 100 points, thus corresponding to the score value in the comment database.

[0143] Step S702: Locate the comments corresponding to the score rate intervals of each indicator in the comment database and provide feedback to the user. The comment database, which contains the comment tables for different indicators across different score rate intervals, is shown in Table 2.

[0144] Table 2. Comments for different indicators in different score ranges

[0145]

[0146] Example 2

[0147] To implement the method corresponding to Embodiment 1 above and achieve the corresponding functions and technical effects, an automatic evaluation system for English handwriting quality is provided below. This automatic evaluation system includes:

[0148] The acquisition module is used to acquire the target calligraphy image; the target calligraphy image is a scanned image of the calligraphy work to be reviewed.

[0149] The evaluation module uses a writing quality evaluation model to evaluate the target calligraphy image and obtain the score and award of the calligraphy work to be evaluated. The score includes: indicator score and total score of the work. The indicator score includes: horizontal angle consistency score, tilt angle consistency score, letter size consistency score, character spacing consistency score and word spacing consistency score.

[0150] The handwriting quality assessment model includes: a scoring mode selection module, a scoring module, and a post-processing module; the scoring module includes: a coarse scoring network and a fine scoring network.

[0151] The scoring mode selection module is used to perform binary classification on the target calligraphy image to obtain the classification result; the classification result includes: the calligraphy work to be evaluated is a work that is definitely not eligible for an award or the calligraphy work to be evaluated is a candidate for an award.

[0152] The scoring module is used to score the target calligraphy image based on the classification results, and obtain the index score of each line of the calligraphy work to be evaluated; wherein, the coarse scoring network is used to score the works that are definitely not eligible for awards, and the fine scoring network is used to score the works that are candidates for awards.

[0153] The post-processing module is used to determine the score and award of the calligraphy work to be reviewed based on the index score of each line of the calligraphy work to be reviewed.

[0154] Example 3

[0155] This invention also provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor runs the computer program to enable the electronic device to perform the automatic English handwriting quality assessment method described in Embodiment 1. The electronic device may be a server.

[0156] In addition, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the automatic evaluation method for English handwriting quality in Embodiment 1.

[0157] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0158] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An automatic method for evaluating the quality of English handwriting, characterized in that, include: Acquire the target calligraphy image; The target calligraphy image is a scanned image of the calligraphy work to be reviewed; The target calligraphy image is evaluated using a writing quality assessment model to obtain the score and award of the calligraphy work to be evaluated; The scores include: indicator scores and total work scores; the indicator scores include: horizontal angle consistency score, tilt angle consistency score, letter size consistency score, character spacing consistency score, and word spacing consistency score. The handwriting quality assessment model includes: a scoring mode selection module, a scoring module, and a post-processing module; the scoring module includes: a coarse scoring network and a fine scoring network. The scoring mode selection module is used to perform binary classification on the target calligraphy image to obtain a classification result; the classification result includes: the calligraphy work to be evaluated is a work that is definitely not eligible for an award or the calligraphy work to be evaluated is a candidate for an award; The scoring module is used to score the target calligraphy image based on the classification results, and obtain the index score of each line of the calligraphy work to be evaluated; wherein, the coarse scoring network is used to score the works that are definitely not eligible for awards, and the fine scoring network is used to score the works that are candidate for awards. The post-processing module is used to determine the score and award of the calligraphy work to be reviewed based on the index score of each line of the calligraphy work to be reviewed; The specific methods for determining the scoring mode selection module and the scoring module in the writing quality assessment model include: Obtain a calligraphy work dataset; the calligraphy work dataset includes a strongly labeled sample dataset and a weakly labeled sample dataset; the strongly labeled sample dataset includes several sample calligraphy images and their corresponding scores and awards; the weakly labeled sample dataset includes several sample calligraphy images and their corresponding awards; the sample calligraphy images are scanned images of the reviewed calligraphy works; Based on the score cutoff, the strongly labeled sample dataset is divided into a dataset of works that do not win awards and a dataset of works that are candidates for awards. A first convolutional neural network is trained using the dataset of works that are definitely not eligible for awards and the dataset of works that are candidate for awards to obtain a scoring mode selection module. A second convolutional neural network was trained using the dataset of works that were definitely not eligible for awards to obtain a coarse scoring network. A second convolutional neural network was trained using the dataset of award-winning candidate works to obtain the initial scoring network; The weakly labeled sample dataset is supplemented with pseudo-labels using the initial scoring network to obtain a supplemented labeled sample dataset. The initial scoring network is trained using the strongly labeled sample dataset and the supplementary labeled sample dataset to obtain the refined scoring network.

2. The automatic evaluation method for English handwriting quality according to claim 1, characterized in that, The weakly labeled sample dataset is augmented with pseudo-labels using the initial scoring network to obtain an augmented labeled sample dataset, specifically including: Based on the range of the total score of the works, the dataset of award-winning candidate works is divided into a first training set, a second training set, and a third training set. The total score of the works corresponding to the sample calligraphy images in the first training set is greater than the first score value and less than or equal to the full score value. The total score of the works corresponding to the sample calligraphy images in the second training set is greater than the second score value and less than or equal to the third score value. The total score of the works corresponding to the sample calligraphy images in the third training set is greater than the score threshold and less than or equal to the first score value. Among them, the score threshold value is less than the second score value, the second score value is less than the first score value, the first score value is less than the third score value, and the third score value is less than the full score value. The initial scoring network is trained using the first training set to obtain a first teacher model; the initial scoring network is trained using the second training set to obtain a second teacher model; and the initial scoring network is trained using the third training set to obtain a third teacher model. Based on the award level, the weakly labeled sample dataset is divided into a high-award sample dataset and a low-award sample dataset; The first teacher model is used to perform a first round of pseudo-label supplementation and confidence calculation on the sample calligraphy images in the high award sample dataset to obtain the first supplemented dataset and the first candidate dataset. The third teacher model is used to perform a first round of pseudo-label supplementation and confidence calculation on the sample calligraphy images in the low award sample dataset to obtain a second supplemented dataset and a second candidate dataset. The second teacher model is used to perform a second round of pseudo-label supplementation and confidence calculation on the sample calligraphy images in the first candidate dataset and the second candidate dataset to obtain the third supplemented dataset and the abandoned dataset. The first supplementary dataset, the second supplementary dataset, and the third supplementary dataset are determined as the supplementary label sample dataset.

3. The automatic evaluation method for English handwriting quality according to claim 1, characterized in that, The initial scoring network is trained using the strongly labeled sample dataset and the supplementary labeled sample dataset to obtain a refined scoring network, specifically including: The sample calligraphy images from the strongly labeled sample dataset are input into the initial scoring network to obtain the prediction results of the strongly labeled data; the strongly labeled data are the sample calligraphy images from the strongly labeled sample dataset. The sample calligraphy images from the supplementary label sample dataset are input into the initial scoring network to obtain the prediction results of the weak label data; the weak label data are the sample calligraphy images from the supplementary label sample dataset. The target loss function is determined based on the prediction results and corresponding real labels of the strongly labeled data, the prediction results and corresponding pseudo labels of the weakly labeled data, and the confidence level; wherein, the real label is the score corresponding to the sample calligraphy image in the strongly labeled sample dataset; the pseudo label is the score corresponding to the sample calligraphy image in the supplementary labeled sample dataset; and the confidence level is determined based on the probability distribution of the correspondence between score intervals and weak labels and the probability distribution of scores within each interval. With the convergence of the target loss function as the objective, the initial scoring network is trained to obtain a refined scoring network.

4. The automatic evaluation method for English handwriting quality according to claim 3, characterized in that, The expression for the target loss function is: in, This is the third loss value. For the first Prediction results for strongly labeled data For the first The true label of a strongly labeled data set. For the first Prediction results for weakly labeled data For the first The pseudo-labels for the weakly labeled data are: p = p_indicator of the sample calligraphy image in the strongly labeled sample dataset, P_total number of sample calligraphy images in the strongly labeled sample dataset, q = q_indicator of the sample calligraphy image in the supplementary labeled sample dataset, and Q_total number of sample calligraphy images in the supplementary labeled sample dataset. Pseudo-label Confidence level, Indicates the first The score of each weakly labeled data point Indicates the first A teacher model, Indicates the first There are 5 weakly labeled data points, and L1() represents the L1 loss function.

5. The automatic evaluation method for English handwriting quality according to claim 4, characterized in that, The formula for calculating the confidence level of the pseudo-label is as follows: in, This represents the probability distribution of the correspondence between the score intervals of the pseudo-labels and the weak labels. This represents the probability distribution of scores within the score interval where the pseudo-label is located. Indicates the first A score range, Indicates the first The length of each fraction interval.

6. The automatic evaluation method for English handwriting quality according to claim 1, characterized in that, Also includes: The comments on the calligraphy works to be reviewed are determined based on the comment database and the scores of the calligraphy works to be reviewed; The comment library includes comments corresponding to different indicators in different score ranges; the indicators include: horizontal angle consistency, tilt angle consistency, letter size consistency, character spacing consistency, and word spacing consistency.

7. An automatic evaluation system for English handwriting quality, characterized in that, include: The acquisition module is used to acquire a target calligraphy image; the target calligraphy image is a scanned image of the calligraphy work to be reviewed. The evaluation module uses a writing quality evaluation model to evaluate the target calligraphy image and obtain the score and award of the calligraphy work to be evaluated; The scores include: indicator scores and total work scores; the indicator scores include: horizontal angle consistency score, tilt angle consistency score, letter size consistency score, character spacing consistency score, and word spacing consistency score. The handwriting quality assessment model includes: a scoring mode selection module, a scoring module, and a post-processing module; the scoring module includes: a coarse scoring network and a fine scoring network. The scoring mode selection module is used to perform binary classification on the target calligraphy image to obtain a classification result; the classification result includes: the calligraphy work to be evaluated is a work that is definitely not eligible for an award or the calligraphy work to be evaluated is a candidate for an award; The scoring module is used to score the target calligraphy image based on the classification results, and obtain the index score of each line of the calligraphy work to be evaluated; wherein, the coarse scoring network is used to score the works that are definitely not eligible for awards, and the fine scoring network is used to score the works that are candidate for awards. The post-processing module is used to determine the score and award of the calligraphy work to be reviewed based on the index score of each line of the calligraphy work to be reviewed; The specific methods for determining the scoring mode selection module and the scoring module in the writing quality assessment model include: Obtain a calligraphy work dataset; the calligraphy work dataset includes a strongly labeled sample dataset and a weakly labeled sample dataset; the strongly labeled sample dataset includes several sample calligraphy images and their corresponding scores and awards; the weakly labeled sample dataset includes several sample calligraphy images and their corresponding awards; the sample calligraphy images are scanned images of the reviewed calligraphy works; Based on the score cutoff, the strongly labeled sample dataset is divided into a dataset of works that do not win awards and a dataset of works that are candidates for awards. A first convolutional neural network is trained using the dataset of works that are definitely not eligible for awards and the dataset of works that are candidate for awards to obtain a scoring mode selection module. A second convolutional neural network was trained using the dataset of works that were definitely not eligible for awards to obtain a coarse scoring network. A second convolutional neural network was trained using the dataset of award-winning candidate works to obtain the initial scoring network; The weakly labeled sample dataset is supplemented with pseudo-labels using the initial scoring network to obtain a supplemented labeled sample dataset. The initial scoring network is trained using the strongly labeled sample dataset and the supplementary labeled sample dataset to obtain the refined scoring network.

8. An electronic device, characterized in that, The device includes a memory and a processor, the memory being used to store a computer program, and the processor running the computer program to cause the electronic device to perform the automatic evaluation method for English handwriting quality as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the automatic evaluation method for English handwriting quality as described in any one of claims 1 to 6.

Images