A calligraphy evaluation method based on residual network and expert knowledge distillation

Abstract

The present invention discloses a calligraphy work evaluation method based on residual network and expert knowledge distillation, which belongs to the field of information technology. In view of the problem that the existing feature extraction method is mainly oriented to image processing and cannot effectively deal with the delicate strokes and unique structural features in calligraphy works, this application conducts research based on improved deep residual network and expert knowledge distillation, designs an improved deep residual network model to meet the extraction needs of delicate strokes and complex structural features in calligraphy works, and at the same time introduces expert knowledge distillation technology, guides model training based on expert knowledge soft labels, realizes accurate extraction of multi-dimensional features of calligraphy works, and provides effective support for calligraphy evaluation.

Description

Technical Field

[0001] The present invention belongs to the field of information technology, and in particular relates to a calligraphy work evaluation method based on residual network and expert knowledge distillation. Background Art

[0002] As an important part of traditional Chinese culture, calligraphy has always relied on the subjective judgment of experts for its aesthetic evaluation, lacking objective and quantitative evaluation standards. With the development of deep learning technology, especially the successful application of convolutional neural networks (CNN) in the field of image recognition and classification, it has provided new possibilities for the automated evaluation of calligraphy works. The residual network model (ResNet) has been widely used in various image processing tasks due to its superior performance in image feature extraction.

[0003] In calligraphy intelligent evaluation, it is crucial to accurately extract calligraphy features and perform multi-dimensional scoring. However, existing feature extraction methods have the following problems in calligraphy intelligent evaluation:

[0004] 1. It is oriented towards general image processing and has difficulty in dealing with the delicate stroke structure and unique artistic style of calligraphy works;

[0005] 2. The single-task evaluation model cannot efficiently cope with multi-dimensional evaluation tasks and is difficult to handle the diversity and complexity of calligraphy scoring;

[0006] 3. Based on the hard label training strategy, it is difficult to capture rich evaluation distribution information, resulting in limited scoring accuracy. Summary of the invention

[0007] In order to solve the problems raised in the above background technology, the present invention provides a calligraphy work evaluation method based on residual network and expert knowledge distillation, so as to solve the problems that the existing technology is difficult to cope with the delicate stroke structure and unique artistic style in calligraphy works, cannot efficiently cope with multi-dimensional evaluation tasks, and is difficult to capture rich evaluation distribution information based on hard label training strategies.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] A calligraphy work evaluation method based on residual network and expert knowledge distillation includes the following steps:

[0010] S1: Setting a deep residual network model, using an improved special convolution kernel in the first layer of the deep residual network model, the size of the special convolution kernel is set to completely cover the size of a single word in the calligraphy work, and the stride of the special convolution kernel is set to the size of a single word in the calligraphy work;

[0011] S2: Set five evaluation classification heads, namely, the overall evaluation classification head, the brushwork evaluation classification head, the structure evaluation classification head, the composition evaluation classification head and the paper surface evaluation classification head. Each classification head uses an independent neural network structure, and all five evaluation classification heads are connected to the deep residual network model.

[0012] S3: Collect multiple calligraphy works from different candidates. The collected calligraphy works include different calligraphy question types and calligraphy grades. Multiple experts score and vote on each calligraphy work from five dimensions: overall evaluation, brushwork, structure, composition and paper surface. Each dimension is set with multiple scoring grades, and a data set including calligraphy works and their multi-dimensional scores is obtained;

[0013] S4: The dataset is divided into a training set and a validation set according to a preset ratio. Each sample in the training set and the validation set includes an image of a calligraphy work and its corresponding multi-dimensional score label. For each calligraphy work in the training set, its multi-dimensional score is smoothed by introducing a temperature parameter, and a soft label is constructed based on the expert knowledge distillation strategy. The training set is used to train the deep residual network model, and the validation set is used to evaluate the performance and generalization ability of the deep residual network model.

[0014] S5: Perform a preset number of rounds of training, each round of training includes:

[0015] S5.1: Input the training set into the deep residual network model to extract calligraphy features;

[0016] S5.2: Input the calligraphy work features into five evaluation classification heads, and each classification head independently outputs the corresponding scoring results;

[0017] S5.3: Construct a weighted total loss function by combining the outputs of all classification heads with the soft labels, and use the soft labels as supervision signals to update the parameters of the deep residual network model through back propagation;

[0018] S5.4: Input the validation set into the deep residual network model to verify the performance of the deep residual network model;

[0019] S6: Deploy the trained deep residual network model, input the calligraphy works into the deep residual network model, obtain the scores of the five dimensions of the input calligraphy works, and complete the evaluation of the calligraphy works.

[0020] Preferably, the scoring levels include four levels: "excellent", "qualified", "unqualified" and "zero".

[0021] Preferably, the construction of soft labels in S4 is as follows: for each calligraphy work, the number of votes for each rating level is converted into a probability distribution, the distribution of ratings is smoothed by introducing a temperature parameter, and each expert's multi-dimensional rating of the calligraphy work is converted into a corresponding soft label vector.

[0022] Preferably, T is set as a temperature parameter, where T>1.

[0023] Preferably, in S4, the data set is divided into a training set and a validation set in a ratio of 7:3.

[0024] Preferably, in S5.3, the total loss function adopts cross entropy loss.

[0025] Preferably, the five evaluation classification heads share the underlying feature extraction layer.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] In view of the fact that existing feature extraction methods are mainly oriented towards image processing and cannot effectively deal with the problem of delicate strokes and unique structural features in calligraphy works, this application conducts research based on improved deep residual networks and expert knowledge distillation, designs and improves deep residual network models to meet the needs of extracting delicate strokes and complex structural features in calligraphy works, and at the same time introduces expert knowledge distillation technology to guide model training based on expert knowledge soft labels, so as to achieve accurate extraction of multi-dimensional features of calligraphy works and provide effective support for calligraphy evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a schematic diagram of the process of this application. DETAILED DESCRIPTION

[0029] In order to facilitate those skilled in the art to understand the technical content of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific examples. It should be understood that the specific examples described herein are only used to explain the present invention and are not used to limit the present invention.

[0030] Embodiment 1:

[0031] like Figure 1 As shown, a calligraphy work evaluation method based on residual network and expert knowledge distillation includes the following steps:

[0032] S1: Set up a deep residual network model. In the first layer of the deep residual network model, an improved special convolution kernel is used. The size of the special convolution kernel is set to completely cover the size of a single word in the calligraphy work to capture the detailed features of the word. The stride of the special convolution kernel is set to the size of a single word in the calligraphy work to ensure that each convolution operation covers the next word to accurately extract the features of each word, avoid information loss, and reduce the impact of background noise on the evaluation. Each position of the feature map generated by the special convolution kernel corresponds to the feature representation of a word, ensuring that the features of each word can be analyzed and evaluated separately. After the first convolution layer, the standard design of the deep residual network is used, which retains the advantages of the deep residual network while having special processing capabilities for calligraphy features.

[0033] S2: Set up five evaluation and classification heads, namely the overall evaluation and classification head, the brushwork evaluation and classification head, the structure evaluation and classification head, the composition evaluation and classification head, and the paper surface evaluation and classification head. Each classification head uses an independent neural network structure. The five evaluation and classification heads share the underlying feature extraction layer to improve learning efficiency and effectively utilize the correlation between multi-dimensional tasks. The five evaluation and classification heads are all connected to the deep residual network model.

[0034] S3: Collect 10,000 calligraphy works from different candidates. The collected calligraphy works include different calligraphy question types (such as imitation and copying) and calligraphy levels (level 1, level 2, level 3). Multiple experts score and vote on each calligraphy work from five dimensions: overall evaluation, brushwork, structure, composition, and paper surface. Each dimension is set with four scoring levels including "excellent", "qualified", "unqualified" and "zero points". A data set including calligraphy works and their multi-dimensional scores is obtained. The final score is based on the results of all expert votes to ensure the fairness of the score and provide rich annotated data for subsequent training;

[0035] S4: The dataset is divided into a training set and a validation set in a ratio of 7:3, and the corresponding image path, label, and text information are returned. This part ensures that the deep residual network model can receive data in a consistent format. For each calligraphy work in the training set, the number of votes for each rating level is converted into a probability distribution. The temperature parameter T is introduced to smooth the rating distribution and then construct a soft label, where T>1. The multi-dimensional rating of each expert on the work is converted into a corresponding soft label vector. The soft label is used as the target value output by the deep residual network model. While helping the deep residual network model learn the correct category output, it can also obtain more information from the relationship between other categories.

[0036] S5: Perform a preset number of rounds of deep residual network model training. Each round of training includes:

[0037] S5.1: Input the training set into the deep residual network model to extract calligraphy features;

[0038] S5.2: Input the calligraphy work features into five evaluation classification heads, and each classification head independently outputs the corresponding scoring results;

[0039] S5.3: The output of all classification heads and the soft labels are used to construct a weighted total loss function. The total loss function adopts cross entropy loss and uses the soft labels as supervision signals. The parameters of the deep residual network model are updated through back propagation. The cross entropy loss function is used to measure the difference between the output of the deep residual network model and the soft labels generated by experts. The probability of each rating level predicted by the deep residual network model is calculated to be similar to the corresponding soft labels to guide the training of the deep residual network model.

[0040] S5.4: Input the validation set into the deep residual network model to verify the performance of the deep residual network model;

[0041] S6: Deploy the trained deep residual network model, input the calligraphy works into the deep residual network model, obtain the scores of the five dimensions of the input calligraphy works, and complete the evaluation of the calligraphy works.

[0042] In this embodiment, the present application targets the problem that existing feature extraction methods are mainly oriented towards image processing and cannot effectively deal with the delicate strokes and unique structural features in calligraphy works. A study on a deep residual network model based on an improved deep residual network and expert knowledge distillation is conducted, and an improved deep residual network model is designed to meet the needs of extracting delicate strokes and complex structural features in calligraphy works. At the same time, expert knowledge distillation technology is introduced, and the training of the deep residual network model is guided by expert knowledge soft labels, so as to achieve accurate extraction of multi-dimensional features of calligraphy works and provide effective support for calligraphy evaluation.

[0043] This application can be applied to the intelligent examination marking platform of calligraphy education institutions, or online calligraphy learning platforms, through which students can upload their own calligraphy works for intelligent evaluation and obtain detailed feedback.

[0044] Students upload their calligraphy works on the platform, and the works are directly passed into the trained deep residual network model. The feature map of the calligraphy works forms calligraphy embedding features through the feature embedding layer. The calligraphy question types and evaluation levels (such as level 2 imitation, level 1 off-the-post, etc.) form standard embedding features through the standard embedding layer. The embedded features of calligraphy works are passed into five classification heads: overall evaluation, brushwork, structure, composition, and paper surface, and the five-dimensional scores of the works are output. After obtaining the evaluation results, students can view the scores of each dimension and the corresponding feedback information. These feedbacks will provide specific improvement suggestions for each evaluation dimension to help students conduct targeted practice.

[0045] This application can complete the evaluation of students' calligraphy works in a short period of time. The overall evaluation time is shortened by 20% compared with traditional manual evaluation, which greatly improves work efficiency.

Claims

1. A calligraphy work evaluation method based on residual network and expert knowledge distillation, characterized in that: The following steps are involved: S1: Setting a deep residual network model, using an improved special convolution kernel in the first layer of the deep residual network model, the size of the special convolution kernel is set to completely cover the size of a single word in the calligraphy work, and the stride of the special convolution kernel is set to the size of a single word in the calligraphy work; S2: Set five evaluation classification heads, namely, the overall evaluation classification head, the brushwork evaluation classification head, the structure evaluation classification head, the composition evaluation classification head and the paper surface evaluation classification head. Each classification head uses an independent neural network structure, and all five evaluation classification heads are connected to the deep residual network model. S3: Collect multiple calligraphy works from different candidates. The collected calligraphy works include different calligraphy question types and calligraphy grades. Multiple experts score and vote on each calligraphy work from five dimensions: overall evaluation, brushwork, structure, composition and paper surface. Each dimension is set with multiple scoring grades, and a data set including calligraphy works and their multi-dimensional scores is obtained; S4: The dataset is divided into a training set and a validation set according to a preset ratio. Each sample in the training set and the validation set includes an image of a calligraphy work and its corresponding multi-dimensional score label. For each calligraphy work in the training set, its multi-dimensional score is smoothed by introducing a temperature parameter, and a soft label is constructed based on the expert knowledge distillation strategy. The training set is used to train the deep residual network model, and the validation set is used to evaluate the performance and generalization ability of the deep residual network model. S5: Perform a preset number of rounds of training, each round of training includes: S5.1: Input the training set into the deep residual network model to extract calligraphy features; S5.2: Input the calligraphy work features into five evaluation classification heads, and each classification head independently outputs the corresponding scoring results; S5.3: Construct a weighted total loss function by combining the outputs of all classification heads with the soft labels, and use the soft labels as supervision signals to update the parameters of the deep residual network model through back propagation; S5.4: Input the validation set into the deep residual network model to verify the performance of the deep residual network model; S6: Deploy the trained deep residual network model, input the calligraphy works into the deep residual network model, obtain the scores of the five dimensions of the input calligraphy works, and complete the evaluation of the calligraphy works.

2. According to claim 1, a calligraphy work evaluation method based on residual network and expert knowledge distillation is characterized in that: The scoring levels include "excellent", "qualified", "unqualified" and "zero".

3. According to claim 1, a calligraphy work evaluation method based on residual network and expert knowledge distillation is characterized in that: The construction of soft labels in S4 is as follows: for each calligraphy work, the number of votes for each rating level is converted into a probability distribution, the distribution of ratings is smoothed by introducing a temperature parameter, and the multi-dimensional ratings of each expert on the calligraphy work are converted into corresponding soft label vectors.

4. According to claim 3, a calligraphy work evaluation method based on residual network and expert knowledge distillation is characterized in that: Let T be the temperature parameter, where T>1.

5. According to claim 1, a calligraphy work evaluation method based on residual network and expert knowledge distillation is characterized in that: In S4, the data set is divided into a training set and a validation set in a ratio of 7:

3.

6. The calligraphy work evaluation method based on residual network and expert knowledge distillation according to claim 1 is characterized in that: In S5.3, the total loss function uses cross entropy loss.

7. The calligraphy work evaluation method based on residual network and expert knowledge distillation according to claim 1 is characterized in that: The five evaluated classification heads share the underlying feature extraction layers.

Images