A calligraphy character imitation evaluation system

By using deep learning image preprocessing and two-stage CPD point set registration technology, combined with calligraphy teaching experience, the problems of image distortion and low accuracy in calligraphy character copying evaluation have been solved, achieving efficient and accurate calligraphy character evaluation and providing multi-faceted guidance.

CN115641308BActive Publication Date: 2026-03-17INST OF AUTOMATION CHINESE ACAD OF SCI +1
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-10-17
Publication Date
2026-03-17

AI Technical Summary

Technical Problem

Existing methods for evaluating calligraphy copying suffer from problems such as image acquisition distortion, low evaluation accuracy, and lack of fine-grained analysis, and therefore cannot provide effective guidance for improvement.

Method used

Deep learning-based image preprocessing techniques are used for binarization and angle correction. Strokes are extracted by combining two-stage CPD point set registration technology. An evaluation strategy is constructed based on multi-angle calligraphy teaching experience to achieve high-precision evaluation of calligraphy characters.

Benefits of technology

It improves the accuracy and practical efficiency of calligraphy copying evaluation, enabling high-precision evaluation at multiple granular levels from individual strokes to overall structure, and providing accurate guidance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115641308B_ABST
    Figure CN115641308B_ABST
Patent Text Reader

Abstract

This invention belongs to the field of image processing technology, specifically relating to a calligraphy copying evaluation system. It aims to solve the problems of color and size distortion in the images of student practice works collected by existing calligraphy copying evaluation methods, as well as insufficient granularity in evaluation, leading to low evaluation accuracy. The system includes: an image acquisition module configured to acquire images of the copied characters and reference characters; a preprocessing module configured to preprocess the images; a stroke extraction module configured to extract strokes; a writing consistency evaluation module configured to perform stroke consistency evaluation; a stroke assessment module configured to compare strokes one by one based on set stroke morphological features; a structural component evaluation module configured to compare structural component morphological features one by one to obtain structural component evaluation results; and a whole-character qualitative evaluation module configured to obtain the evaluation results of student practice works. This invention improves the evaluation accuracy and the actual efficiency of the evaluation system.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a calligraphy character imitation evaluation system. Background Technology

[0002] Calligraphy is a cultural treasure, and calligraphy education is an important part of promoting traditional culture. Currently, in response to the call for change, more and more schools are offering calligraphy courses. However, the severe shortage of calligraphy teachers is a major obstacle to the promotion of calligraphy education. Calligraphy copying and evaluation technology can assist calligraphy teaching, intelligently assessing students' calligraphy writing quality and providing guidance, effectively alleviating the problem of insufficient calligraphy teachers.

[0003] Current calligraphy copying assessment techniques typically use software systems to perform simple morphological comparisons between student works and model characters, providing basic evaluations. However, these methods only yield general results and are largely intended to stimulate student interest and increase classroom engagement. The assessment results do not identify specific problems with the students' copying, and students cannot intuitively understand areas for improvement. Current calligraphy copying assessment techniques suffer from the following problems:

[0004] (1) The assessment strategy is too simplistic and lacks sufficient experience in assessing actual calligraphy teaching, making it impossible to develop substantive guidance and improvement suggestions;

[0005] (2) Stroke shape assessment is an important part of calligraphy copying assessment. Most existing methods start from the overall structure and at most point out the problem of stroke spacing, lacking fine-grained analysis of stroke shape.

[0006] (3) The calligraphy teaching environment is complex. Directly collected images of students' works are prone to distortion in color, size and angle. Existing methods generally lack sufficient preprocessing means or strictly limit the collection process to prevent distortion. The former will affect the accuracy of subsequent evaluation, while the latter will increase the difficulty of collecting students' works and reduce the actual efficiency of the evaluation method.

[0007] To address the above issues and enhance the professionalism and substantive guiding value of calligraphy copying evaluation techniques, this invention proposes a new method for evaluating calligraphy. Summary of the Invention

[0008] To address the aforementioned problems in existing technologies, namely, the color and size distortions in student practice images collected by current calligraphy copying evaluation methods, and the low evaluation accuracy due to insufficient granularity in the evaluation process, this invention proposes a calligraphy copying evaluation system. This system includes: an image acquisition module, a preprocessing module, a stroke extraction module, a writing consistency evaluation module, a stroke evaluation module, a structural component evaluation module, and a whole-character qualitative evaluation module.

[0009] The image acquisition module is configured to acquire images of characters to be copied and images of reference characters; the reference character images are images of each calligraphy character in a standard calligraphy work with known strokes, radicals, and writing order; the image of characters to be copied is an image of each calligraphy character in a student's practice work copying a standard calligraphy work.

[0010] The preprocessing module is configured to input the copied character image into a pre-constructed image preprocessing network, and perform binarization, image size and angle correction processing on the copied character image to obtain a preprocessed copied character image; the image preprocessing network is constructed based on a deep learning neural network.

[0011] The stroke extraction module is configured to acquire the stroke outline point set of the calligraphy characters in the preprocessed copy character image and the reference character image, and obtain the strokes of each calligraphy character in the preprocessed copy character image by using a stroke extraction method based on two-stage CPD point set registration.

[0012] The writing consistency evaluation module is configured to extract features from the reference character image and the preprocessed copy character image, and calculate the distance between the features. If the distance is greater than a set second distance threshold, the writing content is considered to be inconsistent with the reference character, that is, the student's copy character and the reference character are not the same character.

[0013] The stroke evaluation module is configured to compare the strokes of each calligraphy character in the preprocessed copy image extracted by the stroke extraction module with the strokes of each calligraphy character in the reference image based on set stroke morphological features. If the distance between the features is greater than a set threshold, there is a corresponding stroke morphological writing problem. The set morphological features include stroke length, curvature, thickness, slant, starting and ending stroke outlines, and whether strokes are written in excess or omitted.

[0014] The structural component evaluation module is configured to compare the strokes of each calligraphy character in the preprocessed copy image extracted by the stroke extraction module with the strokes of each calligraphy character in the reference image based on the set structural component morphological features. If the distance between the features is greater than a set threshold, there is a corresponding component morphological writing problem. The set structural component morphological features include key stroke layout, component tightness, overall tightness, and component relative position.

[0015] The whole-character qualitative assessment module is configured to sort the writing problems evaluated by the writing consistency assessment module, the stroke assessment module, and the structural component assessment module according to a set priority, and locate them in the corresponding area of ​​the student's practice work; based on the priority and the area location information of the problems, a qualitative assessment is performed, thereby obtaining the assessment result of the student's practice work.

[0016] In some preferred embodiments, the image preprocessing network is constructed based on ResBlock, ASPP module, upconvolution module, and STN Block; the upconvolution module is constructed based on multiple convolutional layers and upsampling layers; in the upconvolution module, the convolutional layers and upsampling layers are arranged alternately, and when arranged alternately, the convolutional layers are in front of the upsampling layers;

[0017] The ResBlock is connected to the ASPP module and the STN Block respectively; the input of the ResBlock is the image of the copied character.

[0018] The ASPP module is connected to the upper convolution module; the ResBlock is skipped and connected to the upper convolution module. During the skipped connection, the output of the ResBlock is... The output of the convolutional layer and the upper convolutional module is N. The convolutional layers are connected by skip connections, i.e., convolutional layers of the same size are skipped; the upper convolutional module is connected to the STN Block; the STNBlock outputs a preprocessed image of the copied character.

[0019] In some preferred embodiments, the copied character image is input into a pre-constructed image preprocessing network to perform binarization, image size and angle correction processing on the copied character image to obtain a preprocessed copied character image. The method is as follows:

[0020] The image of the copied character is input into the ResBlock of the image preprocessing network for residual processing, and the feature after residual processing is used as the first feature.

[0021] The first feature is input into the ASPP of the image preprocessing network for processing. After processing, it is convolved and combined with the features input by the ResBlock skip connection to obtain the binarization result corresponding to the copied character image.

[0022] The first feature is input into the CNN of the image preprocessing network STN Block to obtain the corresponding projective transformation parameters;

[0023] The binarization result and the projective transformation parameters are input into the spatial transformation module of the STN Block for spatial transformation, thereby obtaining the preprocessed tracing character image.

[0024] In some preferred embodiments, the stroke outline point sets of the calligraphy characters in the preprocessed copy character image and the reference character image are obtained, and the strokes of each calligraphy character in the preprocessed copy character image are obtained by a stroke extraction method based on two-stage CPD point set registration. The method is as follows:

[0025] The reference character image is registered with the stroke outline point set of the calligraphy characters in the preprocessed copy character image using the CPD point set registration method, so as to obtain the stroke outline point set of each calligraphy character in the reference character image after stroke transformation, which is used as the first point set.

[0026] For each stroke outline point set in the first point set, extract one stroke outline point set in sequence as the second point set; traverse the second point set, remove the outline points in the stroke outline point set of the calligraphy characters in the preprocessed copying character image whose Euclidean distance is greater than a set first distance threshold, and use the removed stroke outline point set of the calligraphy characters in the preprocessed copying character image as the third point set.

[0027] The third point set is registered with the second point set using the CPD point set registration method to obtain the transformed second point set, and the transformed second point set is used as the strokes of the calligraphy characters in the preprocessed copying image.

[0028] Continue extracting the next stroke outline point set from the first point set in sequence, and perform removal and registration processing until all stroke outline point sets in the first point set have been processed.

[0029] In some preferred embodiments, features are extracted from the reference character image and the preprocessed copy character image, and the distance between the features is calculated. If the distance is greater than a set second distance threshold, the written content is considered inconsistent with the reference character, that is, the student's copy character and the reference character are considered not to be the same character. The method is as follows:

[0030] The reference character image and the preprocessed copy character image are input into a pre-constructed character recognition algorithm model. The features corresponding to the reference character image and the preprocessed copy character image output by the last layer of the character recognition algorithm model are obtained as the first feature and the second feature, and L2 normalization is performed. The Euclidean distance between the normalized first feature and the second feature is calculated, and it is determined whether it exceeds the set second distance threshold. If it exceeds the threshold, it is considered that the written content and the reference character are not consistent.

[0031] In some preferred embodiments, the strokes of each calligraphic character in the preprocessed copy image extracted by the stroke extraction module are compared one by one with the strokes of each calligraphic character in the reference image based on set stroke shape features. If the distance between the features is greater than a set threshold, then there is a corresponding stroke shape writing problem. The method is as follows:

[0032] Single stroke tilt: Calculate the minimum bounding rectangle of the stroke skeleton image, and take the angle between the long side and the horizontal axis as the stroke tilt angle; calculate the average tilt angle of all strokes as the overall tilt, and subtract the overall tilt from the tilt of each stroke to obtain the final tilt of a single stroke; calculate the error between the final tilt of a single stroke and the corresponding reference character stroke; if the error exceeds the threshold, there is a problem; the stroke skeleton image is the skeleton image corresponding to the strokes of each calligraphy character in the preprocessed copy character image extracted by the stroke extraction module.

[0033] Single stroke thickness: The ratio of the average thickness of each stroke in the stroke skeleton image to the size of the whole character is calculated as the quantized thickness value of the single stroke. The error between this quantized thickness value and the quantized thickness value of the corresponding stroke of the reference character is calculated. If it exceeds the threshold, there is a problem.

[0034] Single stroke length: The ratio of the length of each stroke in the stroke skeleton image to the size of the whole character is calculated as the length value. The error between this length value and the length value of the corresponding stroke of the reference character is calculated. If it exceeds the threshold, there is a problem.

[0035] Single stroke straightness: In the stroke skeleton image, a set number of pixels are uniformly sampled on each stroke, and the average angle transformation rate of these pixels is calculated as the straightness. The error between this straightness and the straightness of the corresponding stroke of the reference character is calculated. If it exceeds the threshold, there is a problem.

[0036] Starting and ending strokes of a single stroke: Taking the two endpoints of each stroke in the stroke skeleton image as the center, the starting and ending stroke images are extracted. The category of the starting and ending strokes is obtained through a pre-trained contour recognition model. When the category is different from the category of the stroke corresponding to the reference character, there is a problem. The categories of the starting and ending strokes include square, round, pointed, and others. The contour recognition model is constructed based on sequentially connected convolutional layers and fully connected layers.

[0037] Missed and overwritten: During the registration of the second point set with the third point set using the CPD point set registration method, the average migration distance of each point in the second point set before and after registration is calculated to obtain the average migration distance. If the average migration distance is greater than a set threshold, there is a missed write; the migration distance is the Euclidean distance between the positions of each point in the second point set before and after registration.

[0038] All detected strokes are obtained and compared with the entire character being copied. The extra parts in the entire character are the areas of extra strokes written.

[0039] In some preferred embodiments, the strokes of each calligraphic character in the preprocessed copy image extracted by the stroke extraction module are compared one by one with the strokes of each calligraphic character in the reference image based on the set structural component morphological features. If the distance between the features is greater than a set threshold, then there is a corresponding component morphological writing problem. The method is as follows:

[0040] Key stroke layout: Extract the strokes marked as key strokes in the reference character image, calculate the ratio of the Euclidean distance between the key strokes and the surrounding strokes in each calligraphy character of the preprocessed copy character image to the size of the whole character, calculate the deviation between this ratio and the corresponding ratio of the reference character, if the deviation is greater than the set first deviation threshold, then there is a key stroke layout deviation problem.

[0041] Component tightness: Extract the centroid of the component corresponding to each calligraphic character in the reference character image, and calculate the ratio of the distance from the stroke of each calligraphic character in the reference character image to the centroid of the corresponding component to the component size, as the first centroid distance; the component includes a radical and a part composed of multiple strokes;

[0042] Extract the centroid of the corresponding component of each calligraphy character in the preprocessed copying image, and calculate the ratio of the distance from the stroke of each calligraphy character to the centroid of the corresponding component to the component size, as the second centroid distance; calculate the deviation between the first centroid distance and the second centroid distance. If the deviation is greater than the set second deviation threshold, there is a component tightness deviation problem.

[0043] Overall tightness: Extract the centroid of each corresponding component of each calligraphy character in the preprocessed copying image, calculate the ratio of the average relative distance between each pair of centroids to the size of the whole character, and use it as the tightness. Calculate the deviation between this tightness and the tightness of each calligraphy character component in the reference image. If it exceeds the set third deviation threshold, there is an overall tightness problem.

[0044] Component relative position: The component closest to the center of the reference character image is taken as the reference component. The ratio of the centroid distance between other components and the reference component in the preprocessed copy character image to the size of the whole character is calculated as the relative position of other components to the reference component. The deviation between this relative position and the relative position of each calligraphy component in the reference character image is calculated. When it exceeds the set fourth deviation threshold, there is a component relative position problem.

[0045] In some preferred embodiments, a qualitative assessment is performed based on priority and the regional location information of the problem to obtain the evaluation results of the student's practice work. The method is as follows:

[0046] The total score of the student's practice work set by the calligraphy copying evaluation system is obtained as the first score; based on the writing problem deduction rule table, the priority of writing problems in the student's practice work and the regional location information of the problems are matched to obtain the score deducted for writing problems, and the sum is obtained as the second score.

[0047] Subtract the second score from the first score to get the final score of the student's work, which is the evaluation result of the student's work.

[0048] The writing problem deduction rule table is a mapping relationship between the preset writing problem priority and the regional location information of the writing problem and the corresponding writing problem deduction points.

[0049] In some preferred embodiments, the calligraphy copying evaluation system further includes an output module;

[0050] The output module is configured to integrate the evaluation results of the writing consistency evaluation module, stroke evaluation module, structural component evaluation module, and student practice works to form a standardized output in JSON format.

[0051] The beneficial effects of this invention are:

[0052] This invention improves the accuracy of evaluation and the actual efficiency of the evaluation system.

[0053] 1) A deep learning-based image correction and binarization model, combining a binary semantic segmentation model and a spatial transformation model, enables preprocessing of student artwork, including binarization and size / angle correction. In practical applications, this model can stably and efficiently complete the preprocessing of student artwork, improving the accuracy of subsequent assessments and its efficiency in supporting teaching.

[0054] 2) Stroke extraction technology based on CPD two-stage point set registration: This technology accurately extracts the outline of each stroke in student artwork from the whole to the part through CPD click registration. In practical applications, this model can accurately and quickly extract strokes from student artwork.

[0055] 3) Calligraphy Assessment Strategies Based on Calligraphy Teaching Experience. By combining calligraphy teaching experience, assessment rules with professionalism and substantial teaching guidance significance are constructed. On the one hand, from multiple perspectives—including the shape of individual strokes, structural components, and overall shape—the writing problems of calligraphy exercises compared to copied characters are detected, achieving a multi-granular, high-precision evaluation of calligraphy characters from individual stroke details to overall structure. On the other hand, through a whole-character qualitative assessment model, the severity of writing problems is ranked, writing problems are accurately located, and qualitative assessment guidance is provided, resulting in accurate, efficient, and substantially instructive assessment results. This invention, as a tool to assist calligraphy teaching and aid self-study, can provide intelligent calligraphy assessment and guidance for primary and secondary school teachers and students, promoting calligraphy teaching and improving the writing ability and artistic literacy of primary and secondary school students. Attached Figure Description

[0056] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings.

[0057] Figure 1 This is a schematic diagram of the framework of a calligraphy copying evaluation system according to an embodiment of the present invention;

[0058] Figure 2 This is a schematic diagram of the image preprocessing network performing preprocessing according to an embodiment of the present invention;

[0059] Figure 3 This is a schematic diagram of the calligraphy copying evaluation process according to an embodiment of the present invention;

[0060] Figure 4 This is a schematic diagram of the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions in the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0062] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0063] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0064] The calligraphy copying evaluation system of the present invention, such as Figure 1 As shown, the system includes: an image acquisition module, a preprocessing module, a stroke extraction module, a handwriting consistency evaluation module, a stroke evaluation module, a structural component evaluation module, and a whole character qualitative evaluation module;

[0065] The image acquisition module is configured to acquire images of characters to be copied and images of reference characters; the reference character images are images of each calligraphy character in a standard calligraphy work with known strokes, radicals, and writing order; the image of characters to be copied is an image of each calligraphy character in a student's practice work copying a standard calligraphy work.

[0066] The preprocessing module is configured to input the copied character image into a pre-constructed image preprocessing network, and perform binarization, image size and angle correction processing on the copied character image to obtain a preprocessed copied character image; the image preprocessing network is constructed based on a deep learning neural network.

[0067] The stroke extraction module is configured to acquire the stroke outline point set of the calligraphy characters in the preprocessed copy character image and the reference character image, and obtain the strokes of each calligraphy character in the preprocessed copy character image by using a stroke extraction method based on two-stage CPD point set registration.

[0068] The writing consistency evaluation module is configured to extract features from the reference character image and the preprocessed copy character image, and calculate the distance between the features. If the distance is greater than a set second distance threshold, the strokes are considered inconsistent.

[0069] The stroke evaluation module is configured to compare the strokes of each calligraphy character in the preprocessed copy image extracted by the stroke extraction module with the strokes of each calligraphy character in the reference image based on set stroke morphological features. If the distance between the features is greater than a set threshold, there is a corresponding stroke morphological writing problem. The set morphological features include stroke length, curvature, thickness, slant, starting and ending stroke outlines, and whether strokes are written in excess or omitted.

[0070] The structural component evaluation module is configured to compare the strokes of each calligraphy character in the preprocessed copy image extracted by the stroke extraction module with the strokes of each calligraphy character in the reference image based on the set structural component morphological features. If the distance between the features is greater than a set threshold, there is a corresponding component morphological writing problem. The set structural component morphological features include key stroke layout, component tightness, overall tightness, and component relative position.

[0071] The whole-character qualitative assessment module is configured to sort the writing problems evaluated by the writing consistency assessment module, the stroke assessment module, and the structural component assessment module according to a set priority, and locate them in the corresponding area of ​​the student's practice work; based on the priority and the area location information of the problems, a qualitative assessment is performed, thereby obtaining the assessment result of the student's practice work.

[0072] To more clearly explain the calligraphy copying evaluation system of this invention, the following description is in conjunction with the appendix. Figure 3 The modules of one embodiment of the system of the present invention will be described in detail below.

[0073] This invention proposes a novel calligraphy character imitation evaluation system. By combining evaluation rules and stroke extraction algorithms built upon fundamental knowledge of calligraphy, it further improves the accuracy and granularity of calligraphy character evaluation. The system builds peripheral service modules around the evaluation algorithm, including an image acquisition module, a preprocessing module, a stroke extraction module, a writing consistency evaluation module, a stroke evaluation module, a structural component evaluation module, and a whole-character qualitative evaluation module, thus realizing a comprehensive calligraphy character imitation evaluation system. Details are as follows:

[0074] The image acquisition module is configured to acquire images of characters to be copied and images of reference characters; the reference character images are images of each calligraphy character in a standard calligraphy work with known strokes, radicals, and writing order; the image of characters to be copied is an image of each calligraphy character in a student's practice work copying a standard calligraphy work.

[0075] In this embodiment, the reference characters refer to standard model characters, rubbings, works by famous calligraphers, etc. The obtained reference character images are the images of each calligraphic character in a standard calligraphy work with known strokes, radicals, and writing order. The tracing characters are calligraphy practice works written by students imitating the reference characters; the tracing character images are the images of each calligraphic character in student practice works imitating standard calligraphy works.

[0076] Furthermore, this invention constructs a reference character annotation database to annotate reference characters. Specifically, using open-source image annotation tools such as Labelme, users can annotate the stroke outlines of reference characters. Each stroke outline is represented by a set of coordinate values ​​and accompanied by a text label. Additionally, the radicals and key strokes of the reference characters are also annotated. The annotation data is stored in a relational database such as MySQL. The comprehensive information of the reference characters consists of the reference character image and the reference character annotation information.

[0077] The preprocessing module is configured to input the copied character image into a pre-constructed image preprocessing network, and perform binarization, image size and angle correction processing on the copied character image to obtain a preprocessed copied character image; the image preprocessing network is constructed based on a deep learning neural network.

[0078] The images of students' calligraphy practice works are collected during the teaching process. In the complex teaching environment, the images obtained by this method are distorted in terms of angle, size and color, which affects the subsequent evaluation process.

[0079] In this embodiment, to address this issue, a deep learning-based image correction and binarization model is constructed to standardize these inputs to a uniform format. The preprocessing model structure is as follows: Figure 2 As shown, the main architecture is an image preprocessing network similar to Deeplabv3, which determines whether each pixel in a student's calligraphy practice work (i.e., a copy of a character image) is a written stroke. During the downward convolution process, the result of the last layer of ResBlock is input into STNBlock (Spatial Transformation Module). STN Block predicts a projective transformation matrix, and then performs a projective transformation on the extracted binary written stroke result through Spatial Transformation to correct for size and angle. The preprocessing unit in this invention has good generalization ability and can adapt to various student works collected in complex teaching environments. It can effectively correct the size, angle, and color distortions caused by the acquisition process of student works, which helps to improve the accuracy of subsequent calligraphy evaluation. The image preprocessing network is constructed based on ResBlock, ASPP module, upconvolution module, and STNBlock. The upconvolution module is constructed based on multiple convolutional layers and upsampling layers (in this invention, the sampling factor of all upsampling layers is 2). In the upconvolution module, convolutional layers and upsampling layers are arranged alternately, with the convolutional layers preceding the upsampling layers. ResBlock is connected to both the ASPP module and the STN Block. The input to ResBlock is the image of the copied character. The ASPP module is connected to the upconvolution module. To enhance the processing of low-level features, ResBlock is used in a skip connection with the upconvolution module. During the skip connection, the output of ResBlock is... The output of the convolutional layer and the upper convolutional module is N. The convolutional layers are connected in a skip connection manner, i.e., convolutional layers of the same size are skipped; the upper convolutional module is connected to the STN Block; the STN Block outputs a preprocessed image of the copied character.

[0080] The image of the copied character is input into a pre-constructed image preprocessing network, which performs binarization, image size and angle correction on the image of the copied character to obtain a preprocessed image of the copied character. The method is as follows:

[0081] The image of the copied character is input into the ResBlock of the image preprocessing network for residual processing, and the feature after residual processing is used as the first feature.

[0082] The first feature is input into the ASPP of the image preprocessing network for processing. After processing, it is convolved and combined with the features input by the ResBlock skip connection to obtain the binarization result corresponding to the copied character image.

[0083] The first feature is input into the CNN of the image preprocessing network STN Block to obtain the corresponding projective transformation parameters;

[0084] The binarization result and the projective transformation parameters are input into the spatial transformation module of the STN Block for spatial transformation, thereby obtaining the preprocessed tracing character image.

[0085] The stroke extraction module is configured to acquire the stroke outline point set of the calligraphy characters in the preprocessed copy character image and the reference character image, and obtain the strokes of each calligraphy character in the preprocessed copy character image by using a stroke extraction method based on two-stage CPD point set registration.

[0086] In this embodiment, a stroke extraction technique based on two-stage CPD point set registration is proposed. In the first stage, CPD technology is used to establish an overall registration relationship between the copied character and the target calligraphy work, achieving preliminary stroke transformation for individual copied characters. In the second stage, local CPD point set registration technology is used to achieve precise registration between the strokes of individual copied characters and the corresponding strokes of the calligraphy work, thus obtaining the strokes of the calligraphy work. Specifically:

[0087] In the first stage, the stroke outline point sets (including overall outline point sets and individual stroke point sets) of the calligraphy characters in the reference character images stored in the database are obtained, and the overall outline point sets of the calligraphy characters in the preprocessed copying character images are extracted. The CPD point set registration method is used to establish the morphological registration relationship between the outline points of the student's calligraphy practice works and the outline points of the reference characters, so as to obtain the transformed stroke outline point sets of each calligraphy character in the reference character images, which are used as the first point sets (the information of the individual stroke outline point sets and the overall outline point sets is basically the same, the difference is that the outline point set of each stroke is recorded separately and includes some outline point sets of overlapping areas). At this time, these transformed individual copying character stroke outline point sets and the stroke outline point sets corresponding to the reference characters have similar positions and shapes.

[0088] In the second stage, for each stroke outline point set in the first point set, one stroke outline point set is extracted sequentially as the second point set; the second point set is traversed, and outline points in the stroke outline point set of the calligraphy characters in the preprocessed copying image whose Euclidean distance to the corresponding outline point of the calligraphy character is greater than a set first distance threshold (in this invention, the first distance threshold is set to 0.3 of the size of the student's calligraphy practice work, that is, delete outline points in the first point set whose Euclidean distance to the corresponding outline point of the reference character is greater than 0.3), to obtain the outline point set of local calligraphy practice works, and the stroke outline point set of the calligraphy characters in the preprocessed copying image after the removal is used as the third point set;

[0089] The third point set is registered with the second point set using the CPD point set registration method to obtain the transformed second point set, and the transformed second point set is used as the strokes of the calligraphy characters in the preprocessed copying image.

[0090] Finally, continue to extract the next stroke outline point set from the first point set in sequence, and perform removal and registration processing until all stroke outline point sets in the first point set have been processed.

[0091] This method achieves precise registration of the stroke outline points of a single reference character after transformation to the outline point set of a local calligraphy practice piece. The final transformed stroke outline points of the single reference character are used as the outline points of the corresponding strokes in the student's calligraphy practice piece. Following the same method, the outline point set of each stroke in each calligraphy practice piece is extracted sequentially, and the stroke is filled around each stroke outline point to obtain each stroke of the student's calligraphy practice piece.

[0092] The writing consistency evaluation module is configured to extract features from the reference character image and the preprocessed copy character image, and calculate the distance between the features. If the distance is greater than a set second distance threshold, the writing content is considered to be inconsistent with the reference character, that is, the student's copy character and the reference character are not the same character.

[0093] In this embodiment, the normalized distance of character features is used as the consistency evaluation index through the character recognition algorithm model. The handwriting consistency evaluation module is used to initially determine whether the input student's calligraphy practice work and the reference character are the same character. Specifically, through the TPN-based Chinese character recognition algorithm, the preprocessed student's calligraphy practice work (i.e., the preprocessed copying character image) and the reference character (i.e., the reference character image) are input into the character recognition algorithm model to obtain the last layer of feature data. The feature data corresponding to the reference character image is used as the first feature, and the feature data corresponding to the preprocessed copying character image is used as the second feature. L2 normalization is performed on the first and second features, and the Euclidean distance between the normalized first and second features is calculated as the consistency difference between the student's calligraphy practice work and the reference character. When the difference value exceeds the set second distance threshold (preferably set to 0.9 in this invention), the writing content and the reference character are considered to be inconsistent.

[0094] Before conducting stroke assessment, an assessment strategy should be developed.

[0095] The calligraphy character assessment strategy module is configured to construct rules for calligraphy character assessment. Based on the general methods of calligraphy character evaluation in calligraphy teaching, it is refined and decomposed into comprehensive, multi-faceted, fine-grained, and quantifiable assessment requirements, defined as various strategy rules. Each strategy includes different combinations of stroke assessment and comprehensive information from reference characters.

[0096] Assessment strategies are used to construct general methods for assessment. By analyzing the general experience and approaches to calligraphy assessment in years of calligraphy teaching, and combining calligraphy knowledge with computer knowledge, abstract calligraphy assessment ideas are summarized into quantifiable assessment rules that can be calculated. The rules mainly include two parts: single-stroke assessment strategies and overall structural component assessment strategies. The format of the strategy is: (Assessment Index: Difference Threshold), where the difference threshold refers to the difference between the student's calligraphy practice work and the reference character, such as (Stroke Length: 0.8). Single-stroke assessment strategies include assessment indicators such as "stroke length," "curvature," "thickness," "slant," "starting and ending stroke outline," and "excessive or omitted strokes." Overall structural component assessment strategies include assessment strategy indicators such as "overall size," "position," "component structure," and "character shape."

[0097] The stroke evaluation module is configured to compare the strokes of each calligraphy character in the preprocessed copy image extracted by the stroke extraction module with the strokes of each calligraphy character in the reference image based on set stroke morphological features. If the distance between the features is greater than a set threshold, there is a corresponding stroke morphological writing problem. The set morphological features include stroke length, curvature, thickness, slant, starting and ending stroke outlines, and whether strokes are written in excess or omitted.

[0098] In this embodiment, based on the stroke evaluation strategy of the calligraphy evaluation strategy, the morphological features of multiple aspects of the strokes are calculated. Based on these morphological features, the differences between the strokes of the work and the strokes of the reference character are calculated one by one, and quantitative evaluation results are given. Specifically, combining the stroke information of the reference character and the stroke evaluation part of the calligraphy evaluation strategy, the normalized differences between the strokes and the corresponding reference character strokes are calculated for each stroke based on multiple evaluation strategy indicators, including stroke length, curvature, thickness, slant, starting and ending stroke outlines, and whether any strokes are written incorrectly or omitted. According to the marked key strokes of the calligraphy character, for each key stroke, the normalized differences between the positional relationship features of the key stroke and other strokes and the features of the reference character are calculated. The details are as follows:

[0099] Single stroke tilt: Calculate the minimum bounding rectangle of the stroke skeleton image, and take the angle between the long side and the horizontal axis as the stroke tilt angle; calculate the average tilt angle of all strokes as the overall tilt, and subtract the overall tilt from the tilt of each stroke to obtain the final tilt of a single stroke; calculate the error between the final tilt of a single stroke and the corresponding reference character stroke; if the error exceeds the threshold, there is a problem; the stroke skeleton image is the skeleton image corresponding to the strokes of each calligraphy character in the preprocessed copy character image extracted by the stroke extraction module.

[0100] Single stroke thickness: The ratio of the average thickness of each stroke in the stroke skeleton image to the size of the whole character is calculated as the quantized thickness value of the single stroke. The error between this quantized thickness value and the quantized thickness value of the corresponding stroke of the reference character is calculated. If it exceeds the threshold, there is a problem.

[0101] Single stroke length: The ratio of the length of each stroke in the stroke skeleton image to the size of the whole character is calculated as the length value. The error between this length value and the length value of the corresponding stroke of the reference character is calculated. If it exceeds the threshold, there is a problem.

[0102] Single stroke straightness: In the stroke skeleton image, a set number of pixels are uniformly sampled on each stroke, and the average angle transformation rate of these pixels is calculated as the straightness. The error between this straightness and the straightness of the corresponding stroke of the reference character is calculated. If it exceeds the threshold, there is a problem.

[0103] Starting and ending strokes of a single stroke: Taking the two endpoints of each stroke in the stroke skeleton image as the center, the starting and ending stroke images are extracted. The category of the starting and ending strokes is obtained through a pre-trained contour recognition model. When the category is different from the category of the stroke corresponding to the reference character, there is a problem. The categories of the starting and ending strokes include square, round, pointed, and others. The contour recognition model is constructed based on sequentially connected convolutional layers and fully connected layers.

[0104] Missed and overwritten: During the registration of the second point set with the third point set using the CPD point set registration method, the average migration distance of each point in the second point set before and after registration is calculated to obtain the average migration distance. If the average migration distance is greater than a set threshold, there is a missed write; the migration distance is the Euclidean distance between the positions of each point in the second point set before and after registration.

[0105] All detected strokes are obtained and compared with the entire character being copied. The extra parts in the entire character are the areas of extra strokes written.

[0106] The structural component evaluation module is configured to compare the strokes of each calligraphy character in the preprocessed copy image extracted by the stroke extraction module with the strokes of each calligraphy character in the reference image based on the set structural component morphological features. If the distance between the features is greater than a set threshold, there is a corresponding component morphological writing problem. The set structural component morphological features include key stroke layout, component tightness, overall tightness, and component relative position.

[0107] In this embodiment, based on the structural component rules of the calligraphy evaluation strategy, the morphological features of multiple aspects of the component structure are calculated. The differences between these morphological features and those of the reference character are compared, and quantitative evaluation results are given. Overall size, position, and shape deviations are calculated by obtaining the overall circumscribed rectangle, and the quantitative deviations of the student's calligraphy practice work and the reference character's outer rectangle in these aspects are calculated.

[0108] The assessment of calligraphy character structure is the focus of calligraphy character evaluation. Based on calligraphy teaching experience, and combined with the marked key strokes and calligraphy character component information, calculations and assessments are carried out from four aspects: key stroke layout deviation, component internal tightness deviation, overall tightness deviation, and component relative position deviation.

[0109] Key stroke layout: Extract the strokes marked as key strokes in the reference character image, calculate the ratio of the Euclidean distance between the key strokes and the surrounding strokes in each calligraphy character of the preprocessed copy character image to the whole character size, and calculate the deviation of this ratio from the corresponding ratio of the reference character (i.e., the ratio of the corresponding key strokes in the reference character to the surrounding strokes (set according to actual needs) and the whole character size (the whole character size of each calligraphy character in the reference character image)). If the deviation is greater than the set first deviation threshold, there is a key stroke layout deviation problem.

[0110] Component tightness: Extract the centroid of the component corresponding to each calligraphic character in the reference character image, and calculate the ratio of the distance from the stroke of each calligraphic character in the reference character image to the centroid of the corresponding component to the component size, as the first centroid distance; the component includes a radical and a part composed of multiple strokes (i.e., the component is composed of multiple strokes, which can be pre-marked by annotating the reference character; there is no limit, and it can be composed of any number of strokes. Generally, based on calligraphy teaching experience, those strokes that need to be checked on the component will be marked accordingly).

[0111] Extract the centroid of the corresponding component of each calligraphy character in the preprocessed copying image, and calculate the ratio of the distance from the stroke of each calligraphy character to the centroid of the corresponding component to the component size, as the second centroid distance; calculate the deviation between the first centroid distance and the second centroid distance. If the deviation is greater than the set second deviation threshold, there is a component tightness deviation problem.

[0112] Overall tightness: Extract the centroid of each corresponding component of each calligraphy character in the preprocessed copying image, calculate the ratio of the average relative distance between each pair of centroids to the size of the whole character, and use it as the tightness. Calculate the deviation between this tightness and the tightness of each calligraphy character component in the reference image. If it exceeds the set third deviation threshold, there is an overall tightness problem.

[0113] Component relative position: The component closest to the center of the reference character image is taken as the reference component. The ratio of the centroid distance between other components and the reference component in the preprocessed copy character image to the size of the whole character is calculated as the relative position of other components to the reference component. The deviation between this relative position and the relative position of each calligraphy component in the reference character image is calculated. When it exceeds the set fourth deviation threshold, there is a component relative position problem.

[0114] The whole-character qualitative assessment module is configured to sort the writing problems evaluated by the writing consistency assessment module, the stroke assessment module, and the structural component assessment module according to a set priority, and locate them in the corresponding area of ​​the student's practice work; based on the priority and the area location information of the problems, a qualitative assessment is performed, thereby obtaining the assessment result of the student's practice work.

[0115] In this embodiment, the whole-character qualitative assessment module comprises three units: calligraphy problem sorting, problem localization, and qualitative evaluation. It is used to convert the algorithm evaluation results into assessment information suitable for teaching.

[0116] Writing Problem Ranking Unit: The writing problems are ranked according to the rules, highlighting serious problems. Specifically, the calligraphy problem ranking is based on the severity of the detected calligraphy problems, with component structure problems first, and the problems are arranged in descending order of severity; the rules for stroke problems are as follows: (1) Problems of writing too many / missing strokes have the highest priority; (2) The weights of the indicators of length / straightness / thickness / inclination are the same, and they are arranged in descending order of the severity of the stroke problems (i.e., the more serious the problem, the higher the ranking). When the quantitative scores of the indicators are the same, they are ranked according to the writing order of the strokes.

[0117] Problem localization unit: Combining stroke assessment, it determines the location of whole-character writing problems, i.e., the problematic strokes. Specifically, problem localization is used to visualize the component structure and stroke areas with assessment problems. For the assessment problems selected after sorting, the assessment problems can be traced back to the approximate area of ​​the student's calligraphy practice work according to the morphological characteristics in the assessment strategy. The outline points of the approximate area are collected as the localization of the calligraphy assessment problem.

[0118] Qualitative Assessment Unit: Combining the severity ranking and location information of the problems mentioned above, a qualitative assessment is constructed. Specifically, the qualitative assessment unit, based on the calligraphy problem ranking results and location information, selects the more serious assessment problems and summarizes the complex calligraphy assessment results into an ordered sequence of results. Each calligraphy assessment result includes the assessment problem name, assessment result, assessment quantifiable value, problem location area, and score. The specific calculation process for the score is as follows:

[0119] The total score of the student's practice work set by the calligraphy copying evaluation system is obtained as the first score; based on the writing problem deduction rule table, the priority of writing problems in the student's practice work and the regional location information of the problems are matched to obtain the score deducted for writing problems, and the sum is obtained as the second score.

[0120] Subtract the second score from the first score to get the final score of the student's work, which is the evaluation result of the student's work.

[0121] The writing problem deduction rule table is a mapping relationship between the preset writing problem priority and the regional location information of the writing problem and the corresponding writing problem deduction points.

[0122] In addition, the calligraphy copying evaluation system of the present invention also includes an output module;

[0123] The output module is configured to integrate the evaluation results of the writing consistency evaluation module, stroke evaluation module, structural component evaluation module, and student practice works to form a standardized output in JSON format.

[0124] It should be noted that the calligraphy copying evaluation system provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be further decomposed or combined. For example, the modules in the above embodiments can be merged into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the various modules or steps and are not considered as an improper limitation of the present invention.

[0125] The following is for reference. Figure 4 It shows a schematic diagram of the structure of a computer system suitable for implementing the methods, systems, and apparatus embodiments of this application. Figure 4 The server shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0126] like Figure 4 As shown, the computer system includes a Central Processing Unit (CPU) 401, which can perform various appropriate actions and processes based on programs stored in Read Only Memory (ROM) 402 or programs loaded from storage section 408 into Random Access Memory (RAM) 403. RAM 403 also stores various programs and data required for system operation. The CPU 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0127] The following components are connected to I / O interface 405: an input section 306 including a keyboard, mouse, etc.; an output section 407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to I / O interface 405 as needed. A removable medium 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 410 as needed so that computer programs read from it can be installed into storage section 408 as needed.

[0128] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by central processing unit (CPU) 401, it performs the functions defined in the methods of this application. It should be noted that the computer-readable medium described above in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0129] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0130] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0131] The terms “first” and “second” are used to distinguish similar objects, rather than to describe or indicate a specific order or sequence.

[0132] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent in such process, method, article, or apparatus / device.

[0133] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A calligraphy character imitation evaluation system, characterized by comprising: The system comprises an image acquisition module, a preprocessing module, a stroke extraction module, a writing consistency evaluation module, a stroke evaluation module, a structure component evaluation module, and a whole word qualitative evaluation module. The image acquisition module is configured to acquire a copy word image and a reference word image; the reference word image is each calligraphy word image in a standard calligraphy work with known strokes, radicals, and writing order; and the copy word image is each calligraphy word image in a student's practice work for copying the standard calligraphy work. The preprocessing module is configured to input the copy word image into a pre-constructed image preprocessing network, perform binarization, image size and angle correction processing on the copy word image, and obtain a pre-processed copy word image; and the image preprocessing network is constructed based on a deep learning neural network. The stroke extraction module is configured to acquire stroke contour point sets of calligraphy words in the pre-processed copy word image and the reference word image, and obtain strokes of each calligraphy word in the pre-processed copy word image through a stroke extraction method based on two-stage CPD point set registration. The writing consistency evaluation module is configured to extract features of the reference word image and the pre-processed copy word image, calculate the distance between the features, and if the distance is greater than a set second distance threshold, it is considered that the writing content and the reference word are not consistent, i.e., the student's copy word and the reference word are not the same word. The stroke evaluation module is configured to compare the strokes of each calligraphy word in the pre-processed copy word image extracted by the stroke extraction module with the strokes of each calligraphy word in the reference word image from a set of stroke shape features one by one, and if the distance between the features is greater than a set threshold, there is a corresponding stroke shape writing problem; and the set shape features include stroke length, curvature, thickness, inclination, stroke outline, and stroke omission. The structure component evaluation module is configured to compare the strokes of each calligraphy word in the pre-processed copy word image extracted by the stroke extraction module with the strokes of each calligraphy word in the reference word image from a set of structure component shape features one by one, and if the distance between the features is greater than a set threshold, there is a corresponding component shape writing problem; and the set structure component shape features include key stroke layout, component tightness, overall tightness, and component relative position. The whole word qualitative evaluation module is configured to sort the writing problems evaluated by the writing consistency evaluation module, the stroke evaluation module, and the structure component evaluation module according to a set priority, and locate to the corresponding area of the student's practice work; and perform qualitative evaluation according to the priority and the area positioning information of the problem, and further obtain the evaluation result of the student's practice work.

2. The calligraphy character imitation evaluation system according to claim 1, wherein The image preprocessing network is constructed based on a ResBlock, an ASPP module, an up-convolution module, and an STN Block; the up-convolution module is constructed based on multiple convolution layers and up-sampling layers; and the convolution layers and the up-sampling layers are arranged alternately in the up-convolution module, with the convolution layers in front of the up-sampling layers. The ResBlock is connected with the ASPP module and the STN Block respectively, and the input of the ResBlock is the calligraphy image; The ASPP module is connected with the up-convolution module; the ResBlock is jump-connected with the up-convolution module, when the jump connection, the output of the ResBlock is The convolution layer of the up-convolution module is connected with the convolution layer with N The jump connection is performed on the convolution layers with the same size; the up-convolution module is connected with the STN Block; the output of the STN Block is the preprocessed imitation word image.

3. The calligraphy character imitation evaluation system according to claim 1, wherein The calligraphy image is input into a pre-constructed image preprocessing network for binarization, image size and angle correction processing to obtain a pre-processed calligraphy image, and the method is as follows: The calligraphy image is input into the ResBlock of the image preprocessing network for residual processing, and the features after residual processing are taken as first features; The first features are input into the ASPP of the image preprocessing network for processing, and then up-convolution is performed, the features input by the ResBlock jump connection are combined, and then the binarization processing result corresponding to the calligraphy image is obtained; The first features are input into the CNN of the STN Block of the image preprocessing network to obtain corresponding projective transformation parameters; The binarization processing result and the projective transformation parameters are input into the space conversion module of the STN Block for space conversion, and then the pre-processed calligraphy image is obtained.

4. The calligraphy character imitation evaluation system according to claim 1, wherein The stroke contour point sets of the calligraphy characters in the pre-processed calligraphy image and the reference character image are obtained, and a stroke extraction method based on a two-stage CPD point set registration is used to obtain the strokes of each calligraphy character in the pre-processed calligraphy image, and the method is as follows: The stroke contour point sets of the calligraphy characters in the reference character image and the pre-processed calligraphy image are registered by a CPD point set registration method to obtain the stroke contour point sets of the calligraphy characters in the reference character image after stroke transformation, which are taken as first point sets; For each stroke contour point set in the first point set, one stroke contour point set is extracted as a second point set in order; the second point set is traversed, and the stroke contour points in the pre-processed calligraphy image whose Euclidean distance from the contour points in the second point set is greater than a set first distance threshold are removed, and the pre-processed calligraphy image after the removal of the stroke contour points is taken as a third point set; The third point set and the second point set are registered by a CPD point set registration method to obtain a transformed second point set, and the transformed second point set is taken as the strokes of the calligraphy characters in the pre-processed calligraphy image; The next stroke contour point set in the first point set is continuously extracted in order, and the removal and registration processing are performed until all the stroke contour point sets in the first point set are processed.

5. The calligraphy character imitation evaluation system according to claim 1, wherein The features of the reference character image and the pre-processed calligraphy image are extracted, and the distance between the features is calculated; if the distance is greater than a set second distance threshold, it is considered that the writing content and the reference character do not have consistency, that is, the calligraphy of the student and the reference character are not a character, and the method is as follows: The reference character image and the preprocessed copy character image are input into a pre-constructed character recognition algorithm model, features corresponding to the reference character image and the preprocessed copy character image output by the last layer of the character recognition algorithm model are obtained as first features and second features, and L2 normalization processing is performed; a Euclidean distance between the first features and the second features after normalization processing is calculated, and it is determined whether the Euclidean distance exceeds a second distance threshold value; if the Euclidean distance exceeds the second distance threshold value, it is considered that the writing content and the reference character do not have consistency.

6. The calligraphy character imitation evaluation system according to claim 4, wherein The strokes of each calligraphy character in the preprocessed copy character image extracted by the stroke extraction module and the strokes of each calligraphy character in the reference character image are compared one by one from a set stroke feature; if a distance between the features is greater than a set threshold value, there is a corresponding stroke form writing problem, and the method is as follows: Single stroke inclination: a minimum circumscribed rectangle of a stroke skeleton image is calculated, an angle between a long side direction and a horizontal axis is taken as a stroke inclination angle; an average value of inclination angles of all strokes is calculated as an overall inclination; a single stroke inclination is obtained by subtracting the overall inclination from an inclination of each stroke; an error between the single stroke inclination and a final inclination of a corresponding reference character stroke is calculated; if the error exceeds a threshold value, there is a problem; the stroke skeleton image is a skeleton image corresponding to each stroke of the preprocessed copy character image extracted by the stroke extraction module; Single stroke thickness: a ratio between an average thickness of each stroke in the stroke skeleton image and a size of a whole character is calculated as a quantitative thickness value of a single stroke; an error between the quantitative thickness value and a quantitative thickness value of a corresponding stroke of a reference character is calculated; if the error exceeds a threshold value, there is a problem; Single stroke length: a ratio between a length of each stroke in the stroke skeleton image and a size of a whole character is calculated as a length value; an error between the length value and a length value of a corresponding stroke of a reference character is calculated; if the error exceeds a threshold value, there is a problem; Single stroke curvature: a set number of pixel points are uniformly sampled on each stroke in the stroke skeleton image; an average angle transformation rate of the pixel points is calculated as a curvature; an error between the curvature and a curvature of a corresponding stroke of a reference character is calculated; if the error exceeds a threshold value, there is a problem; Single stroke starting and ending: starting and ending images are obtained by taking two endpoints of each stroke in the stroke skeleton image as centers; a category of starting and ending is obtained by a pre-trained contour recognition model; if the category is different from a category of a corresponding stroke of a reference character, there is a problem; the category of starting and ending includes square, circle, sharp, and other; the contour recognition model is constructed based on sequentially connected convolution layers and fully connected layers; Omission and overwriting: in a process of registering the second point set and the third point set by a CPD point set registration method, average migration distances of each point of the second point set before and after registration are calculated to obtain average migration distances; if the average migration distances are greater than a set threshold value, there is omission; the migration distance is a Euclidean distance between positions of each point in the second point set before and after registration; all strokes detected are compared with a whole character of a copy character; a part that is extra in the whole character is a stroke area of overwriting.

7. The calligraphy character imitation evaluation system according to claim 6, wherein The strokes of each calligraphy character in the pre-processed copy image extracted by the stroke extraction module are compared with the strokes of each calligraphy character in the reference image from the set structural component feature. If the distance between the features is greater than the set threshold, there is a corresponding component form writing problem. The method is as follows: Key stroke layout: the strokes marked as key strokes in the reference image are extracted. The ratio of the Euclidean distance between the key strokes and the surrounding strokes in the pre-processed copy image to the size of the whole character is calculated. The deviation of the ratio and the corresponding ratio in the reference image is calculated. If the deviation is greater than the set first deviation threshold, there is a key stroke layout deviation problem. Component tightness: the centroid of each component of each calligraphy character in the reference image is extracted. The ratio of the distance between the strokes and the centroid of the corresponding component to the size of the component is calculated as the first centroid distance. The component includes the component and the part composed of a plurality of strokes. The centroid of each component of each calligraphy character in the pre-processed copy image is extracted. The ratio of the distance between the strokes and the centroid of the corresponding component to the size of the component is calculated as the second centroid distance. The deviation between the first centroid distance and the second centroid distance is calculated. If the deviation is greater than the set second deviation threshold, there is a component tightness deviation problem. Overall tightness: the centroid of each component of each calligraphy character in the pre-processed copy image is extracted. The ratio of the average distance between the centroids to the size of the whole character is calculated as the tightness. The deviation of the tightness and the tightness of each component of each calligraphy character in the reference image is calculated. If the deviation exceeds the set third deviation threshold, there is an overall tightness problem. Component relative position: the component closest to the center of the reference image is taken as the reference component. The relative position of other components to the reference component is calculated as the ratio of the centroid distance between the other components and the reference component to the size of the whole character. The deviation of the relative position and the relative position of each component of each calligraphy character in the reference image is calculated. If the deviation exceeds the set fourth deviation threshold, there is a component relative position problem.

8. The calligraphy character imitation evaluation system according to claim 1, wherein According to the priority and the regional positioning information of the problem, qualitative evaluation is carried out to obtain the evaluation result of the student's practice work. The method is as follows: The total score of the student's practice work set by the calligraphy character copy evaluation system is obtained as the first score. Based on the writing problem deduction rule table, the priority and the regional positioning information of the student's practice work are matched to obtain the score deducted by the writing problem and the sum as the second score. The first score minus the second score is the final score of the student's practice work, which is the evaluation result of the student's practice work. The writing problem deduction rule table is a pre-set mapping relationship between the priority of the writing problem, the regional positioning information of the writing problem, and the corresponding deduction score.

9. The calligraphy character imitation evaluation system according to claim 1, wherein The calligraphy character copy evaluation system also includes an output module. The output module is configured to integrate the evaluation results of the writing consistency evaluation module, the stroke evaluation module, the structural component evaluation module and the student practice work to form a standardized output in JSON format.

Citation Information

Patent Citations

  • Target grid and copy copybook for calligraphy practice, and calligraphy copy comparison and evaluation system and comparison and evaluation steps based on target grid

    CN114141099A

  • Information processing device, character recognition method, and character recognition program

    JP2020013460A