Part-based handwritten Chinese character evaluation method

Abstract

The invention discloses a handwritten Chinese character evaluation method based on components, and relates to computer graphics. Comprising the following steps: 1) carrying out random movement and size change adjustment on part of parts of given handwritten Chinese characters; 2) using unchanged standard layout handwritten Chinese characters and the data generated in the step 1) as a data set, and pre-training a Chinese character component reconstruction module; 3) extracting layout characteristics of handwritten Chinese characters by using a pre-trained Chinese character component reconstruction module; 4) performing handwritten Chinese character layout evaluation by taking the layout characteristics of the handwritten Chinese characters as input and combining a layout evaluation module to obtain an evaluation conclusion and a modification suggestion; according to the method, the defects of the prior art in the aspects of Chinese character evaluation modes and evaluation result display are overcome, meanwhile, the period of traditional manual design evaluation templates is effectively shortened, the cost is reduced, and the method has wide application prospects in the fields of calligraphy education, font design, character recognition and the like.

Description

Technical Field

[0001] The invention relates to the research field of computer graphics, and in particular to a component-based handwritten Chinese character evaluation method. Background Art

[0002] The art of calligraphy is based on Chinese characters, which are ideographic characters with unique regularity and aesthetic characteristics in structure. Calligraphers use pen and ink to depict the beauty of the structure and layout of Chinese characters, showing the rhythm and beauty of Chinese characters. Therefore, evaluating the layout structure of Chinese characters is of great significance to the art of calligraphy.

[0003] Computational evaluation of handwritten Chinese characters is a method for computationally evaluating the aesthetic quality of calligraphy, which provides substantial convenience for Chinese character writing education. Handwriting evaluation plays a pivotal role in many fields such as Chinese handwriting generation and Chinese character beautification. However, previous studies have rarely explored the use of Chinese character decomposition and layout reconstruction to achieve automatic evaluation of Chinese characters. Most early calligraphy evaluation methods used template matching to measure the visual features of the target character, such as stroke length, width, stroke angle, number of strokes, and stroke order, by comparing it with the template character. However, these methods are greatly affected by the selection of template characters, and corresponding evaluation criteria must be formulated for each template character, making the evaluation rules too complicated.

[0004] With the development of deep learning, a possible solution to calligraphy evaluation is to use deep learning methods such as convolutional neural networks. However, compared with other languages, Chinese characters have a unique structure because they can be considered to be composed of multiple components. The shapes and relationships of these components together affect the aesthetics of Chinese characters. Therefore, the evaluation of Chinese characters must consider the rationality of their internal structure. Summary of the invention

[0005] The purpose of the present invention is to provide a component-based handwritten Chinese character evaluation method to address the problem that the prior art fails to consider the relationship between components when evaluating handwritten Chinese characters. The method aims to simulate the layout problem of handwritten Chinese characters, use deep learning technology to perform an overall evaluation of the components of handwritten Chinese characters, and give corresponding evaluation conclusions and modification suggestions.

[0006] The present invention comprises the following steps:

[0007] 1) Generate handwritten Chinese characters with layout problems: perform “random movement” and “size change” adjustments on the components of a given handwritten Chinese character to obtain handwritten Chinese character data with layout problems;

[0008] 2) Pre-training Chinese character component reconstruction module: Use the unchanged standard layout handwritten Chinese characters and the data generated in step 1) as the data set to pre-train the Chinese character component reconstruction module; the input of the Chinese character component reconstruction module is a multi-channel image, including standard layout handwritten Chinese characters and problem layout handwritten Chinese characters, and the network output is the features of the problem layout handwritten Chinese character components and the reconstructed standard layout handwritten Chinese characters;

[0009] 3) Extracting layout features: Using the Chinese character component reconstruction module pre-trained in step 2), the layout features of handwritten Chinese characters are extracted;

[0010] 4) Conduct layout evaluation: Taking the layout features of handwritten Chinese characters as input, the layout evaluation module is used to conduct layout evaluation of handwritten Chinese characters, and based on the pre-set classification criteria, evaluation conclusions and modification suggestions are obtained;

[0011] 5) Adjustment based on evaluation suggestions: Based on the evaluation conclusions and modification suggestions obtained in step 4), the handwritten Chinese characters are adjusted according to pre-set classification standards, including position adjustment and size adjustment of components.

[0012] In step 1), the given handwritten Chinese character components refer to the first-level 3755 commonly used Chinese characters in GB2312_80, and each handwritten Chinese character stroke is saved independently.

[0013] In step 1), the "random movement" refers to randomly moving one or more components in the handwritten Chinese character, thereby changing the layout of the handwritten Chinese character components.

[0014] In step 1), the “size change” refers to adjusting the size of one or more components in the handwritten Chinese character.

[0015] In step 2), the unchanged standard layout handwritten Chinese characters refer to the original handwritten Chinese characters that have not been adjusted by “random movement” and “size change”.

[0016] In step 3), the layout features of the handwritten Chinese characters are extracted by sending the input handwritten Chinese characters to the Chinese character component reconstruction module to obtain the layout features of the problematic handwritten Chinese characters, and then subtracting the layout features of the problematic handwritten Chinese characters from the layout features of the standard handwritten Chinese characters to obtain the image features.

[0017] In step 4), the input of the layout evaluation module is the layout features of the handwritten Chinese characters. In the layout evaluation module, the positional relationship between each component of the handwritten Chinese characters is classified one by one until the classification of the relationship between all components is completed, and corresponding evaluation conclusions and text suggestions are given.

[0018] In step 4), the pre-set classification criteria include: (1) appropriate position compared with other parts; (2) higher position compared with other parts; (3) lower position compared with other parts; (4) left position compared with other parts; (5) right position compared with other parts; (6) smaller than other parts; (7) larger than other parts.

[0019] Compared with the prior art, the present invention has the following outstanding technical effects:

[0020] The present invention proposes a component-based Chinese character evaluation method, which evaluates handwritten Chinese characters according to their component relationships and gives evaluation conclusions and modification suggestions. First, a reconstruction module for the structure and layout of Chinese characters is proposed, and reconstruction features are used instead of template words, so that various handwriting styles can be evaluated without extracting a large number of template word features; second, a handwritten Chinese character evaluation method based on layout reconstruction is proposed, and a Chinese character component layout evaluation module based on graph structure is designed; the layout structure of handwritten Chinese characters can be evaluated more accurately; third, in view of the problem that the use of scores to evaluate handwritten Chinese characters is not specific and the use of template matching to evaluate Chinese characters is not flexible, the method of comment evaluation and modification suggestions is adopted, which has great practicality for calligraphy teaching. The present invention realizes the intelligent evaluation of the component relationship of handwritten Chinese characters by comprehensively applying preprocessing, feature extraction, component layout evaluation and other steps, provides an efficient solution for the evaluation of handwritten Chinese characters, and is of great significance to the application of computer graphics. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 The figure is a flow chart of the method of the present invention.

[0022] Figure 2 The figure is a schematic diagram of the handwritten Chinese character component movement method, where: a. the original character and the corresponding IDS; b. random movement of components; c. resizing of components.

[0023] Figure 3 Reconstruct the module diagram for Chinese character components.

[0024] Figure 4 The diagram of the handwritten Chinese character layout evaluation module is shown in Figure 1. a. The component sequence is obtained after the handwritten Chinese character is segmented; b. The component layout is reconstructed to obtain the handwritten Chinese character spatial features; c. The handwritten Chinese character spatial features are input into the graph network for layout relationship prediction.

[0025] Figure 5 This is an example of modifying handwritten Chinese characters based on the evaluation conclusions. DETAILED DESCRIPTION

[0026] The following description of at least one exemplary embodiment is merely illustrative and in no way restrictive of the present invention or its application or use. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0027] The present invention constructs a handwritten Chinese character layout evaluation and adjustment system, which can automatically identify the layout problems of handwritten Chinese characters and provide specific adjustment suggestions. By combining deep learning technology and image processing algorithms, this system will provide users with convenient and efficient Chinese character writing guidance.

[0028] The system mainly consists of the following modules:

[0029] The data preprocessing module is used to generate a dataset of handwritten Chinese characters with layout problems and preprocess the dataset.

[0030] The Chinese character component reconstruction module is used for pre-training to learn the layout features of handwritten Chinese characters.

[0031] The layout feature extraction module is used to extract the layout features of the handwritten Chinese characters to be evaluated.

[0032] The layout evaluation module is used to evaluate the extracted layout features and give adjustment suggestions.

[0033] The adjustment execution module is used to adjust the handwritten Chinese characters according to the evaluation suggestions.

[0034] As Figure 1 , the component-based handwritten Chinese character evaluation method according to the embodiments of the present invention includes the following steps:

[0035] 1) Generate handwritten Chinese characters with layout problems: Adjust the components of the given handwritten Chinese characters by "random movement" and "size change" to obtain handwritten Chinese characters with layout problems:

[0036] Step 1: Read the SVG format handwritten Chinese characters with stroke markings and the corresponding ideographic character description sequences of the handwritten Chinese characters. These stroke marking ideographic character sequences are used for subsequent recognition and operation of Chinese character components. Figure 2 Taking the character "陷" as an example in

[0037] Step 2: Randomly select one of the components of the read handwritten Chinese character as the adjustment object.

[0038] Step 3: Move the selected component or adjust the size of the component. The moving distance and the size change ratio should be randomly set within a reasonable range to ensure the diversity and authenticity of the generated handwritten Chinese characters with layout problems. (See Figure 2 b in Figure 2 c) in.

[0039] 2) Use the unchanged standard layout handwritten Chinese characters and the data generated in step 1) as the dataset to pre-train the Chinese character component reconstruction module:

[0040] Step 1: Establish a Chinese character component reconstruction module, which should have the ability to recognize and reconstruct handwritten Chinese characters with standard layout and problem layout. Figure 3 As shown, construct a layout reconstruction encoder Extract features of handwritten Chinese characters with layout problems and obtain layout features . Using a pre-trained encoder Extract features of standard layout handwritten Chinese characters and problem layout handwritten Chinese characters and In addition, using a component classifier To determine whether the components of the handwritten Chinese characters are accurately contained in the content features, use the reconstruction decoder Decode the features of the Chinese characters in the standard layout to obtain the reconstructed standard Chinese characters. Reconstruction encoder , the pre-trained encoder , component classifier , reconstruct the decoder The model structure can refer to the convolutional neural network (CNN) in deep learning.

[0041] Step 2: Use standard layout handwritten Chinese characters and problem layout handwritten Chinese characters as training data, where the problem layout handwritten Chinese characters are generated by step 1); use character layout loss as the loss function to train the Chinese character component reconstruction module, where the input includes standard layout handwritten Chinese character images, problem layout handwritten Chinese character images, Chinese character component sequences, and Chinese character names.

[0042] The calculation of character layout loss should comprehensively consider the character loss and component loss in the character encoding and decoding process to ensure that the features in the handwritten character encoding process are consistent with their corresponding standard character features. The VGG16 network model is used as the feature extractor.

[0043] The calculation formula of character layout loss is as follows:

[0044]

[0045] in, Indicates the loss of characters during character encoding and decoding; Indicates component loss during character encoding and decoding; Used to penalize the inconsistency between VGG features and their corresponding standard character features during handwritten character encoding; , and is a pre-set hyperparameter, , , .

[0046] Character layout loss , and The calculation formula is:

[0047]

[0048]

[0049]

[0050] in, represents the Chinese character predicted by the Chinese character component reconstruction module; Indicates the label value of the Chinese character; represents the Chinese character component sequence predicted by the Chinese character component reconstruction module; The tag value representing the Chinese character component sequence; Represents the VGG16 network model; Indicates the problem layout handwritten Chinese character picture, Represents a picture of handwritten Chinese characters in a standard layout.

[0051] 3) Use the Chinese character component reconstruction module pre-trained in step 2) to extract the layout features of handwritten Chinese characters:

[0052] Step 1: Input a handwritten Chinese character image and its name.

[0053] Step 2: Input the handwritten Chinese character image and name into the pre-trained Chinese character component reconstruction module.

[0054] Step 3: The Chinese character component reconstruction module extracts the handwritten Chinese character image and reconstructs the encoder through layout Image features , and with its pre-trained encoder The obtained image features Subtract and obtain the layout features of the processed handwritten Chinese characters.

[0055] 4) Using the layout features of handwritten Chinese characters as input, the layout evaluation module is combined to evaluate the layout of handwritten Chinese characters, and the evaluation conclusions and modification suggestions are obtained according to the pre-set classification standards (such as Figure 4 ):

[0056] Step 1: Establish a layout relationship prediction network. Figure 4The spatial features of handwritten Chinese characters reconstructed after component layout shown in b are combined with the middle-level structural features of the graph neural network into a message matrix of a node. This matrix is ​​used to represent the spatial relationship and structural features between Chinese character components and is defined as:

[0057]

[0058] in, For structural characteristics, is the spatial feature, is a hyperparameter, set to 0.5, and the component nodes in the graph neural network are defined as nodes of The dimension indicates that the feature update of each node is consistent with that of its neighboring nodes. Related, feature updates are performed through learnable weights and biases, and the update formula is:

[0059]

[0060]

[0061]

[0062] in, is the message matrix, is a hyperparameter, set to 0.4; is the degree matrix, is the adjacency matrix; is the characteristic of layer t; are learnable weights, are learnable biases, which are initialized in Gaussian random form; the function Accumulate the information transmitted by the graph neural network layer to update the radical representation, and the graph network performs layout relationship prediction such as Figure 4 As shown in c.

[0063] Step 2: The layout relationship prediction network predicts the relationship categories between the components of handwritten Chinese characters based on the layout features of handwritten Chinese characters. These relationship categories include (1) appropriate position compared to other components; (2) higher position compared to other components; (3) lower position compared to other components; (4) to the left of other components; (5) to the right of other components; (6) smaller than other components; and (7) larger than other components.

[0064] Step 3: The layout evaluation module converts the predicted category information according to a predefined evaluation method, and converts it into specific evaluation conclusions and modification suggestions.

[0065] Step 4: Output evaluation conclusions and modification suggestions. The evaluation conclusions clearly point out the problems in the layout of handwritten Chinese characters, and the modification suggestions provide specific adjustment directions and steps.

[0066] 5) Use the layout evaluation suggestions to adjust the handwritten Chinese characters:

[0067] Step 1: With the help of the evaluation conclusions and modification suggestions output by the layout evaluation module, adjust the position or size of the problematic components of the handwritten Chinese characters. During the adjustment process, ensure that the relative position and size relationship between the components conform to the Chinese character writing standards.

[0068] Step 2: Verify the adjusted handwritten Chinese characters to ensure that the adjusted layout meets the requirements of the evaluation suggestions. If necessary, multiple adjustments and verifications can be performed until a satisfactory layout effect is achieved.

[0069] Figure 5 An example of modifying handwritten Chinese characters based on the evaluation conclusion is given. By comparing the handwritten Chinese characters before and after the adjustment, it can be clearly seen that the layout of the adjusted Chinese characters is more reasonable and beautiful. Figure 5 The examples in also demonstrate the effectiveness and accuracy of this method in recognizing and adjusting the layout of handwritten Chinese characters.

[0070] The present invention uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A component-based handwritten Chinese character evaluation method, characterized in that include: 1) Perform "random movement" and "size change" adjustments on the components of a given handwritten Chinese character to obtain a handwritten Chinese character with layout problems; 2) Using the unchanged standard layout handwritten Chinese characters and the data generated in step 1) as a dataset, pre-training the Chinese character component reconstruction module; 3) Using the Chinese character component reconstruction module pre-trained in step 2), the layout features of the handwritten Chinese characters are extracted; 4) Using the layout features of handwritten Chinese characters as input, the layout evaluation module is combined to evaluate the layout of handwritten Chinese characters, and the evaluation conclusion and modification suggestions are obtained according to the pre-set classification standards; 5) Use the layout evaluation suggestions to make adjustments to the handwritten Chinese characters.

2. A component-based handwritten Chinese character evaluation method as claimed in claim 1, characterized in that In step 1), the given handwritten Chinese character components refer to the first-level 3755 commonly used Chinese characters in GB2312_80, and each handwritten Chinese character stroke is saved independently.

3. A component-based handwritten Chinese character evaluation method as claimed in claim 1, characterized in that In step 1), the “random movement” refers to randomly moving one or more components in the handwritten Chinese character, thereby changing the layout of the handwritten Chinese character components.

4. A component-based handwritten Chinese character evaluation method as claimed in claim 1, characterized in that In step 1), the “size change” refers to adjusting the size of one or more components in the handwritten Chinese character.

5. A component-based handwritten Chinese character evaluation method as claimed in claim 1, characterized in that In step 2), the unchanged standard layout handwritten Chinese characters refer to the original handwritten Chinese characters that have not been adjusted by "random movement" and "size change".

6. A component-based handwritten Chinese character evaluation method as claimed in claim 1, characterized in that In step 2), the input of the Chinese character component reconstruction module is a multi-channel image, which contains handwritten Chinese characters with standard layout and handwritten Chinese characters with problem layout. The network output is the features of the handwritten Chinese characters with problem layout and the reconstructed handwritten Chinese characters with standard layout.

7. A component-based handwritten Chinese character evaluation method as claimed in claim 1, characterized in that In step 3), the extraction of layout features of handwritten Chinese characters refers to sending the input handwritten Chinese characters to the Chinese character component reconstruction module to obtain the layout features of the problematic handwritten Chinese characters, and then subtracting the layout features of the problematic handwritten Chinese characters from the layout features of the standard handwritten Chinese characters to obtain image features.

8. A component-based handwritten Chinese character evaluation method as claimed in claim 1, characterized in that In step 4), the input of the layout evaluation module is the layout features of the handwritten Chinese characters. In the layout evaluation module, the positional relationship between each component of the handwritten Chinese characters is classified one by one until the classification of the relationship between all components is completed, and corresponding evaluation conclusions and text suggestions are given.

9. A component-based handwritten Chinese character evaluation method as claimed in claim 1, characterized in that In step 4), the pre-set classification criteria include: (1) appropriate position compared with other parts; (2) higher position compared with other parts; (3) lower position compared with other parts; (4) left position compared with other parts; (5) right position compared with other parts; (6) smaller than other parts; (7) larger than other parts.

10. A component-based handwritten Chinese character evaluation method as claimed in claim 1, characterized in that In step 5), adjusting the handwritten Chinese characters using suggestions from the layout evaluation refers to making corresponding adjustments to the handwritten Chinese characters according to the seven classification standards preset in claim 9, including position adjustment and size adjustment of components.

Images