A method for splitting strokes of Chinese characters

Abstract

The application discloses a Chinese character stroke splitting method, comprising the following steps: acquiring an original Chinese character image and performing standard binary processing to serve as the input of a network; dividing Chinese character strokes into 10 categories according to shape structure features to construct stroke category labels; acquiring Chinese character information to obtain prior information of Chinese character types, stroke categories and quantities to construct a prior feature layer; fusing the prior feature layer and image features of the Chinese character image after extraction and processing, decoding the fused information, and finally outputting stroke results through a multi-label output layer. The application not only significantly improves the accuracy of Chinese character stroke segmentation, but also reduces the calculation amount, and can better serve downstream tasks.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and deep learning, and particularly relates to a Chinese character stroke splitting method based on multi-label segmentation. BACKGROUND

[0002] Stroke segmentation of Chinese characters can improve the computer's cognition of Chinese character images, and thus plays an important role in application scenarios such as calligraphy teaching evaluation, aesthetic evaluation of painting and calligraphy works, and artistic grading.

[0003] The main difficulty of Chinese character stroke segmentation technology is the intersection region between different strokes. The intersection region belongs to two or more strokes at the same time, but the conventional image segmentation method can only divide each pixel point into one category. Therefore, so far, the research on Chinese character stroke splitting mainly focuses on different processing methods for intersection points. For example, the invention patent CN103927537 A discloses an adaptive Chinese character stroke intersection region extraction algorithm, which dynamically obtains a segmentation point by using the distance statistical characteristics of the stroke intersection point to the stroke contour, obtains the smallest intersection region for separating the Chinese character strokes, and is suitable for strokes of different fonts, strokes with different stroke thicknesses, and strokes with different intersection structures, solving the problem of fixed size and fixed shape extraction of intersection regions.

[0004] However, in the existing methods, whether the strokes are extracted using the skeleton information of the refined Chinese characters or the intersection region is taken as a new category of independent strokes in the deep learning network, the original information of the strokes will be lost. SUMMARY

[0005] To solve the above technical problems, the present application provides a Chinese character stroke splitting method based on multi-label segmentation.

[0006] The purpose of the present application is achieved by the following technical solutions:

[0007] The present application provides a Chinese character stroke splitting method, which aims to solve the accuracy and efficiency problems of soft pen calligraphy Chinese character stroke segmentation in the prior art.

[0008] A Chinese character stroke splitting method, the method comprising the following steps:

[0009] S1: obtaining Chinese character information of Chinese characters, and constructing a prior feature information table; the Chinese character information includes Chinese character type information, stroke category information included in the Chinese character, and stroke number information;

[0010] S2: obtain an original Chinese character image, standard binaryzation is carried out to the original Chinese character image to obtain a processed Chinese character image, and meanwhile, the Chinese character type information of the original Chinese character image is used to search in a prior feature information table, so that the Chinese character type information, stroke category information and stroke number information corresponding to the original Chinese character image are output; the Chinese character type information, stroke category information and stroke number information corresponding to the original Chinese character image are subjected to information coding to obtain a prior feature layer, and then the prior feature layer and the processed Chinese character image are input into a neural network to perform training of the neural network, so that a trained neural network is obtained; the neural network comprises an input layer comprising an image and the prior feature layer, a coding module for extracting image features of the processed Chinese character image, a fusion module for fusing the image features and the prior feature layer, a decoding module for decoding the fused information and generating a final structure, and a multi-label output layer;

[0011] S3: standard binaryzation is performed on a to-be-split Chinese character image to obtain a processed to-be-split Chinese character image, and the Chinese character information of the Chinese character in the to-be-split Chinese character image is input into the prior feature layer to obtain corresponding Chinese character information of the to-be-split Chinese character; the Chinese character information of the to-be-split Chinese character and the processed to-be-split Chinese character image are input into the trained neural network, and the trained neural network outputs all strokes in the processed output to-be-split Chinese character image at one time.

[0012] Further improvement, the 10 categories of strokes are as follows:

[0013]

[0014] Further improvement, the 10 categories of strokes segmented by the character are represented as S=s1,s2…,s 10 ; then the mapping relationship between the to-be-split Chinese character image and the to-be-split Chinese character image is as follows:

[0015] S=N(P stroke )

[0016] Wherein, N represents a network function, P stroke is a to-be-split Chinese character image, S represents the output of the network, and the dimension is (10,w,h), which is 10 stroke results of each category with the same size as the original input image, s 10 represents the 10th category; w represents the width of the output image, and h represents the height of the output image.

[0017] Further improvement, the encoding module of the image feature of the extracted processed Chinese character image adopts the ASPP structure in the DeepLab V3+ segmentation network structure, the encoding module completes the feature extraction of the processed Chinese character image, then the fusion module splices the prior feature layer and the extracted image feature to obtain the fused feature layer, the decoding module decodes the fused feature layer, and the feature map with the same size as the input picture is obtained through dimension reduction and up-sampling. Finally, the feature map outputs all strokes in the to-be-split Chinese character image according to stroke categories through a multi-label output layer.

[0018] Further improvement, the specific steps of the data standard binary processing in step S2 include:

[0019] S21: the original Chinese character image is subjected to image binary processing, and then only the Chinese character part is retained to obtain a binary image;

[0020] S22: the binary image is subjected to size normalization.

[0021] Further improvement, in step S1, in the process of constructing the prior feature layer, the Chinese character information is standardized, and the specific calculation formula is as follows:

[0022]

[0023]

[0024] wherein, V 汉字类别信息 and V 笔画信息 respectively represent the value of the standardized Chinese category information and the value of the stroke information when the stroke information of the Chinese character type of the prior feature layer is input into the network; V info is the Chinese character related information recorded in the constructed prior feature information table; n character is the maximum value of the Chinese character number in the prior feature information table; n stroke is the maximum threshold value of the stroke, and max(V layer ) represents the upper limit of the median of the splicing layer, which is usually 1.

[0025] Further improvement, n stroke ≤15.

[0026] Further improvement, in step two, when training the neural network, the trained neural network is obtained by minimizing the loss function; wherein the construction method of the loss function is as follows:

[0027] The content output by the multi-label output layer in the neural network is a 10-dimensional stroke mask corresponding to 10 stroke categories, when determining the classification of each pixel point in the multi-label output layer, 10 dimensions will independently calculate and determine whether each pixel point belongs to the category stroke, and the calculation formula is as follows:

[0028] r p = (o p ≥ λ) * 1 + (o p < λ) * 0

[0029] Wherein o p is the prediction output of the pixel point in a certain dimension stroke type, λ is the set determination threshold, r p refers to the determination result of whether the pixel belongs to the category stroke;

[0030] The multi-classification problem is converted into a multi-dimensional binomial distribution problem, that is, the label of each category of the pixel point has only two possibilities: "the pixel point is this category stroke" or "the pixel point is not this category stroke", and a suitable total loss function is constructed according to this, and the total loss function L S is defined as follows:

[0031]

[0032]

[0033] Wherein, is the total loss function corresponding to the output i dimension corresponding to a certain category stroke, which is divided into two parts of the sub-loss function and σ is the weight corresponding to the two kinds of loss, and for the specific sub-loss function and The specific definitions are as follows:

[0034]

[0035]

[0036] Wherein represents the output corresponding to the pixel point j, is the probability value corresponding to the pixel point j in the label.

[0037] The beneficial effects of the present application are:

[0038] 1. The present application combines the type and stroke number of Chinese characters as prior knowledge, proposes stroke categories, constructs neural networks and corresponding loss functions, and realizes more efficient extraction of Chinese character strokes.

[0039] 2. The soft brush calligraphy Chinese character stroke segmentation is a direction that the prior art still involves shallowly, and the present research provides effective technical support for the downstream tasks of future intelligent calligraphy teaching, aesthetic evaluation and artistic examination of painting works, and has strong practical significance. BRIEF DESCRIPTION OF DRAWINGS

[0040] The application will be further described with reference to the drawings, but the contents of the drawings do not constitute any limitation to the application.

[0041] Figure 1 The system flow chart for calligraphy imitation evaluation of the application;

[0042] Figure 2 The network structure diagram for the method;

[0043] Figure 3 The output schematic diagram for stroke splitting of Chinese character image of the application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical scheme and advantages of the application more clear and understandable, the application will be further described in detail below with reference to the drawings and examples.

[0045] Example 1

[0046] As shown in a Chinese character stroke splitting method, the method specifically comprises the following steps: Figure 1

[0047] Step 1, obtain the original Chinese character image and perform standard binaryzation processing as the input of the network;

[0048] The specific steps of data standard binaryzation processing include:

[0049] Step 1.1: use a camera to shoot a written calligraphy Chinese character picture, and then perform image binaryzation processing to retain the Chinese character part;

[0050] Step 1.2: perform size normalization on the binaryzation image, scale the font image to the size of 400*400, convert it into a gray scale image, and perform binaryzation processing to ensure that it can be smoothly input into the stroke splitting network.

[0051] Further improvement, in step two, the specific steps for constructing the prior knowledge layer include:

[0052] Step 2.1: obtain the prior information of Chinese character type, stroke category and quantity by obtaining Chinese character information;

[0053] Step 2.2: in the process of constructing the prior feature layer, information needs to be normalized to prevent the true value number of information from being too large to affect the calculation of the network. The specific calculation formula is as follows:

[0054]

[0055]

[0056] Among them, V facture is the input value of the feature layer, V info ​is the true value of information, n character is the maximum value of Chinese character category number, n stroke is the maximum threshold value of strokes, considering that in Chinese characters, except for a few rare characters, the number of strokes in a category is basically not too large, this patent is set to 15.V layer is the upper limit of the threshold value of the combined layer;

[0057] Step 2, obtain Chinese character information to obtain prior information of Chinese character type, stroke category and number to obtain prior feature layer;

[0058] Step 3, divide the network into two categories according to the structure, one is the network that adds shallow features when upsampling, and the other is the network containing an encoder and a decoder module, then input the prior feature layer to the corresponding network layer according to the network classification;

[0059] The network structure division method is as follows:

[0060] The network is divided into two categories according to the structure: the first category is the network represented by U-Net that adds shallow features when upsampling, and the prior feature layer will be input into the network together with the picture. The second category is the network represented by Deeplab v3 containing an encoder and a decoder module, and the prior feature layer of this category is combined with the features extracted by the encoder and input into the corresponding network layer of the decoder.

[0061] Step 4, divide strokes into 10 categories according to shape structure features, change the multi-classification problem into a segmentation problem under multi-label, adjust the output of the network, construct a 10-dimensional image matrix, and let Stroke-Seg output all strokes at one time according to stroke categories;

[0062] The network output modification method includes the following steps:

[0063] Step 4.1: divide strokes into 10 categories according to shape structure features, and the 10 categories are shown in the following table:

[0064]

[0065] Step 4.2: the invention changes the multi-classification problem into a multi-dimensional binomial distribution problem, that is, the label of each category has only two possibilities: “this pixel point is this type of stroke” or “this pixel point is not this type of stroke”. Then adjust the output of the network, construct a 10-dimensional image matrix, and let Stroke-Seg output all strokes at one time according to stroke categories. For a given Chinese character image, we represent the strokes segmented by the character as S=s1,s2…,s 10 . Then the image X and its mapping relationship are as follows:

[0066] S=N(P stroke )

[0067] where N represents the network function, P stroke is the output of the network, whose dimension is (10, w, h), which is 10 stroke results of each class with the same size as the original input image;

[0068] Step 4.3: Independent judgment is made for each dimension corresponding to the stroke, and the calculation formula is as follows:

[0069] r p = (o p ≥ λ) × 1 + (o p < λ) × 0

[0070] where o p is the predicted output of a pixel point in a certain class, λ is the set judgment threshold, and r p refers to the pix-level result.

[0071] where the network needs to be trained before use, and a suitable loss function is constructed according to the network output format to train the network.

[0072] The specific method of constructing a suitable loss function is as follows:

[0073] According to the network output format, the loss function L S is constructed:

[0074]

[0075]

[0076] where, is the loss function corresponding to the i-th dimension of the output corresponding to a certain class of stroke, which is divided into and two parts. σ is the weight corresponding to the two kinds of losses.

[0077] For the specific loss functions and , their specific definitions are as follows:

[0078]

[0079]

[0080] where represents the output corresponding to the pixel point j, is the value of the corresponding position in the j target image. Thus, the loss function is constructed, and the network can be trained and perform calligraphy stroke separation.

[0081] Figure 3Six segmentation examples of the present application are shown, where the first column is the input image. The second to eleventh columns are the ten output images of the network, corresponding to the ten stroke types listed in the table. The pure black image indicates that the Chinese character does not have this stroke, so the output is empty. Among them, "wait" and "once" are not included in the training set. The network can successfully segment all strokes at the first time, which to some extent proves the generality of the method.

[0082] Currently, there are few studies on stroke extraction, and there are two limitations. First, most studies focus on hard pen calligraphy [1-4]. Hard pen calligraphy is easier to handle than brush calligraphy, and its form is more regular, with consistent stroke thickness. In contrast, the strokes of soft pen calligraphy are rich in variation, which to some extent increases the difficulty of stroke segmentation. Second, these methods have to some extent destroyed the original information of Chinese characters. For example, using the skeleton information of Chinese characters to segment the refined strokes [3-4] will lose the rich stroke features of soft pen calligraphy, making the segmented strokes method unable to be used for downstream tasks such as calligraphy aesthetics evaluation. If the intersection area between strokes is segmented as a new stroke class in the deep network [5], the network can only learn to the truncated stroke cluster, which increases the difficulty of the network to understand the stroke segmentation, and this method needs to do a stroke combination operation again after completing the segmentation, making the operation process more complex.

[0083] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the present application and are not intended to limit the scope of protection of the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the essence and scope of the technical solutions of the present application.

Claims

1. A method for splitting Chinese character strokes, characterized in that, The method comprises the following steps: S1: strokes are divided into 10 categories according to shape structure characteristics; obtain the Chinese character information of the Chinese character, and construct a prior feature information table based on the divided stroke classification method; the Chinese character information comprises Chinese character type information, stroke category information and stroke quantity information included in the Chinese character; S2: obtain an original Chinese character image, perform standard binaryzation on the original Chinese character image to obtain a processed Chinese character image, and search in the prior feature information table using the Chinese character type information of the original Chinese character image, thereby outputting the Chinese character type information, stroke category information and stroke quantity information corresponding to the original Chinese character image; S3: the Chinese character type information, stroke category information and stroke quantity information corresponding to the original Chinese character image are subjected to information coding to obtain a prior feature layer, and then the prior feature layer and the processed Chinese character image are input into a neural network for training of the neural network, thereby obtaining a trained neural network; the neural network comprises an input layer comprising an image and the prior feature layer, an encoding module for extracting image features of the processed Chinese character image, a fusion module for fusing the image features and the prior feature layer, a decoding module for decoding the fused information and generating a final structure, and a multi-label output layer; S4: perform standard binaryzation on a to-be-split Chinese character image to obtain a processed to-be-split Chinese character image, and input Chinese character information of the Chinese character in the to-be-split Chinese character image into the prior feature layer to obtain corresponding Chinese character information of the to-be-split Chinese character; input the Chinese character information of the to-be-split Chinese character and the processed to-be-split Chinese character image into the trained neural network, and the trained neural network outputs all strokes in the processed output to-be-split Chinese character image at one time; The encoding module for extracting image features of the processed Chinese character image adopts an ASPP structure in a DeepLab V3+ segmentation network structure, the encoding module completes feature extraction of the processed Chinese character image, then the fusion module splices the prior feature layer and the extracted image features to obtain a fused feature layer, the decoding module decodes the fused feature layer to obtain a feature map with the same size as the input image through dimension reduction and up-sampling; finally, the feature map outputs all strokes in the to-be-split Chinese character image according to stroke categories through the multi-label output layer; In step two, the trained neural network is obtained by minimizing a loss function; the loss function is constructed as follows: The content output by the multi-label output layer in the neural network is a 10-dimensional stroke mask corresponding to 10 stroke categories, when determining the classification of each pixel point in the multi-label output layer, 10 dimensions will independently calculate and determine whether each pixel point belongs to the category stroke, and the calculation formula is as follows: r p = (o p ≥ λ) x 1 + (o p < λ) x 0 wherein o p is the prediction output of the pixel in one dimension stroke type, λ is the set decision threshold, r p refers to the decision result of whether the pixel belongs to the category stroke. The multi-classification problem is converted into a multi-dimensional binomial distribution problem, i.e. setting the label of each class of the pixel point to have only two possibilities: "the pixel point is the stroke of this class" or "the pixel point is not the stroke of this class", and constructing a total loss function suitable therefor, the total loss function L S is defined as follows: wherein, is the total loss function corresponding to the i-th dimension of the output for a certain type of stroke, which is divided into two sub-loss functions and ; σ is the weight corresponding to the two losses, and the specific sub-loss function and is defined as follows: wherein represents the output corresponding to pixel point j, is the probability value corresponding to pixel point j in the label.

2. The Chinese character stroke splitting method of claim 1, wherein, The stroke categories are divided into 10 categories, and the specific categories are as follows:

3. The method for splitting Chinese character strokes according to claim 2, wherein, The 10 major categories of strokes segmented by the character are represented as S = s1, s2, …, s 10 ; then the image to be split Chinese character and the mapping relationship of the image to be split Chinese character are as follows: S = N(P stroke ) where N represents a network function, P stroke is the input image, S represents the output of the network, and its dimension is (10, w, h), which is 10 stroke results of each class with the same size as the original input image, s 10 represents the 10th class; w represents the width of the output image, and h represents the height of the output image.

4. The method of claim 1, wherein the strokes are separated by a stroke separator. The specific steps of the data standard binaryzation processing in step S2 comprise: S21: perform image binaryzation processing on the original Chinese character image, and then only keep the Chinese character part to obtain a binaryzation image; S22: perform size normalization on the binaryzation image.

5. The method of claim 1, wherein the strokes are separated by a stroke separator. In step S1, during the construction of the prior feature layer, the Chinese character information is standardized, and the specific calculation formula is as follows: wherein, V 汉字类别信息 and V 笔画信息 respectively represent the value of the standardized Chinese category information and the value of the stroke information of the Chinese character type of the prior feature layer; V info is the Chinese character related information recorded in the constructed prior feature information table; n character is the maximum value of the Chinese character number in the prior feature information table; n stroke is the maximum threshold value of the stroke, and max(V layer ) represents the upper limit of the median value of the combination layer, which is 1.

6. The Chinese character stroke disassembling method of claim 5, wherein, n stroke ≤15。

Images