Chinese character stroke feature extraction method

Abstract

The invention belongs to the technical field of computer calligraphy, and particularly relates to a Chinese character stroke feature extraction method which comprises the following steps: constructing a feature extraction model based on a deep learning algorithm; inputting an original image and carrying out binarization processing to obtain a preprocessed image; converting the preprocessed image into a tensor, and performing normalization processing to obtain a process image; by constructing a feature extraction model, when a photographed Chinese character image is input, the Chinese character image can be automatically processed, a pre-processed image is obtained by adopting a binarization processing mode, then tensor conversion and normalization processing are performed on the pre-processed image to obtain a process image, each stroke in the process image is identified through an algorithm, and the accuracy of the process image is improved. A plurality of strokes of Chinese characters are formed through rectangular frame segmentation, and finally, the strokes surrounded by the rectangular frame are optimized and recombined, so that a prediction image # imgabs0 # can be obtained, and a handwritten Chinese character image is converted into a computer image with controllable Chinese character scoring and guidance.

Description

Technical Field

[0001] The invention belongs to the technical field of computer calligraphy, and specifically is a method for extracting features of Chinese character strokes. Background Art

[0002] Chinese character stroke feature extraction is the process of preprocessing the written Chinese character image and extracting each stroke of the Chinese character from the image. This technology provides reliable data support for subsequent research on handwritten Chinese character comparison, scoring and guidance; it can also provide data reference for research on Chinese character style conversion.

[0003] Handwritten Chinese character stroke extraction in the field of computer calligraphy is mainly divided into online stroke extraction and offline stroke extraction. Online stroke extraction mainly uses electronic screens and other devices to collect the writer's stroke path, and then extracts stroke information through the starting point and the end point. Offline stroke extraction, as one of the key contents of computer calligraphy research in recent years, can be divided into stroke extraction based on binary images and stroke extraction based on skeletons.

[0004] The method of stroke extraction based on binary images is as follows: first, the Chinese character image is binarized, and then the corner points of the Chinese character outline or the stroke pixels are analyzed to extract the intersection area of ​​the strokes, and the strokes are segmented using the intersection area. Finally, the relationship between the stroke segments is analyzed, and the segmented stroke segments are recombined to extract the strokes.

[0005] The method of skeleton-based stroke extraction is as follows: skeleton-based stroke extraction first extracts the skeleton of Chinese characters, and then preprocesses the obtained skeleton. A high-quality skeleton is obtained through preprocessing, and finally the stroke segments in the intersection area are analyzed, so as to combine the appropriate stroke segments to obtain independent strokes. The main methods are trend-based stroke extraction and template-based stroke extraction.

[0006] In summary, the online stroke extraction method is relatively simple. It uses specific equipment to write, and the extracted strokes are based on the writing process. Therefore, it is subject to specific equipment limitations and space limitations. Compared with offline stroke extraction, it is not universal. In the offline stroke extraction method, whether it is based on binary images or skeletons, the program usually repeatedly traverses the pixels in the image many times, which has the defects of large amount of calculation and long time consumption. Therefore, the larger the image pixels, the longer it takes.

[0007] To this end, the present invention provides a method for extracting Chinese character stroke features. Summary of the invention

[0008] In order to make up for the deficiencies of the prior art, at least one technical problem raised in the background technology is solved.

[0009] The technical solution adopted by the present invention to solve the technical problem is: a Chinese character stroke feature extraction method described in the present invention comprises the following steps: Build a feature extraction model based on deep learning algorithm; Input the original image and perform binarization processing to obtain the preprocessed image; The preprocessed image is converted into a tensor and normalized to obtain a process image; Each stroke instance in the process image is selected with a rectangular box based on the instance segmentation technology; Based on semantic segmentation technology, the pixels in the rectangular frame are identified to obtain the feature image ; For feature images Reorganize and get the predicted image .

[0010] Preferably, the method for constructing the feature extraction model is: Collect Chinese character image data; Preprocess the collected Chinese character image data, including binarization, conversion to tensor and normalization; Build a feature extraction model based on deep learning algorithm; The preprocessed Chinese character image data is input into the feature extraction model for training; Calculate the accuracy of the feature extraction model ; When the accuracy of the feature extraction model Higher than expected , then the feature extraction model is obtained; When the accuracy of the feature extraction model Not higher than expected , then adjust the parameters to optimize the model until the accuracy Higher than expected , and obtain the optimized feature extraction model.

[0011] Preferably, the method for calculating the accuracy of the feature extraction model is: Get the original image And the predicted image output by the feature extraction model , to ensure the original image With the predicted image The size of ; Calculate the pixel-by-pixel difference according to the formula:

[0012] Calculate the value of all pixels Loss, according to the formula:

[0013] Said The loss is the quantized predicted image With the original image If The smaller the loss, the better the predicted image With the original image The closer, the predicted image The accuracy The higher based on Loss calculation feature model accuracy , according to the formula:

[0014] in, To adjust the parameters to control the accuracy The steepness of the function.

[0015] Preferably, the method of selecting each stroke instance in the process image with a rectangular frame based on the instance segmentation technology is: Based on the edge detection algorithm, the contour edge of the Chinese character in the process image is identified to obtain the contour of the Chinese character; Use the corner points and intersection points in the Chinese character outline to segment the stroke instances and obtain the stroke outline; The segmented stroke contours are numbered and recorded as ; The identified stroke outline is selected by using a rectangular frame, and the rectangular frame satisfies that the area of ​​the circumscribed rectangle of the stroke outline is minimal.

[0016] Preferably, the method of selecting each stroke instance in the process image with a rectangular frame based on the instance segmentation technology further comprises: Based on the numbered stroke contours, position information of the circumscribed rectangular frame of each stroke contour in the recognition process image is identified; All rectangular boxes in the process image are extracted separately, and the stroke contours in the rectangular boxes are numbered Naming and sorting to obtain feature images .

[0017] Preferably, the position information of the circumscribed rectangular frame is the position of the corner points of the rectangular frame. The position coordinates of the process image of the size; the corner points of the rectangular box are the diagonal points of the rectangular box.

[0018] Preferably, the method for identifying pixels within the rectangular frame based on semantic segmentation technology is: Identify the pixel points in the rectangular frame and determine the type of the stroke outline based on a clustering algorithm, the type including but not limited to horizontal, vertical, left-falling, right-falling and dot; Determine the stroke outline based on its type, and optimize the stroke outline; The optimized stroke contours are represented by different colors to obtain the feature image .

[0019] Preferably, the feature image Recombination to get the predicted image The method is: Get all feature images The position information of the middle rectangular box; According to the rectangular frame position information, the feature image Fixed to size In the blank canvas of By feature image Process all feature images in the order , get the predicted image .

[0020] Preferably, the method for optimizing the stroke contour is: Use morphological techniques to remove isolated points and short connecting line segments in stroke outlines, and Smooths the stroke outline to remove jagged edges.

[0021] Preferably, the deep learning algorithm includes but is not limited to a convolutional neural network CNN, a recurrent neural network RNN, a long short-term memory network LSTM or a combination thereof; the edge detection algorithm includes but is not limited to a Canny edge detector, a Sobel operator or a Laplacian operator.

[0022] The beneficial effects of the present invention are as follows: The Chinese character stroke feature extraction method of the present invention constructs a feature extraction model. When a photographed Chinese character image is input, the Chinese character image can be automatically processed, a binarization processing method is used to obtain a pre-processed image, and then the pre-processed image is converted into a tensor and normalized to obtain a process image. Each stroke in the process image is identified by an algorithm, and a rectangular frame is used to segment multiple strokes constituting the Chinese character. Finally, the strokes surrounded by the rectangular frame are optimized and reorganized to obtain a predicted image. , which realizes the transformation of handwritten Chinese character images into computer images for controllable Chinese character scoring and guidance. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The present invention will be further described below in conjunction with the accompanying drawings.

[0024] Figure 1is a flow chart of the present invention; Figure 2 It is a logic diagram of the present invention. DETAILED DESCRIPTION

[0025] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0026] like Figure 1 to Figure 2 As shown, a Chinese character stroke feature extraction method according to an embodiment of the present invention comprises the following steps: Build a feature extraction model based on deep learning algorithm; Input original image And perform binarization processing to obtain a preprocessed image; The preprocessed image is converted into a tensor and normalized to obtain a process image; Each stroke instance in the process image is selected with a rectangular box based on the instance segmentation technology; Based on semantic segmentation technology, the pixels in the rectangular frame are identified to obtain the feature image ; For feature images Reorganize and get the predicted image .

[0027] In the prior art, the online stroke extraction method is relatively simple. It uses a specific device to write, and the extracted strokes are based on the writing process. Therefore, it is subject to specific device limitations and space limitations, and is not as universal as offline stroke extraction. In the offline stroke extraction method, whether it is based on a binary image or a skeleton, the program usually repeatedly traverses the pixels in the image many times, which has the defects of large amount of calculation and long time consumption. Therefore, the larger the image pixels, the longer it takes.

[0028] Based on the above, in one embodiment of the present invention, by constructing a feature extraction model, the stroke extraction task is transformed into a panoramic segmentation task, and an end-to-end stroke extraction method is implemented. It is only necessary to take a photo of the handwritten Chinese character and input it. Based on the feature extraction model, the extracted stroke image can be output, and the accuracy and robustness of the stroke extraction can be improved by only designing and optimizing the model, and the speed of stroke extraction is improved. Specifically, by constructing a feature extraction model, when the photographed Chinese character image is input, the Chinese character image can be automatically processed, and a binarization processing method is used to obtain a preprocessed image. The preprocessed image is then converted into a tensor and normalized to obtain a process image. Each stroke in the process image is identified by an algorithm, and a rectangular frame is used to segment the multiple strokes that make up the Chinese character. Finally, the strokes surrounded by the rectangular frame are optimized and reorganized to obtain a predicted image. , which realizes the transformation of handwritten Chinese character images into computer images for controllable Chinese character scoring and guidance; Among them, instance segmentation technology and semantic segmentation technology both belong to the feature extraction model, and the panoramic segmentation task is based on instance segmentation technology and semantic segmentation technology. The training process is to firstly The instance part of the rectangular box of each stroke is selected, and then the stroke part in each rectangular box instance is labeled; during the training process, instance segmentation is responsible for learning the size of the rectangular box and how the rectangular box selects the stroke instance; after the instance is extracted, the semantic segmentation framework only needs to learn which pixels are judged as the strokes to be recognized; The reasoning process is to input the original image The data is binarized, converted into a tensor and normalized. After normalization, the model is inferred. First, a rectangular box is inferred to identify each stroke. Then, each pixel in the rectangular box needs to be inferred to see whether it belongs to the pixel of the stroke.

[0029] In one embodiment, the method for constructing the feature extraction model is: Collect Chinese character image data; Preprocess the collected Chinese character image data, including binarization, conversion to tensor and normalization; Build a feature extraction model based on deep learning algorithm; The preprocessed Chinese character image data is input into the feature extraction model for training; Calculate the accuracy of the feature extraction model ; When the accuracy of the feature extraction model Higher than expected , then the feature extraction model is obtained; When the accuracy of the feature extraction model Not higher than expected , then adjust the parameters to optimize the model until the accuracy Higher than expected , and obtain the optimized feature extraction model.

[0030] A feature extraction model is built based on a deep learning algorithm to analyze and reason about the original image input to the computer, that is, the handwritten Chinese character image, in order to output a predicted image that conforms to the handwritten Chinese character image. , which is used for scoring and guiding handwritten Chinese characters. In one embodiment of the present invention, by collecting raw data, namely Chinese character image data, preprocessing the Chinese character image data, and then training the feature extraction model based on the preprocessed Chinese character image data, the predicted image output by the feature extraction model is The accuracy Higher than expected , where the initial expected value It is artificially set, and over time, the expected value Adaptive adjustment can be used to characterize the optimization results of the feature extraction model over time; when the accuracy of the feature extraction model Higher than expected When , the feature extraction model is considered to be trained, and the handwritten Chinese character image can be directly recognized and output. It can be used as an alternative to scoring and guiding handwritten Chinese characters; Based on the above, the expected value The expression is as follows:

[0031] in, is the initial expected value, set by humans. is the expected growth rate per unit time, also set manually, Apply time for feature extraction model; Based on the above, calculate the expected value corresponding to the application time of the feature extraction model , and then compared with the calculated accuracy Compare and determine whether the feature extraction model needs to be optimized.

[0032] In one embodiment, the method for calculating the accuracy of the feature extraction model is: Get the original image And the predicted image output by the feature extraction model , to ensure the original image With the predicted image The size of ; Calculate the pixel-by-pixel difference according to the formula:

[0033] Calculate the value of all pixels Loss, according to the formula:

[0034] Said The loss is the quantized predicted image With the original image If The smaller the loss, the better the predicted image With the original image The closer, the predicted image The accuracy The higher based on Loss calculation feature model accuracy , according to the formula:

[0035] in, To adjust the parameters to control the accuracy The steepness of the function.

[0036] As mentioned above, the constructed feature extraction model needs to pay attention to the output prediction image The accuracy of the output is It may not match the handwritten Chinese character image, that is, the original image, resulting in There is a large deviation between the obtained score and the actual score result. Based on the above, in one embodiment of the present invention, the constructed feature extraction model needs to be verified for accuracy. Loss calculation, get the original image With the predicted image The difference between If the loss is large, it means that the predicted image output based on the feature extraction model is With the original image There is a large deviation between them, so the feature extraction model needs to be further optimized. The loss is small, which means that the predicted image output by the feature extraction model is With the original image The deviation between them is small; based on the solution Value, substitute into accuracy In the function, we can find the accuracy The value of is used to characterize the accuracy of the feature extraction model ; in, Represents any pixel in the original image With the predicted image Any pixel in In addition, since the predicted image is binary, its pixels can only be expressed as 0 or 1, so we use Indicates the difference.

[0037] In one embodiment, the method of selecting each stroke instance in the process image with a rectangular box based on the instance segmentation technology is: Based on the edge detection algorithm, the contour edge of the Chinese character in the process image is identified to obtain the contour of the Chinese character; Use the corner points and intersection points in the Chinese character outline to segment the stroke instances and obtain the stroke outline; The segmented stroke contours are numbered and recorded as ; The identified stroke outline is selected by using a rectangular frame, and the rectangular frame satisfies that the area of ​​the circumscribed rectangle of the stroke outline is minimal.

[0038] Based on the feature extraction model, the predicted image is identified and output In the process of identifying each stroke in the process image, each stroke is marked to output a predicted image that matches the original image. In one embodiment of the present invention, the edge detection algorithm can be used to identify the outline of Chinese characters in the process image, so as to identify and screen the outline of Chinese characters from the original image, and then mark the intersections and corners between the strokes in the outline of Chinese characters, and segment the strokes based on the marks to obtain independent stroke outlines. In order to facilitate the subsequent reorganization of the strokes to obtain the predicted image , the segmented stroke contours are numbered here. After the strokes are segmented, they need to be surrounded by a rectangular frame to facilitate the extraction of independent stroke contours from the process image. It is worth noting that the size of the rectangular frame needs to be set adaptively. The size of the rectangular frame is set according to the size of the stroke contour so that the rectangular frame meets the requirement of minimizing the area of ​​the circumscribed rectangle surrounding the stroke contour. The purpose is to separate the rectangular frames between strokes as much as possible to avoid overlapping and causing difficulties in extracting the stroke contours.

[0039] In one embodiment, the method of selecting each stroke instance in the process image with a rectangular frame based on the instance segmentation technology further includes: Based on the numbered stroke contours, position information of the circumscribed rectangular frame of each stroke contour in the recognition process image is identified; All rectangular boxes in the process image are extracted separately, and the stroke contours in the rectangular boxes are numbered Naming and sorting to obtain feature images .

[0040] Based on the above, after enclosing the stroke outline in a rectangular frame, each stroke in the original image can be segmented, in order to better identify each stroke outline to generate a predicted image , it is also necessary to extract each stroke outline from the process image and then reconstruct it to obtain the predicted image In one embodiment of the present invention, based on the stroke outline surrounded by the rectangular frame, the corner points of the rectangular frame are located at The position coordinates of the process image of the size are extracted for each rectangular frame and the stroke outline in the rectangular frame, and each rectangular frame and the stroke outline in the rectangular frame are distinguished. Specifically, the extracted feature image is named according to the number of the stroke outline and recorded as the feature image Based on this, when the stroke contours are subsequently reorganized, the feature image can be generated according to the order of the rectangular frame position information and the stroke contours. Reconstruction and prediction of images Generation of.

[0041] In one embodiment, the position information of the circumscribed rectangular frame is the position of the corner points of the rectangular frame. The position coordinates of the process image of the size; the corner points of the rectangular box are the diagonal points of the rectangular box.

[0042] In one embodiment, the method for identifying pixels within a rectangular frame based on semantic segmentation technology is: Identify the pixel points in the rectangular frame and determine the type of the stroke outline based on a clustering algorithm, the type including but not limited to horizontal, vertical, left-falling, right-falling and dot; Determine the stroke outline based on its type, and optimize the stroke outline; The optimized stroke contours are represented by different colors to obtain the feature image .

[0043] In one embodiment of the present invention, after the stroke outline is segmented, it is necessary to optimize the stroke outline and identify the type. Specifically, by obtaining all the extracted feature images , and then identify the stroke contours in the adjusted image, and determine the type of stroke contours through a clustering algorithm, so as to ensure that the predicted image The accuracy of the generated image is also , independent stroke outlines can also be optimized and represented by different colors to facilitate the subsequent reorganization of multiple stroke outlines.

[0044] In one embodiment, the feature image Recombination to get the predicted image The method is: Get all feature images The position information of the middle rectangular box; According to the rectangular frame position information, the feature image Fixed to size In the blank canvas of By feature image Process all feature images in the order , get the predicted image .

[0045] As an embodiment of the present invention, in the case of multiple feature images When reorganizing, the coordinates of the corner points of the rectangular box surrounding the stroke outline can be used to fix it at a size of In the blank canvas, according to the feature image The order of processing each feature image in turn , and finally the predicted image can be obtained , so that it can be scored and given guidance suggestions.

[0046] In one embodiment, the method for optimizing the stroke contour is: Use morphological techniques to remove isolated points and short connecting line segments in stroke outlines, and Smooths the stroke outline to remove jagged edges.

[0047] In one embodiment, the deep learning algorithm includes but is not limited to a convolutional neural network CNN, a recurrent neural network RNN, a long short-term memory network LSTM or a combination thereof; the edge detection algorithm includes but is not limited to a Canny edge detector, a Sobel operator or a Laplacian operator.

[0048] Working principle: By constructing a feature extraction model, the stroke extraction task is transformed into a panoramic segmentation task, and an end-to-end stroke extraction method is implemented. It only needs to take a photo of the handwritten Chinese characters as input. Based on the feature extraction model, the extracted stroke image can be output. Moreover, it only needs to design and optimize the model to improve the accuracy and robustness of stroke extraction and the speed of stroke extraction. Specifically, by constructing a feature extraction model, when the photographed Chinese character image is input, the Chinese character image can be automatically processed, and a binarization method is used to obtain a preprocessed image. The preprocessed image is then converted into a tensor and normalized to obtain a process image. Each stroke in the process image is identified by an algorithm, and a rectangular frame is used to segment the multiple strokes that make up the Chinese characters. Finally, the strokes surrounded by the rectangular frame are optimized and reorganized to obtain a predicted image. , which realizes the transformation of handwritten Chinese character images into computer images for controllable Chinese character scoring and guidance.

[0049] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A Chinese character stroke feature extraction method, characterized in that: The following steps are involved: Build a feature extraction model based on deep learning algorithm; Input original image And perform binarization processing to obtain a preprocessed image; The preprocessed image is converted into a tensor and normalized to obtain a process image; Each stroke instance in the process image is selected with a rectangular box based on the instance segmentation technology; Based on semantic segmentation technology, the pixels in the rectangular frame are identified to obtain the feature image ; For feature images Reorganize and get the predicted image .

2. A Chinese character stroke feature extraction method according to claim 1, characterized in that: The method for constructing the feature extraction model is: Collect Chinese character image data; Preprocess the collected Chinese character image data, including binarization, conversion to tensor and normalization; Build a feature extraction model based on deep learning algorithm; The preprocessed Chinese character image data is input into the feature extraction model for training; Calculate the accuracy of the feature extraction model ; When the accuracy of the feature extraction model Higher than expected , then the feature extraction model is obtained; When the accuracy of the feature extraction model Not higher than expected , then adjust the parameters to optimize the model until the accuracy Higher than expected , and obtain the optimized feature extraction model.

3. A Chinese character stroke feature extraction method according to claim 2, characterized in that: The method for calculating the accuracy of the feature extraction model is: Get the original image And the predicted image output by the feature extraction model , ensuring the original image With the predicted image The size of ; Calculate the pixel-by-pixel difference according to the formula: Calculate the value of all pixels Loss, according to the formula: Said The loss is the quantized predicted image With the original image If The smaller the loss, the better the predicted image With the original image The closer, the predicted image The accuracy The higher based on Loss calculation feature model accuracy , according to the formula: in, To adjust the parameters to control the accuracy The steepness of the function.

4. A Chinese character stroke feature extraction method according to claim 3, characterized in that: The method of selecting each stroke instance in the process image with a rectangular frame based on the instance segmentation technology is: Based on the edge detection algorithm, the contour edge of the Chinese character in the process image is identified to obtain the contour of the Chinese character; Use the corner points and intersection points in the Chinese character outline to segment the stroke instances and obtain the stroke outline; The segmented stroke contours are numbered and recorded as ; The identified stroke outline is selected by using a rectangular frame, and the rectangular frame satisfies that the area of ​​the circumscribed rectangle of the stroke outline is minimal.

5. The method for extracting Chinese character stroke features according to claim 1, characterized in that: The method of selecting each stroke instance in the process image with a rectangular frame based on the instance segmentation technology also includes: Based on the numbered stroke contours, position information of the circumscribed rectangular frame of each stroke contour in the recognition process image is identified; All rectangular boxes in the process image are extracted separately, and the stroke contours in the rectangular boxes are numbered Naming and sorting to obtain feature images .

6. A Chinese character stroke feature extraction method according to claim 5, characterized in that: The position information of the circumscribed rectangular frame is the position of the corner points of the rectangular frame. The position coordinates of the process image of the size; the corner points of the rectangular box are the diagonal points of the rectangular box.

7. A Chinese character stroke feature extraction method according to claim 6, characterized in that: The method for identifying pixels within a rectangular frame based on semantic segmentation technology is: Identify the pixel points in the rectangular frame and determine the type of the stroke outline based on a clustering algorithm, the type including but not limited to horizontal, vertical, left-falling, right-falling and dot; Determine the stroke outline based on its type, and optimize the stroke outline; The optimized stroke contours are represented by different colors to obtain the feature image .

8. A Chinese character stroke feature extraction method according to claim 7, characterized in that: For the feature image Recombination to get the predicted image The method is: Get all feature images The position information of the middle rectangular box; According to the rectangular frame position information, the feature image Fixed to size In the blank canvas of By feature image Process all feature images in the order , get the predicted image .

9. A Chinese character stroke feature extraction method according to claim 8, characterized in that: The method for optimizing the stroke contour is: Use morphological techniques to remove isolated points and short connecting line segments in stroke outlines, and Smooths the stroke outline to remove jagged edges.

10. A Chinese character stroke feature extraction method according to claim 9, characterized in that: The deep learning algorithm includes but is not limited to a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory (LSTM) network or a combination thereof; the edge detection algorithm includes but is not limited to a Canny edge detector, a Sobel operator or a Laplacian operator.

Images