Font stroke extraction method and device, terminal and storage medium

Abstract

The invention relates to a font stroke extraction method and device, a terminal and a storage medium. The font stroke extraction method comprises the steps of obtaining a font stroke image needing to be extracted; the obtained font stroke image is optimized, so that the image contour is more obvious; calling a neural network model to perform stroke segmentation on the target font image; matching the image segmented by the neural network with an original image; and outputting the matched stroke image. After the font stroke image is marked, training is conducted through a neural network, multiple times of model training are conducted, and during training, a small number of negative samples are added, and convolution kernel parameters are adjusted to enable model training to achieve the optimal effect. Semantic segmentation and matching are carried out on the target strokes, and finally matched stroke images are output. By adopting the font stroke extraction method, automatic stroke extraction can be performed on the handwritten calligraphy font, and the stroke extraction efficiency and accuracy are improved; the intelligent and efficient calligraphy and painting work font stroke extraction function is achieved, and the requirements of calligraphy works for font style analysis and detection are met.

Description

Technical Field

[0001] The present invention relates to the field of font stroke extraction, and in particular to a font stroke extraction method, device, terminal and storage medium. Background Art

[0002] Strokes are the basic components of Chinese characters. Stroke extraction plays an important role in Chinese character classification, font recognition, text retrieval, and automatic generation of Chinese characters. Chinese character strokes can be divided into several categories: horizontal, vertical, left-falling, right-falling, and right-falling. There are more than 30 specific subdivisions, among which some strokes can be further subdivided into more categories. Calligraphy is a handwritten form of Chinese characters. Handwritten writing is divided into two methods: hard pen writing and brush writing. The deformation of brush calligraphy strokes, adhesion between strokes, and changes in stroke thickness all bring difficulties to the extraction of calligraphy strokes. However, this information is unique to calligraphy works and plays an important role in calligraphy style recognition, copying, and generation of calligraphy characters of different styles.

[0003] At present, the main method for extracting Chinese character strokes adopts the traditional method of morphology or statistical model. The traditional method is not effective in extracting strokes of handwritten fonts with irregular forms, especially for fonts with irregular writing, it is difficult to accurately extract the target strokes; at the same time, due to the complex structure of Chinese characters and the huge character set, the strokes of different fonts vary in connectivity, length, thickness, the position between strokes and components will change, and the inclination angle of the strokes will change, which brings a lot of difficulties to the stroke extraction work. Summary of the invention

[0004] The technical problem to be solved by the present invention is that the traditional method is not effective in extracting the strokes of handwritten fonts with irregular forms, especially for fonts with irregular writing, it is difficult to accurately extract the target strokes. In view of the above-mentioned defects of the prior art, a font stroke extraction method, device, terminal and storage medium are provided.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A font stroke extraction method is constructed, comprising: obtaining a font stroke image to be extracted, wherein the font stroke image can be used for subsequent neural network model training and font extraction;

[0007] The obtained font stroke image is optimized to make the image outline more obvious;

[0008] Call the neural network model to perform stroke segmentation on the target font image;

[0009] Use neural network to segment the image and match it with the original image;

[0010] Output the matched stroke image.

[0011] Preferably, before calling the neural network model to perform stroke segmentation on the target font image, the method further includes: using the neural network to train the data set composed of the target stroke image to output an optimized model.

[0012] Preferably, a neural network is used to train a data set consisting of target stroke images to output an optimized model, including:

[0013] First, mark the target strokes in the font to form a training data set;

[0014] Using a neural network to train the data training set;

[0015] Adjust the convolution kernel parameters according to the loss function results of the training validation set to ensure that the model training achieves the optimal effect;

[0016] The optimal model is output based on the iteration value comparison during model training.

[0017] Preferably, the process of training the data set using a neural network includes performing multiple model training on the same data set;

[0018] During the model training process, negative samples are added to the training data set to prevent overfitting of the model training process.

[0019] Preferably, the negative sample is a sample in which no font strokes exist in the image.

[0020] Preferably, calling a neural network model to perform stroke segmentation on a target font image includes: using a trained optimization model to perform semantic segmentation on the target strokes to obtain target stroke contours;

[0021] The image segmented by the neural network is matched with the original image, including:

[0022] The target stroke contour obtained after semantic segmentation is compared with the original font to obtain the contour on the original image.

[0023] Preferably, the obtained font stroke image is optimized to make the image outline more obvious, including:

[0024] The font stroke image is gray-scaled, and then the gray-scaled image is filtered, and finally the filtered image is binarized to obtain single-channel font stroke image data.

[0025] A font stroke extraction device, the device comprising:

[0026] A graphics processing module, used to pre-process the font image of the target file for model training and / or font extraction;

[0027] A model training module is used to train the font stroke extraction network model to obtain an optimized model;

[0028] The stroke extraction module is used to extract the strokes of the target font by applying the optimization model.

[0029] A smart terminal comprises a memory, a processor and a font stroke extraction program stored in the memory and executable on the processor. When the font stroke extraction program is executed by the processor, the steps of the font stroke extraction method described above are implemented.

[0030] A computer-readable storage medium stores a font stroke extraction program, which implements the steps of the above-mentioned font stroke extraction method when executed by a processor.

[0031] The beneficial effects of the present invention are as follows: by gray-scale processing the acquired font image, filtering processing, and finally binarization processing, the noise background interference of the font image is reduced, and the outline of the font stroke image is made more obvious. Then, after the font stroke image is annotated, a neural network is used for training, through semantic segmentation of different strokes, and multiple model trainings are performed. During training, a small amount of negative samples are added and the convolution kernel parameters are adjusted to achieve the optimal effect of model training. The trained model is then used to semantically segment and match the target strokes, and finally the matched stroke image is output. The above-mentioned font stroke extraction method can be used to automatically extract the strokes of handwritten calligraphy fonts, and the efficiency and accuracy of stroke extraction can be improved; thereby realizing an intelligent and efficient font stroke extraction function for calligraphy and painting works, and meeting the needs of calligraphy works for font style analysis and detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the present invention will be further described below in conjunction with the accompanying drawings and embodiments. The drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work:

[0033] Figure 1 A flowchart of a method for extracting font strokes according to a preferred embodiment of the present invention;

[0034] Figure 2 A schematic diagram of a process for establishing a neural network training model according to a preferred embodiment of the present invention;

[0035] Figure 3 A schematic diagram of the structure of a font stroke extraction device according to a preferred embodiment of the present invention;

[0036] Figure 4A schematic diagram of stroke annotation of a font stroke extraction method according to a preferred embodiment of the present invention;

[0037] Figure 5 A schematic diagram of a stroke segmentation image of a font stroke extraction method according to a preferred embodiment of the present invention;

[0038] Figure 6 A schematic diagram of matching a segmented image with an original image in a method for extracting font strokes in a preferred embodiment of the present invention;

[0039] Figure 7 A schematic diagram of strokes outputted and matched by a font stroke extraction method according to a preferred embodiment of the present invention;

[0040] Figure 8 The figure is a block diagram of the internal structure principle of the intelligent terminal according to the preferred embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the following will be described clearly and completely in combination with the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are partial embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.

[0042] The font stroke extraction method of the preferred embodiment of the present invention; Figure 1 As shown, it is a flow chart of a font stroke extraction method provided by an embodiment of the present invention. The method can be executed by a device, and the device can be implemented by software and / or hardware.

[0043] Specifically, in this embodiment, the font stroke extraction method includes:

[0044] S21: Get font image from file;

[0045] The preferred embodiment of the present invention can obtain font images in various ways, such as importing font images from a local computer, obtaining font images from a network, obtaining font images from an image device, etc. The image device includes a camera, a scanner, etc. There are various formats of font images obtained. The font image data obtained in the present invention is in the commonly used JPG image format, and image formats such as PNG, GIF, and BMP may also be used.

[0046] After the font image file is obtained, it is loaded into the software system for processing to modify the image data format.

[0047] In the embodiment of the present invention, the opencv imdecode function is used to load the font image file into the software system so as to process the image in the next step. The image data is loaded into the software system through opencv to obtain the font image file of the ndarray format image data, so that the font image format is converted.

[0048] S22: Optimizing the acquired font image;

[0049] In a preferred embodiment of the present invention, since the acquired font image may be written in various colors and the background paper also has different colors, directly using it will cause large noise, so the acquired font image needs to be noise processed first. The acquired font image is first grayscale processed to filter its color, and then filtered to reduce the noise of the font image. Finally, binarization is performed to further reduce noise background interference, so that the image is changed to an image with only black and white, and image data with only a single channel is obtained.

[0050] In the present invention, the specific implementation method is to use the opencv cv2.cvtColor(img,cv.COLOR_BGR2GRAY) function to convert the acquired color font image into a grayscale image, then use cv2.medianBlur(img,3) to perform median filtering, and finally use the cv2.threshold(img,60,255,cv2.THRESH_BINARY) function to binarize the grayscale image to generate a black and white font image with only 0 and 255 pixel values. The advantage of the binary image is to reduce the noise background interference, make the font outline of the image more obvious, and facilitate the subsequent extraction of the mark of the font stroke.

[0051] S23: calling a neural network to perform stroke segmentation on the target font image;

[0052] Use the neural network model to semantically segment the strokes of the target font image to obtain the target stroke extraction contour. Specifically, the neural network model is called to perform segmentation prediction on the input target font image, and the output result is a contour image with the target stroke segmented, such as Figure 5 Shown is the segmented image of the stroke "horizontal".

[0053] S24: Match the image segmented by the neural network with the original image;

[0054] The stroke contour obtained after semantic segmentation by the neural network model is matched with the font of the original image to obtain the contour on the original image, thereby reflecting the specific position of the target stroke on the original image, such as Figure 6 shown.

[0055] S25: Output the matched stroke image;

[0056] When the stroke contour obtained after segmentation is matched with the target stroke on the original image, the specific position of the target stroke is obtained, and then the target stroke is extracted to obtain the image of the target stroke, such as Figure 7 shown.

[0057] Specifically, in the above embodiment, the above step S23: calling the neural network to perform stroke segmentation on the target font image includes:

[0058] S230: Obtain font images of a neural network training data set;

[0059] The preferred embodiment of the present invention can obtain the font image of the neural network training data set in various ways, such as importing the font image from the local, obtaining the font image from the network, obtaining the font image from an image device, etc. The image device includes a camera, a scanner, etc. There are various formats of the obtained font image. In the present invention, the font image data of the neural network training data set is obtained in the commonly used JPG image format, and image formats such as PNG, GIF, and BMP can also be used.

[0060] After obtaining the font image file of the neural network training data set, it is loaded into the software system for processing to modify the image data format.

[0061] In the embodiment of the present invention, the font image file of the neural network training data set is loaded into the software system by using the opencv imdecode function so as to process the image in the next step. The image data is loaded into the software system by opencv to obtain the font image file of the ndarray format image data, so that the font image format of the neural network training data set is converted.

[0062] S231: Optimizing the font image of the training data set;

[0063] In a preferred embodiment of the present invention, since the font images of the training data set are usually written in various colors and the background paper also has different colors, the noise is large if it is used directly, so the font images of the training data set need to be noise processed first. The font images of the training data set are first gray-scaled to filter the colors, and then filtered to reduce the noise of the font images. Finally, binarization is performed to further reduce the noise background interference, so that the image is changed to a black and white image, and the image data of the training data set with only a single channel is obtained.

[0064] In the present invention, the specific implementation method is to use the opencv cv2.cvtColor(img,cv.COLOR_BGR2GRAY) function to convert the acquired color font image into a grayscale image, then use cv2.medianBlur(img,3) to perform median filtering, and finally use the cv2.threshold(img,60,255,cv2.THRESH_BINARY) function to binarize the grayscale image to generate a black and white font image with only 0 and 255 pixel values. The advantage of the binary image is to reduce the noise background interference, make the font outline of the training data set image more obvious, and facilitate the subsequent extraction of the mark of the font stroke.

[0065] S232: marking strokes of the font image of the optimized training data set;

[0066] Since the basic strokes are limited, different basic strokes can be arranged and combined to obtain different fonts. Therefore, it is necessary to annotate the target strokes of the font in the font image of the optimized training dataset, and form the annotated strokes into a training dataset as the training dataset for subsequent model training.

[0067] Specifically, the labelme tool is used to mark the strokes of the target font. For crossed strokes, complete and continuous strokes need to be marked, such as Figure 4 As shown in FIG. 1 , different closed-loop annotation paths represent different stroke categories, thereby obtaining annotated training data sets for training the neural network.

[0068] S233: training the training data set using a neural network;

[0069] In the present invention, the Unet semantic segmentation network is used for training. The UNet model is an excellent model for semantic segmentation. It is both lightweight and high-performance, so it is usually used as a baseline test model for semantic segmentation tasks.

[0070] For fonts with few strokes, the input data during training can easily annotate the strokes, and their strokes can be easily semantically segmented through network training; for fonts with many strokes, the annotations in the input data set during training contain strokes of multiple categories, so that semantic segmentation of multiple different categories of strokes can be performed simultaneously through network training. Multi-stroke fonts can better and efficiently improve the speed of neural network training models.

[0071] S234: Tune the network parameters according to the results of the validation set;

[0072] In order to improve the optimality of the neural network model training data, a large number of single font or calligrapher training data sets are required for multiple model training. The loss function results of the validation set are obtained through multiple model training. According to the loss function results of the validation set, the convolution kernel parameters are adjusted to determine whether the model training achieves the optimal effect.

[0073] At the same time, in order to further improve the effect of the neural network model, a small amount of negative samples are added to the training data set to prevent overfitting. Negative samples can be samples of font strokes that do not exist in some images, such as landscape paintings and drawings, to improve the training effect of the neural network model.

[0074] S235: outputting the optimized neural network model;

[0075] In the process of model training on the training data set, the loss value in the later stage of model training will basically tend to be stable, but it will still fluctuate within a range. In order to obtain the optimal model, it is necessary to record the loss value output by each iteration process in each training process, and compare the loss value of the current iteration value with the saved loss value. If the current loss value is less than the saved loss value, the current iteration is saved in the optimal model.

[0076] Since it takes a certain amount of time to save the neural network training model, you can start saving the optimal model in the later stage of model training to speed up model training and save model training time.

[0077] In the present invention, a training data set is trained by using a neural network, and multiple model training is performed for each font image data. During the training process, parameters are adjusted to determine the effect of the training model, and finally an optimized model is output. The font handwriting is segmented and output through the optimized model.

[0078] Using the above method, the data is first used to establish a neural network data model, and then the model is used to automatically extract the strokes of handwritten calligraphy fonts to improve the efficiency and accuracy of stroke extraction; thereby realizing intelligent and efficient calligraphy and painting font stroke extraction function, meeting the needs of calligraphy works for font style analysis and detection.

[0079] Corresponding to the font stroke extraction method, the embodiment of the present invention further provides a font stroke extraction device, specifically, as follows Figure 3 As shown, the font stroke extraction device includes: an image processing module 100, a model training module 200 and a stroke extraction module 300.

[0080] The image processing module 100 is used to pre-process the font image input by the user so as to perform subsequent model training and / or font extraction.

[0081] Specifically, the font image is obtained in a variety of ways, and after the font image file is obtained, it is loaded into the software system for processing to modify the image data format.

[0082] Since the font images of the training data set are usually written in various colors, and the background paper also has different colors, the noise is large if it is used directly, so the font images of the training data set need to be noise processed first. First, the font images of the training data set are grayscale processed to filter the colors, and then filtered to reduce the noise of the font images. Finally, the noise background interference is further reduced through binarization, so that the image is changed to a black and white image, and only a single-channel training data set image data is obtained. The advantage of the binarized image is to reduce the noise background interference, making the font outline of the training data set image more obvious, which is convenient for the subsequent extraction of font stroke marks.

[0083] Model training module 200, used for extracting font strokes and obtaining an optimal model through neural network training;

[0084] Since the basic strokes are limited, different basic strokes can be arranged and combined to obtain different fonts. Therefore, it is necessary to annotate the target strokes of the font in the font image of the optimized training dataset, and form the annotated strokes into a training dataset as the training dataset for subsequent model training.

[0085] First, the strokes of the target font are annotated. For crossed strokes, complete and continuous strokes need to be annotated. As shown in the figure, different closed-loop annotation paths represent different stroke categories. Thus, annotated training data sets are obtained for neural network training.

[0086] For fonts with few strokes, the input data during training can easily annotate the strokes, and their strokes can be easily semantically segmented through network training; for fonts with many strokes, the annotations in the input data set during training contain strokes of multiple categories, so that semantic segmentation of multiple different categories of strokes can be performed simultaneously through network training. Multi-stroke fonts can better and efficiently improve the speed of neural network training models.

[0087] In order to improve the optimality of the neural network model training data, a large number of single font or calligrapher training data sets are required for multiple model training. The loss function results of the validation set are obtained through multiple model training. According to the loss function results of the validation set, the convolution kernel parameters are adjusted to determine whether the model training achieves the optimal effect.

[0088] At the same time, in order to further improve the effect of the neural network model, a small amount of negative samples are added to the training data set to prevent overfitting. Negative samples can be samples of font strokes that do not exist in some images, such as landscape paintings and drawings, to improve the training effect of the neural network model.

[0089] In the process of model training on the training data set, the loss value in the later stage of model training will basically tend to be stable, but it will still fluctuate within a range. In order to obtain the optimal model, it is necessary to record the loss value output by each iteration process in each training process, and compare the loss value of the current iteration value with the saved loss value. If the current loss value is less than the saved loss value, the current iteration is saved in the optimal model.

[0090] Since it takes a certain amount of time to save the neural network training model, you can start saving the optimal model in the later stage of model training to speed up model training and save model training time.

[0091] And a stroke extraction module 300 is used to extract strokes of the target font using the optimization model.

[0092] The trained neural network model is used to semantically segment the strokes of the target font image to obtain the target stroke extraction contour. Specifically, the neural network model is called to perform segmentation prediction on the input target font image, and the output result is a contour image with the target stroke segmented.

[0093] The stroke contour obtained after semantic segmentation by the neural network model is matched with the font of the original image to obtain the contour on the original image, thereby reflecting the specific position of the target stroke on the original image.

[0094] When the stroke contour obtained after segmentation is matched with the target stroke on the original image, the specific position of the target stroke is obtained, and then the target stroke is extracted to obtain the image of the target stroke.

[0095] Based on the above embodiments, the present invention further provides an intelligent terminal, whose principle block diagram is as follows: Figure 8 As shown, the above-mentioned intelligent terminal includes a processor, a memory, a network interface and a display screen connected through a system bus. The processor of the intelligent terminal is used to provide computing and control capabilities. The memory of the intelligent terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a font stroke extraction program. The memory provides an environment for the operation of the operating system and the font stroke extraction program in the non-volatile storage medium. The network interface of the intelligent terminal is used to communicate with an external terminal through a network connection. When the font stroke extraction program is executed by the processor, the steps of any one of the above-mentioned font stroke extraction methods are implemented. The display screen of the intelligent terminal can be a liquid crystal display screen or other display screens.

[0096] Those skilled in the art will understand that Figure 8 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the smart terminal to which the scheme of the present invention is applied. The specific smart terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0097] In an embodiment of the present invention, a smart terminal is provided. The smart terminal includes a memory, a processor, and a font stroke extraction program stored in the memory and executable on the processor. When the font stroke extraction program is executed by the processor, the following operation instructions are performed:

[0098] Get the font image from the target file;

[0099] Optimizing the acquired font image;

[0100] Call the neural network to perform stroke segmentation on the target font image;

[0101] Use neural network to segment the image and match it with the original image;

[0102] Output the matched stroke image.

[0103] An embodiment of the present invention further provides a computer-readable storage medium, on which a font stroke extraction program is stored. When the font stroke extraction program is executed by a processor, the steps of any one of the font stroke extraction methods provided in the embodiments of the present invention are implemented.

[0104] It should be understood that the present invention is described by some embodiments, and those skilled in the art will appreciate that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A font stroke extraction method, characterized in that: include: Acquire a font stroke image to be extracted, wherein the font stroke image can be used for subsequent neural network model training and font extraction; The obtained font stroke image is optimized to make the image outline more obvious; Call the neural network model to perform stroke segmentation on the target font image; Use neural network to segment the image and match it with the original image; Output the matched stroke image.

2. The method for extracting font strokes according to claim 1, characterized in that: Before calling the neural network model to perform stroke segmentation on the target font image, the method also includes: using the neural network to train the data set composed of the target stroke image to output an optimized model.

3. The method for extracting font strokes according to claim 2, characterized in that: The method of using a neural network to train a data set composed of target stroke images to output an optimized model includes: First, mark the target strokes in the font to form a training data set; Using a neural network to train the data training set; Adjust the convolution kernel parameters according to the loss function results of the training validation set to ensure that the model training achieves the optimal effect; The optimal model is output based on the iteration value comparison during model training.

4. The method for extracting font strokes according to claim 3, characterized in that: The process of training the data set using a neural network includes performing multiple model training on the same data set; During the model training process, negative samples are added to the training data set to prevent overfitting of the model training process.

5. The method for extracting font strokes according to claim 4, characterized in that: The negative samples are samples in which the font strokes do not exist in the image.

6. The method for extracting font strokes according to claim 3, characterized in that: The calling of the neural network model to perform stroke segmentation on the target font image includes: using a trained optimization model to perform semantic segmentation on the target strokes to obtain the target stroke contours; The image segmented by the neural network is matched with the original image, including: The target stroke contour obtained after the semantic segmentation is compared with the original image font to obtain the contour on the original image.

7. The method for extracting font strokes according to any one of claims 1 to 6, characterized in that: The step of optimizing the obtained font stroke image to make the image outline more obvious includes: The font stroke image is subjected to grayscale processing, and then the grayscale processed image is subjected to filtering processing, and finally the filtered image is subjected to binarization processing to obtain single-channel font stroke image data.

8. A font stroke extraction device, characterized in that: The device comprises: A graphics processing module, used to pre-process the font image of the target file for model training and / or font extraction; A model training module is used to train the font stroke extraction network model to obtain an optimized model; The stroke extraction module is used to extract the strokes of the target font by applying the optimization model.

9. Intelligent terminal, characterized in that: The intelligent terminal includes a memory, a processor, and a font stroke extraction program stored in the memory and executable on the processor. When the font stroke extraction program is executed by the processor, the steps of the font stroke extraction method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a font stroke extraction program, and when the font stroke extraction program is executed by a processor, the steps of the font stroke extraction method according to any one of claims 1 to 7 are implemented.

Images