A method for generating a Chinese character font
By constructing a font style transfer network model and combining skeleton extraction and attention modules, the complexity and poor quality of Chinese font generation in existing technologies are solved, and high-quality, fast personalized Chinese font generation is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHAANXI NORMAL UNIV
- Filing Date
- 2023-02-21
- Publication Date
- 2026-04-28
AI Technical Summary
Existing methods for generating Chinese character fonts are insufficient to produce personalized Chinese character fonts with high resolution, good effect, clear strokes, and realistic fonts. In particular, it is difficult for ordinary people to quickly build Chinese handwritten fonts that match their own style.
A font style transfer network model is constructed. The model combines a generator and a discriminator, and utilizes an attention module between the skeleton extraction module, encoder, and decoder to perform font style transfer. The generator extracts the skeleton and feature information of the source font and generates a font image. The discriminator judges the authenticity of the font. Data augmentation and loss function are used to optimize the network parameters during training.
It improves the quality and speed of Chinese character font generation, resulting in more complete character shapes and clearer strokes, suitable for the personalized needs of ordinary people, and simplifies the complexity of professional font design.
Smart Images
Figure CN116152374B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer processing technology, and specifically relates to a method for generating Chinese character fonts. Background Technology
[0002] Calligraphy is an art form embodied in Chinese characters, manifested in brushstrokes, style, and composition. With a history spanning over three thousand years, Chinese calligraphy holds a significant place in world art history, embodying the Chinese way of thinking and aesthetic sensibilities. Calligraphic works throughout history are among the most representative symbols of Chinese culture, not only recording historical culture, events, and environments in their content, but also exhibiting distinct historical and personal characteristics in their writing style, such as Su Shi's "Ode to the Red Cliff," Ouyang Xun's "Zhongni Mengdian Tie," and Wang Xizhi's "Preface to the Poems Composed at the Orchid Pavilion." Today, the purpose of learning traditional calligraphy is not merely for personal study, but also for the inheritance and promotion of traditional Chinese culture. Because traditional calligraphy works are mostly found on stone, silk, bamboo slips, or paper, they suffer from long-term historical vicissitudes, including dampness, mold, stains, aging, damage, and weathering, making them difficult to preserve, pass down, and develop. Traditionally, calligraphers created fonts primarily by hand-drawing typefaces, which were then digitally processed and stored in computers. However, creating a complete set of fonts by hand requires a significant amount of time and manpower.
[0003] With the development of science and technology, the digital development of calligraphy art has ushered in new opportunities. Currently, typical Chinese character font generation methods can be divided into two categories: stroke extraction-based methods and image-to-image conversion-based methods. The former divides Chinese character font generation into two stages: stroke extraction and stroke reconstruction. However, due to the complexity of stroke structure and the diversity of handwriting styles, stroke extraction algorithms often produce unreasonable stroke extraction results or incorrect stroke structures. The latter treats Chinese character font generation as an image-to-image conversion problem, utilizing convolutional neural networks, such as generative adversarial networks (GANs) and graph networks, to achieve more realistic and higher-quality Chinese character synthesis. However, since Chinese font generation is not a fault-tolerant task, it is an uncontrolled and unpredictable process. Any blurring, ghosting, or artifacts can lead to the failure of font generation and a deterioration in the quality of the synthesized font.
[0004] Currently, most commercial font generation focuses on professional font design, a time-consuming and labor-intensive task. For the average person, generating calligraphic fonts remains a challenging task due to the following three reasons: 1) The structure and style of Chinese characters are complex, with different people exhibiting different handwriting styles and stroke shapes. 2) The vocabulary of Chinese characters is vast, making it difficult for people to correctly write characters with a consistent style. 3) Handwritten fonts on electronic devices are created manually, primarily by professional font designers. Their performance largely depends on meticulous adjustments to each character shape. The complex and flexible structure of Chinese characters makes it impossible for ordinary customers to perfectly extract strokes or shapes; therefore, quickly building personalized handwritten Chinese fonts suitable for ordinary people is not feasible. Summary of the Invention
[0005] The technical problem to be solved by this invention is to overcome the defects of existing font generation technology and provide a Chinese character font generation method with high resolution, good effect, clear strokes and realistic font.
[0006] The technical solution adopted to solve the above-mentioned technical problems is a Chinese character font generation method, which consists of the following steps:
[0007] S1. Construct the target font dataset
[0008] Based on the target font to be generated, calligraphy works images of the target font are obtained from the Internet. Each calligraphy work image is divided into multiple single character images by character. The size of the single character images is uniformly p×p and the format is .png. The single character images are processed in grayscale to obtain the target font dataset.
[0009] S2. Construct the source font dataset
[0010] Select a font as the source font, unpack the TTF font file of the font to obtain the source font image, and make the size and format of the source font image consistent with the single character image in step S2 to obtain the source font dataset;
[0011] S3. Construct the training dataset
[0012] The target font dataset obtained in step S2 is augmented and expanded by rotating and adding noise to obtain a training dataset for training the model.
[0013] S4. Constructing a font style transfer network model
[0014] The font style transfer network model consists of a generator and a discriminator. The generator is used to extract the font skeleton information and feature information of the input source font image and generate a font image. The discriminator is used to judge the font authenticity of the generated font image.
[0015] The generator consists of a skeleton extraction module, an encoder, a decoder, and an attention module. The output of the skeleton extraction module is connected to the decoder and is used to extract the font skeleton information of the input source font image. The encoder is connected to the decoder through the attention module and is used to extract the font feature information and category information of the input source font image. The decoder is used to fuse and decode the font skeleton information, font feature information, and category information to generate a font image.
[0016] The attention module is composed of multiple W-attention modules and SW-attention modules connected alternately. The W-attention module is used to divide the feature map into windows and perform local attention operations in different windows to reduce the number of parameters. The SW-attention module is used to exchange information between windows.
[0017] S5. Training the font style transfer network model
[0018] The source font dataset and training dataset are paired by character. The font style transfer network model is trained using the paired font images from the source font dataset and training dataset. The source font images from the source font dataset are input into the generator to obtain fake target font images. At the same time, the fake target font images and training dataset images are input into the discriminator. The network model parameters are updated by backpropagating through the discriminator loss function. The training is repeated until the set upper limit of the number of iterations is reached or the loss is less than or equal to the preset minimum value, which completes the training of the font style transfer network model.
[0019] S6. Automatic generation of Chinese character fonts
[0020] By inputting any font image into the font style transfer network model, you can obtain Chinese fonts with that font style.
[0021] As a preferred technical solution, the skeleton extraction module in step S4 is based on the side output residual network SRN, extracts the side output of each layer of the encoder, and adopts a non-direct combination residual learning method. It gradually fuses the feature maps of each layer of the encoder upward through cross-layer short connections, so that the features that the lower layers could not learn are supplemented by the shallow layers to improve the learning ability of the network with limited depth.
[0022] As a preferred technical solution, the encoder in step S4 consists of 4 to 10 downsampling modules connected sequentially via residuals. Each downsampling module consists of a 4×4 convolutional block C4, a 4×4 dilated convolution DC, and a 1×1 convolutional block C1 with a dilation rate r of 2. The output of the 4×4 convolutional block C4 is connected to the input of the 4×4 dilated convolution DC. The output of the 4×4 convolutional block C4 and the output of the 4×4 dilated convolution DC are fused and then connected to the input of the 1×1 convolutional block C1. The output of the convolutional block C1 is connected to the input of the decoder through an attention module.
[0023] As a preferred technical solution, in step S4, the decoder consists of 4 to 10 sequentially connected upsampling modules, and the upsampling modules consist of deconvolution blocks.
[0024] As a preferred technical solution, the discriminator loss function L in step S5 is,
[0025]
[0026] In the formula, L cGAN (G,D) is the adversarial loss function, L L1 Let L be the L1 loss function, L r For the reconstruction loss function, G is the generator, D is the discriminator, λ1 is the weight coefficient of the L1 loss function, and λ2 is the weight coefficient of the reconstruction loss function.
[0027] The adversarial loss function L cGAN (G,D) is used to simultaneously improve the discriminator's judgment ability and the realism of the fonts generated by the generator. The formula is:
[0028]
[0029] In the formula, x represents the true data distribution, z represents the random noise input to the generator, and p data (x) represents the actual data input, p z (z) represents the data distribution of the input noise, G(z) represents the data distribution of the image generated by the generator G, D(·) represents the probability that the discriminator D judges the input image as real, and E(·) is the expected value of the distribution function;
[0030] The L1 loss function is used to obtain the difference between the fake images generated by the generator and the real images. The formula is:
[0031] L L1 =|G n (z|y)-x n |
[0032] In the formula, G n (z|y) represents the nth false data generated by the generator after adding conditional information y, and xn This represents the nth real data point;
[0033] The reconstruction loss function is used to improve the quality of the generated image, and the formula is:
[0034]
[0035] In the formula, E z,y The loss distributions for reconstructing and generating images are defined as follows: G(z|G(z,y)) represents the reconstructed image, G(z|y) represents the generated image, and MSE represents the mean squared error.
[0036] The beneficial effects of this invention are as follows:
[0037] The skeleton extraction module in the font style transfer network model of this invention uses font skeleton information to guide the generation of Chinese characters, making the generated Chinese characters more complete. This invention uses a w-attention module and a sw-attention module between the encoder and decoder to perform attention calculations alternately. This preserves the global information brought by the attention mechanism while greatly optimizing the problems of parameter explosion and increased training difficulty caused by the superposition of attention mechanisms. It improves the problems of stroke adhesion, loss of details, and even missing strokes in fonts generated by existing methods, while reducing the training time of the network and improving the quality of generated Chinese characters. This invention can also input other fonts such as handwritten fonts into the font style transfer network model, providing convenience for the generation and diversified development of Chinese calligraphy. Attached Figure Description
[0038] Figure 1 This is a flowchart illustrating the Chinese character font generation method of the present invention.
[0039] Figure 2 This is a schematic diagram of the font style transfer network model of the present invention.
[0040] Figure 3 This is a schematic diagram of the downsampling module of the present invention.
[0041] Figure 4 This is a schematic diagram of the upsampling module of the present invention.
[0042] Figure 5 This is a schematic diagram of the structure of the W-attention module of the present invention.
[0043] Figure 6 This is a schematic diagram of the SW-attention module of the present invention.
[0044] Figure 7 This invention relates to a font style transfer network model that generates boldface images from input boldface images. Detailed Implementation
[0045] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments, but the present invention is not limited to the following embodiments.
[0046] This example uses the generation of boldface characters.
[0047] exist Figure 1 The Chinese character font generation method of this embodiment consists of the following steps:
[0048] S1. Construct the target font dataset
[0049] Based on the target font to be generated, calligraphy works images of the target font are obtained from the Internet. Each calligraphy work image is divided into multiple single character images by character. The size of the single character images is uniformly 256×256 and the format is .png. The single character images are processed in grayscale to obtain the target font dataset.
[0050] S2. Construct the source font dataset
[0051] Select the sans-serif font as the source font, unpack the sans-serif font's ttf file to obtain the source font image, and process the size and format of the source font image to be consistent with the single character image in step S2 to obtain the source font dataset;
[0052] S3. Construct the training dataset
[0053] The target font dataset obtained in step S2 is augmented and expanded by rotating and adding noise to obtain a training dataset for training the model.
[0054] S4. Constructing a font style transfer network model
[0055] The font style transfer network model consists of a generator and a discriminator, such as... Figure 2 The generator is used to extract the font skeleton information and feature information of the input source font image and generate a font image, while the discriminator is used to judge the font authenticity of the generated font image.
[0056] The generator in this embodiment consists of a skeleton extraction module, an encoder, a decoder, and an attention module. The output of the skeleton extraction module is connected to the decoder and is used to extract the font skeleton information of the input source font image. The encoder is connected to the decoder through the attention module and is used to extract the font feature information and category information of the input source font image. The decoder is used to fuse and decode the font skeleton information, font feature information, and category information to generate a font image.
[0057] The encoder in this embodiment consists of 4 to 10 downsampling modules connected sequentially via residuals. Each downsampling module comprises a 4×4 convolutional block C4, a 4×4 dilated convolutional block DC, and a 1×1 convolutional block C1. Figure 3 The dilation rate r is 2. The output of the 4×4 convolutional block C4 is connected to the input of the 4×4 dilated convolution DC. The output of the 4×4 convolutional block C4 is fused with the output of the 4×4 dilated convolution DC and then connected to the input of the 1×1 convolutional block C1. The output of the convolutional block C1 is connected to the input of the decoder through the attention module.
[0058] The skeleton extraction module in this embodiment is based on the Side Output Residual Network (SRN) to extract the side output of each layer of the encoder. It adopts a non-direct combination residual learning method and gradually fuses the feature maps of each layer of the encoder upwards through cross-layer short connections. This allows the features that the lower layers could not learn to be supplemented by the shallower layers, thereby improving the learning ability of the network with limited depth.
[0059] The attention module in this embodiment consists of multiple W-attention modules and SW-attention modules connected alternately. The W-attention module is used to divide the feature map into windows and perform local attention operations within different windows to reduce the number of parameters. The SW-attention module is used to exchange information between windows.
[0060] The decoder in this embodiment consists of 4 to 10 sequentially connected upsampling modules, each of which is composed of deconvolution blocks, such as... Figure 4 During the decoding process, the feature information obtained by the upsampling module is fused with the encoder features after being calculated by the W-attention module and the SW-attention module, and used as the input for the next layer of decoding.
[0061] By adding a w-attention module and a sw-attention module to the jump connection between the encoder and the decoder to perform attention calculations alternately, the global information brought by the attention mechanism is preserved, and the problems of parameter explosion and increased training difficulty caused by the superposition of attention mechanisms are greatly optimized. This improves the problems of stroke sticking, loss of details, and even missing strokes in the fonts generated by existing methods, while reducing the training time of the network and generating Chinese fonts with higher quality and faster speed.
[0062] S5. Training the font style transfer network model
[0063] The source font dataset and training dataset are paired by character. The font style transfer network model is trained using the paired font images from the source font dataset and training dataset. The source font images from the source font dataset are input into the generator to obtain fake target font images. At the same time, the fake target font images and training dataset images are input into the discriminator. The network model parameters are updated by backpropagating through the discriminator loss function. The training is repeated until the set upper limit of the number of iterations is reached or the loss is less than or equal to the preset minimum value, which completes the training of the font style transfer network model.
[0064] The discriminant loss function L in this embodiment is:
[0065]
[0066] In the formula, L cGAN (G,D) is the adversarial loss function, L L1 Let L be the L1 loss function, L r For the reconstruction loss function, G is the generator, D is the discriminator, λ1 is the weight coefficient of the L1 loss function, λ1 = 0.4, and λ2 is the weight coefficient of the reconstruction loss function, λ2 = 0.6.
[0067] The adversarial loss function L in this embodiment cGAn (G,D) is used to simultaneously improve the discriminator's judgment ability and the realism of the fonts generated by the generator. The formula is:
[0068]
[0069] In the formula, x represents the true data distribution, z represents the random noise input to the generator, and p data (x) represents the actual data input, p z (z) represents the data distribution of the input noise, G(z) represents the data distribution of the image generated by the generator G, D(·) represents the probability that the discriminator D judges the input image as real, and E(·) is the expected value of the distribution function.
[0070] In this embodiment, the L1 loss function is used to obtain the difference between the fake image generated by the generator and the real image. The formula is:
[0071] L L1 =|G n (z|y)-x n |
[0072] In the formula, G n (z|y) represents the nth false data generated by the generator after adding conditional information y, and x n This represents the nth real data point;
[0073] The reconstruction loss function in this embodiment is used to improve the quality of the generated image, and the formula is:
[0074] L r =E z,y [||G(z|G(z,y),G(z|y)|| MSE ]
[0075] In the formula, E z,y The loss distributions for reconstructing and generating images are defined as follows: G(z|G(z,y)) represents the reconstructed image, G(z|y) represents the generated image, and MSE represents the mean squared error.
[0076] S6. Automatic generation of Chinese character fonts
[0077] By inputting the image of the boldface font into the font style transfer network model, you can obtain Chinese fonts with the boldface font style, such as... Figure 7 .
Claims
1. A method for generating Chinese character fonts, characterized in that... It consists of the following steps: S1. Construct the target font dataset Based on the target font to be generated, calligraphy works images of the target font are obtained from the Internet. Each calligraphy work image is divided into multiple single character images by character. The size of the single character images is uniformly p×p and the format is .png. The single character images are processed in grayscale to obtain the target font dataset. S2. Construct the source font dataset Select a font as the source font, unpack the TTF font file of the font to obtain the source font image, and make the size and format of the source font image consistent with the single character image in step S2 to obtain the source font dataset; S3. Construct the training dataset The target font dataset obtained in step S2 is augmented and expanded by rotating and adding noise to obtain a training dataset for training the model. S4. Constructing a font style transfer network model The font style transfer network model consists of a generator and a discriminator. The generator is used to extract the font skeleton information and feature information of the input source font image and generate a font image. The discriminator is used to judge the font authenticity of the generated font image. The generator consists of a skeleton extraction module, an encoder, a decoder, and an attention module. The output of the skeleton extraction module is connected to the decoder and is used to extract the font skeleton information of the input source font image. The encoder is connected to the decoder through the attention module and is used to extract the font feature information and category information of the input source font image. The decoder is used to fuse and decode the font skeleton information, font feature information, and category information to generate a font image. The attention module is composed of multiple W-attention modules and SW-attention modules connected alternately. The W-attention module is used to divide the feature map into windows and perform local attention operations in different windows to reduce the number of parameters. The SW-attention module is used to exchange information between windows. S5. Training the font style transfer network model The source font dataset and training dataset are paired by character. The font style transfer network model is trained using the paired font images from the source font dataset and training dataset. The source font images from the source font dataset are input into the generator to obtain fake target font images. At the same time, the fake target font images and training dataset images are input into the discriminator. The network model parameters are updated by backpropagating through the discriminator loss function. The training is repeated until the set upper limit of the number of iterations is reached or the loss is less than or equal to the preset minimum value, which completes the training of the font style transfer network model. S6. Automatic generation of Chinese character fonts By inputting any font image into the font style transfer network model, you can obtain Chinese fonts with that font style.
2. The Chinese character font generation method according to claim 1, characterized in that, In step S4, the skeleton extraction module is based on the side output residual network SRN. It extracts the side output of each layer of the encoder and adopts a non-direct combination residual learning method. It gradually fuses the feature maps of each layer of the encoder upward through cross-layer short connections, so that the features that the lower layers could not learn are supplemented by the shallow layers to improve the learning ability of the network with limited depth.
3. The Chinese character font generation method according to claim 1, characterized in that, The encoder in step S4 consists of 4 to 10 downsampling modules connected sequentially via residuals. Each downsampling module is composed of a 4×4 convolutional block C4, a 4×4 dilated convolution DC, and a 1×1 convolutional block C1. The dilation rate r is 2. The output of the 4×4 convolutional block C4 is connected to the input of the 4×4 dilated convolution DC. The output of the 4×4 convolutional block C4 and the output of the 4×4 dilated convolution DC are fused and then connected to the input of the 1×1 convolutional block C1. The output of the convolutional block C1 is connected to the input of the decoder through an attention module.
4. The Chinese character font generation method according to claim 1, characterized in that, In step S4, the decoder consists of 4 to 10 sequentially connected upsampling modules, each of which is composed of a deconvolution block.
5. The Chinese character font generation method according to claim 1, characterized in that, In step S5, the discriminant loss function L is: L=L cGAN (G,D)+λ1L L1 (G)+λ2L2(G) In the formula, L cGAN (G,D) is the adversarial loss function, L L1 Let L be the L1 loss function, L r For the reconstruction loss function, G is the generator, D is the discriminator, λ1 is the weight coefficient of the L1 loss function, and λ2 is the weight coefficient of the reconstruction loss function. The adversarial loss function L cGAN (G,D) is used to simultaneously improve the discriminator's judgment ability and the realism of the fonts generated by the generator. The formula is: In the formula, x represents the true data distribution, z represents the random noise input to the generator, and p data (x) represents the actual data input, p z (z) represents the data distribution of the input noise, G(z) represents the data distribution of the image generated by the generator G, D(·) represents the probability that the discriminator D judges the input image as real, and E(·) is the expected value of the distribution function; The L1 loss function is used to obtain the difference between the fake images generated by the generator and the real images. The formula is: L L1 =|G n (z|y)-x n | In the formula, G n (z|y) represents the nth false data generated by the generator after adding conditional information y, and x n This represents the nth real data point; The reconstruction loss function is used to improve the quality of the generated image, and the formula is: L r =E z,y [||G(z|G(z,y),G(z|y)|| MSE ] In the formula, E z,y The loss distributions for reconstructing and generating images are defined as follows: G(z|G(z,y)) represents the reconstructed image, G(z|y) represents the generated image, and MSE represents the mean squared error.