Chinese character font generation algorithm for oversized font library
By using decoupling networks and fusion networks in the super-large font generation algorithm of Chinese character fonts, combined with diffusion model and loss function, the problem of incomplete generated font content is solved, and high-quality Chinese character font image generation is achieved.
Patent Information
- Application Number
- CN202411925005.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-25
- Publication Date
- 2025-04-18
AI Technical Summary
In the existing technology, when generating Chinese characters in the super-large font library, if pre-training is lacking, broken, and multiple strokes, the diffusion model is prone to generate problems such as font missing, broken, and multiple strokes, resulting in incomplete font content.
A Chinese character font generation algorithm for super-large font library is proposed. By decoupling the network, the font content and style are separated, the fusion network and diffusion model are used to generate high-quality Chinese character font images, and the quality of the generated images is adjusted through the MSE loss function and the content loss function.
It effectively solves the problem that the diffusion model generates Chinese characters incomplete font content in the case of a small amount of pre-training or no pre-training, and improves the content quality of the generated images.
Smart Images
Figure CN119849433A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of computer vision and image processing, and particularly relates to a Chinese character font generation algorithm for a super large character library. Background Art
[0002] Chinese characters are a writing system with a long history. They have been deeply integrated into people's lives and cannot be replaced. Chinese characters have a complex structure. Each character is composed of some basic strokes, and different combinations of these basic strokes will form different Chinese characters. It is precisely because of the complex structure and large number of Chinese characters that in order to generate a set of fonts, designers need to carefully consider the shape, proportion and smoothness of each Chinese character. Therefore, creating a complete set of fonts requires a lot of time and effort.
[0003] The time-consuming and laborious problem of manually generating Chinese fonts can be solved by artificial intelligence. At present, the research on font generation is still based on small fonts, and these advances mostly rely on the performance of diffusion models under a large amount of pre-training. However, there are fewer studies on super-large fonts, and large fonts require a lot of pre-training to achieve the desired goals. If there is no pre-training or only a small amount of pre-training, there will be problems such as missing strokes, broken strokes, and multiple strokes in the generated font. Summary of the invention
[0004] In view of this, the purpose of the present invention is to propose a Chinese character font generation algorithm for a very large character library, to improve the content quality of the generated image, and to solve the problem of incomplete content of Chinese character fonts generated by the diffusion model in the case of a small amount of pre-training or no pre-training.
[0005] The present invention is implemented by the following scheme: a Chinese character font generation algorithm for a super large character library, specifically comprising the following steps:
[0006] S1. Input a single target content font of different styles into the decoupling network, decouple the content and style of the fonts of the same content with different styles, and extract the content feature vector of the target content font;
[0007] S2, inputting a single target style font with different contents into a style encoder, and extracting style feature vectors of the target style font with different contents;
[0008] S3, fusing the extracted content feature vector and style feature vector in a fusion network, fusing the information into a diffusion model, and completing the content information of Chinese character fonts;
[0009] S4. Adjust the quality of the generated font image through the MSE loss function and the content loss function of the generated image.
[0010] Furthermore, in step S1, the decoupled network uses an improved ResNet18 network model.
[0011] Furthermore, in step S1, the improved ResNet18 network adds a convolution layer with a convolution kernel of 7×7 to the front end of the ResNet18 network, selects the feature extraction layer of ResNet18, and adds a convolution layer with a convolution kernel of 1×1 to the back end of the network.
[0012] Furthermore, in step S2, the style encoder uses the VGG16 network model.
[0013] Furthermore, in step S3, content stroke encoding is added to the fusion network to assist in the fusion of the content feature vector and the style feature vector.
[0014] Furthermore, in step S3, the content feature vector, the stroke code and the style feature vector are fused into a complete vector, and the complete vector is added to each diffusion step of the diffusion model.
[0015] Further, in step S3, the diffusion model is composed of forward diffusion and backward diffusion. The forward diffusion is to convert the font image into irregular noise, and the backward diffusion is the process of gradually converting the noise into a Chinese font image through a U-Net network model.
[0016] Further, in step S4, the MSE loss function L MSE The formula is:
[0017]
[0018] Where n is the number of samples, yi is the true value of the i-th sample, is the average value of all samples;
[0019] Content loss function L c The formula is:
[0020]
[0021] Where n is the number of samples, X oi is the content feature vector of the same content font in the ith different styles through the content decoupling network, and Xgi is the content feature vector of the i-th generated target font through the content decoupling network.
[0022] Furthermore, after step S4, multiple sets of super-large font libraries are used for pre-training, each set of font libraries has a different font style, and new fonts of super-large font libraries of other styles are generated after the training is completed.
[0023] Furthermore, 10 sets of super-large character libraries are used for pre-training, each of which contains 50,000 Chinese characters.
[0024] Compared with the prior art, the present invention has the following beneficial effects:
[0025] The present invention proposes an innovative Chinese character font generation algorithm for ultra-large character libraries. It separates font content and style through a decoupling network, and uses a fusion network and a diffusion model to generate high-quality Chinese character font images, thereby solving the problem of incomplete Chinese character font content generated by the diffusion model in the case of a small amount of pre-training or no pre-training. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 The figure is a schematic diagram of a method flow of an embodiment of the present invention.
[0027] Figure 2 A schematic diagram of generating Chinese character fonts from a super large character library according to an embodiment of the present invention. DETAILED DESCRIPTION
[0028] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.
[0029] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.
[0030] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.
[0031] like Figure 1 As shown, the present invention is implemented by the following scheme: a Chinese character font generation algorithm for a super large character library, specifically comprising the following steps:
[0032] S1. Input a single target content font of different styles into the decoupling network, decouple the content and style of the fonts of the same content with different styles, and extract the content feature vector of the target content font;
[0033] S2, inputting a single target style font with different contents into a style encoder, and extracting style feature vectors of the target style font with different contents;
[0034] S3, fusing the extracted content feature vector and style feature vector in a fusion network, fusing the information into a diffusion model, and completing the content information of Chinese character fonts;
[0035] S4. Adjust the quality of the generated font image through the MSE loss function and the content loss function of the generated image.
[0036] Furthermore, in step S1, the decoupled network is an improved ResNet18 network model, which can further improve performance and training effects. Specifically, the improved ResNet18 network adds a 7×7 convolution layer to the front end of the ResNet18 network, selects the feature extraction layer of ResNet18, and adds a 1×1 convolution layer to the back end of the network, which can further improve the accuracy of content feature extraction.
[0037] The specific steps of inputting a single target content font of different styles into the improved ResNet18 network model and extracting the content feature vector of the target content font are as follows: forward propagating the input data through the improved ResNet18 network model to extract the content feature vector, and the formula is:
[0038]
[0039] Among them, Xc is the content feature of the extracted target font, is a single input font image with the same content but different styles, f(,) is the decoupled network, and θ is the parameter of the decoupled network.
[0040] Further, in step S2, the style encoder uses a VGG16 network model, and the specific steps of inputting a single target style font with different contents into the VGG16 network model and extracting the style feature vector of the target style font are as follows: the style feature can be obtained by selecting the VGG16 network style layer, and here the output of the VGG16 network model with a fully connected layer is selected as the extracted style feature, and these style features include the unique style features of the font image, and the formula is:
[0041]
[0042] Among them, Xs is the style feature of the extracted target font, is a single input font image with the same style but different contents, g(,) is the style encoder, and θ is the parameter of the style encoder.
[0043] Furthermore, if Figure 2As shown, in step S3, stroke coding is added to the fusion network to assist in fusing the content feature vector and the style feature vector, and to supervise the integrity of the content stroke information. Specifically, the strokes of each font are encoded, and the encoding process is as follows: each font is composed of the most basic 32-bit stroke code. If a stroke is present, it will be marked as 1, and if it is absent, it will be marked as 0.
[0044] Specifically, the content feature vector, stroke encoding and style feature vector are fused into a complete vector through a fusion network, and the complete vector is added to each diffusion step of the diffusion model.
[0045] Furthermore, the specific steps of the diffusion model are as follows: forward diffusion mainly transforms the font image into irregular noise; and backward diffusion mainly transforms the noise into Chinese font image gradually through the U-Net network model.
[0046] Furthermore, in step S4, the quality of the generated image is adjusted by using the MSE loss function and the content feature loss function. The MSE loss function formula is:
[0047]
[0048] Where n is the number of samples, yi is the true value of the i-th sample, is the average value of all samples;
[0049] The content loss function formula is:
[0050]
[0051] Where n is the number of samples, X oi is the content feature vector of the same content font in the ith different styles through the content decoupling network, and Xgi is the content feature vector of the i-th generated target font through the content decoupling network.
[0052] Furthermore, after step S4, the present invention can use multiple sets of super-large character libraries for pre-training, each set of character libraries has different font styles. Specifically, 10 sets of super-large character libraries can be used for pre-training, each set of super-large character libraries has 50,000 Chinese characters, and new fonts of super-large font libraries of other styles are generated after the training is completed.
[0053] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0054] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0055] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0056] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0057] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection of the technical solution of the present invention.
Claims
1. A Chinese character font generation algorithm for a super large character library, characterized in that: The following steps are involved: S1. Input a single target content font of different styles into the decoupling network, decouple the content and style of the fonts of the same content with different styles, and extract the content feature vector of the target content font; S2, inputting a single target style font with different contents into a style encoder, and extracting style feature vectors of the target style font with different contents; S3, fusing the extracted content feature vector and style feature vector in a fusion network, fusing the information into a diffusion model, and completing the content information of Chinese character fonts; S4. Adjust the quality of the generated font image through the MSE loss function and the content loss function of the generated image.
2. The Chinese character font generation algorithm for a super large character library according to claim 1, characterized in that: In step S1, the decoupled network uses an improved ResNet18 network model.
3. The decoupling network according to claim 2, characterized in that: In step S1, the improved ResNet18 network adds a convolution layer with a convolution kernel of 7×7 to the front end of the ResNet18 network, selects the feature extraction layer of ResNet18, and adds a convolution layer with a convolution kernel of 1×1 to the back end of the network.
4. The Chinese character font generation algorithm for a super large character library according to claim 1, characterized in that: In step S2, the style encoder uses the VGG16 network model.
5. The network convergence according to claim 1, characterized in that: In step S3, content stroke encoding is added to the fusion network to assist in the fusion of the content feature vector and the style feature vector.
6. The network convergence according to claim 5, characterized in that: In step S3, the content feature vector, the stroke code and the style feature vector are fused into a complete vector, and the complete vector is added to each diffusion step of the diffusion model.
7. The Chinese character font generation algorithm for a super large character library according to claim 1, characterized in that: In step S3, the diffusion model consists of forward diffusion and backward diffusion. Forward diffusion is a process of converting the font image into irregular noise, and backward diffusion is a process of gradually converting the noise into a Chinese font image through a U-Net network model.
8. The Chinese character font generation algorithm for a super large character library according to claim 1, characterized in that: In step S4, the MSE loss function L MSE The formula is: Where n is the number of samples, yi is the true value of the i-th sample, is the average value of all samples; Content loss function L c The formula is: Where n is the number of samples, X oi is the content feature vector of the same content font in the ith different styles through the content decoupling network, and Xgi is the content feature vector of the i-th generated target font through the content decoupling network.
9. The Chinese character font generation algorithm for a super large character library according to claim 1, characterized in that: After step S4, multiple sets of super-large font libraries are used for pre-training, each set of super-large font libraries has a different font style, and new fonts of super-large font libraries of other styles are generated after the training is completed.
10. The Chinese character font generation algorithm for a super large character library according to claim 9, characterized in that: Use 10 sets of super-large character libraries for pre-training, each of which contains 50,000 Chinese characters.