Attribute-controllable font generation method based on diffusion model

Abstract

The invention relates to the technical field of font generation in the field of computer mode recognition, in particular to an attribute-controllable font generation method based on a diffusion model. Preparing data including font images and font attributes; performing data preprocessing based on the font image and the attribute to obtain a standardized data set; a generation network based on a pre-training potential spatial diffusion model is constructed, a content encoder is introduced to extract font content features, an attribute encoder maps attribute vectors, an attribute control adapter is designed to integrate attribute information into the generation process, and effective decoupling of font content and style attributes is achieved; and finally, generating a high-quality font with a specified style by inputting a font content image and a target attribute vector, and supporting style conversion between any fonts based on an attribute value. The problems of high design cost, single style, inflexible attribute control and the like in the traditional font generation technology can be solved.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and pattern recognition technology, specifically relating to a method for generating attribute-controllable fonts based on a diffusion model. Background Technology

[0002] Font generation technology aims to automatically create character sets that meet specific visual style and structural requirements using computer algorithms. Attribute-controlled font generation further focuses on fine-grained and interpretable manipulation of key visual attributes of fonts during the generation process to meet diverse design needs. This technology has broad application potential in brand identity design, personalized user interfaces, advertising creativity, and digital art creation, providing designers, developers, and ordinary users with efficient, flexible, and high-quality font customization tools. By precisely controlling font attributes, brands can quickly generate uniquely recognizable fonts, strengthening brand visual consistency; by efficiently exploring rich font style combinations, it can inspire artists' creative inspiration and accelerate the design process. Attribute-controlled font generation technology is an important research direction at the intersection of computer vision and artificial intelligence, playing a key role in promoting the automation and personalization of digital content creation.

[0003] The rapid development of generative artificial intelligence, particularly diffusion models, has provided strong technical support for attribute-controllable font generation. Diffusion models, represented by Denoising Diffusion Probability Model (DDPM), Score-based Models, and Latent Diffusion Models (LDM), have achieved significant breakthroughs in recent years, demonstrating potential to surpass traditional Generative Adversarial Networks (GANs) in terms of image synthesis quality, diversity, and controllability. Unlike GANs and other generative networks, diffusion models progressively add noise to the image in the forward stage until it is corrupted into pure Gaussian noise, and then learn the process of restoring the original image from the Gaussian noise in the inverse stage. In the field of font generation, the accumulation of large-scale, high-quality open-source font datasets and the continuous improvement of GPU computing power have laid the data and computing power foundation for training complex diffusion models. Diffusion models have unique advantages in capturing subtle stroke features, structural coherence, and rich stylistic variations in fonts, providing new possibilities for generating highly realistic and structurally accurate fonts. Meanwhile, the iterative denoising process inherent in the diffusion model creates favorable conditions for fine-grained intervention in the generation process (such as through conditional guidance or latent space operations) to control specific font attributes.

[0004] Compared to traditional parametric font design, stroke-based generation, or early GAN-based font generation methods, attribute-controlled font generation technology based on diffusion models faces unique challenges and complexities. First, font generation demands extremely high structural consistency and global coordination. Each generated character not only needs to be structurally accurate and aesthetically pleasing but also needs to maintain strict uniformity in core attributes such as stroke style and proportion across the entire character set. This requires the model to deeply understand and accurately model the inherent structure and design rules of characters. Second, achieving effective decoupling and precise control of attributes is a core challenge. Font attributes (such as "bold" and "italic") are often highly coupled and abstract visual concepts. Finding separable and interpretable control dimensions corresponding to these attributes in the model's latent space or during the generation process, and avoiding accidental impact on other attributes when adjusting one attribute, requires sophisticated model architecture design. Traditional template- or rule-based methods lack flexibility and struggle to cover rich styles; while early deep generative models (such as GANs) have limitations in training stability, pattern coverage, and control precision under complex conditions (especially multiple decoupled attributes), easily leading to problems such as pattern collapse, structural distortion, or attribute adhesion. Therefore, developing attribute-controllable font generation technology based on diffusion models is of key significance for breaking through the bottlenecks of existing methods, fully releasing the creative potential of font design, and improving the efficiency and personalization of digital content creation. Summary of the Invention

[0005] Purpose of the invention: To address the shortcomings of existing technologies, this invention provides a method for generating attribute-controllable fonts based on a diffusion model, which is used to generate high-quality fonts with specified style attributes and supports style conversion between arbitrary fonts based on attribute values.

[0006] Technical solution: This invention constructs a font generation network based on a pre-trained latent space diffusion model. By introducing a content encoder to extract font content features, an attribute encoder to map attribute vectors, and an attribute control adapter to integrate attribute information into the generation process, the invention achieves effective decoupling of font content and style attributes. Finally, by inputting a font content image and a target attribute vector, a high-quality font with a specified style is generated.

[0007] This invention provides a method for generating attribute-controllable fonts based on a diffusion model. The specific steps of the attribute-controllable font generation model based on the diffusion model are as follows:

[0008] S1: Obtain the original font image and attribute data;

[0009] S2: Based on the above font image and attribute data, preprocessing is performed to obtain a standardized dataset.

[0010] S3: The dataset created based on S2 is used to train the generation model of attribute-controllable fonts based on the diffusion model;

[0011] S4: Given a font content image and a target attribute vector, generate a high-quality font with a specified style using the generative model obtained in S3.

[0012] The specific method for obtaining the original font image and attribute data as described in S1 is as follows:

[0013] S11: Read the original font image data and convert it into NumPy data format;

[0014] S12: Read the font attribute data and convert it to NumPy data format through a format conversion operation.

[0015] The specific method for preprocessing the font image and attribute data as described in S2 is as follows:

[0016] S21: For the NumPy format image data obtained in S11, perform binarization and size normalization operations in sequence to remove redundant image information and unify image size, and build a standardized image dataset.

[0017] S22: Standardize the font attribute NumPy data after conversion in S12, linearly mapping the original attribute data in the range of 0-100 to the range of 0-1, thereby normalizing the attribute data and facilitating subsequent model training and feature extraction.

[0018] The specific steps of the attribute-controllable font generation model based on the diffusion model described in S3 are as follows:

[0019] S31: For the content image and target image preprocessed in S21, the latent diffusion model (LDM) is used as the denoising generator. The images are encoded into the latent space by a pre-trained variational autoencoder (VAE) to obtain the content latent vector and the target latent vector.

[0020] S32: The target potential vector obtained in S31 is fused with the noise information containing noise intensity t to form the input feature, which is then input into the UNet denoising structure for training.

[0021] S33: The content latent vector obtained in S31 is processed by the content encoder to extract global and local content features at different scales;

[0022] S34: Input the content features extracted in S33 into the multi-scale content aggregation module of UNet to preserve the font content information during the denoising process;

[0023] S35: Subtract the target font attribute and the content font attribute from the preprocessed S22 to obtain the difference attribute. The difference attribute is processed by the attribute attention module to generate the attribute feature difference vector and attribute channel attention feature maps of different scales.

[0024] S36: Input the difference vector and feature map generated in S35 into the denoising UNet structure to provide style attribute information for font denoising;

[0025] S37: The UNet network architecture based on content feature and style attribute conditional injection can achieve noise prediction. By performing reverse denoising on the noisy image obtained in step S32 using the noise predicted by this UNet network, the target font image can be generated.

[0026] S38: To adapt the font style to the font content, a discriminator based on a CNN network is added during network training. The discriminator outputs two classification heads: one is used to distinguish between real and fake images (the discriminator uses the rasterized image of the given font as the real sample and the output of the denoising generator as the fake sample), and the other is used to predict font attributes.

[0027] Furthermore, the pre-trained variational autoencoder (VAE) used in S31 is trained on a large image dataset (ImageNet), and the parameters of its encoder and decoder remain frozen during training; initially, the encoder transforms the input image (size: H×W) into a four-dimensional latent representation z∈R. 4×H / 8×W / 8 This latent representation serves as the network input for the diffusion and denoising processes in the diffusion model.

[0028] Furthermore, the image noise addition process in S32 can be described as a process involving random variables z0, z1, ..., z T The process of forming a Markov chain, in which noise is injected, can be described as follows:

[0029]

[0030] Where t ~ [0,T], ò ∈R 4×H / 8×W / 8 It is the Gaussian noise added at each step, α t =1-β t , β i ~(0,1) is a fixed parameter of variance.

[0031] Furthermore, the content encoder in S33 outputs multi-scale font content features through cascaded downsampling of deep blocks with spectral normalization, combined with a self-attention module at a specific level, using multi-level residual connections. Achieve multi-scale decoupling and semantic preservation of font content features.

[0032] Furthermore, the multi-scale content aggregation module in S34 will aggregate each content feature from S33. Content features are injected into UNet via the MCA module. Encoder c (MCA) will then extract the content features. Compared with previous UNet block features r i Connection, thereby generating channel information feature I c To improve the capability of adaptive selective channel fusion, in I... c Channel attention is applied, using average pooling, two 1×1 convolutions, and an activation function. This attention produces a global channel-aware vector W. c Used to perform channel information feature I through channel multiplication c We then perform weighting. After the residual connections, we use 1×1 convolutions to reduce IL. c The number of channels is used to obtain the output I. CO .

[0033] Furthermore, the attribute attention module in S35 is used to achieve controllable adjustment of the attributes of the generated font image. Specifically, the attribute subscript index embedding is converted into a two-dimensional feature map, and an attribute feature difference vector γ(a,b) is generated through element-wise operations of attribute differences and attribute embeddings. Global information is generated by dimensionality insertion and weight multiplication, and then an attention map is generated through channel attention and multiplied with the global information. Finally, it is convolved and scaled to various scales of UNet to obtain the attribute channel attention feature map α. * (a,b) is used for personalized adjustments to attributes such as stroke thickness and slant, as well as for focusing on important attributes.

[0034] Furthermore, in S36, the attribute feature difference vector and the UNet feature block are fused through a cross-attention mechanism. The attribute channel attention feature maps of different scales are then concatenated with the corresponding UNet feature blocks. The concatenated features are then enhanced through two layers of channel attention and self-attention layers for cross-channel and cross-space feature interaction. Finally, the multi-scale fused features are output through the activation function, realizing the hierarchical transfer of attribute style information in the denoising UNet structure.

[0035] Furthermore, in step S37, during the reverse denoising or sampling process, the UNet model can predict the noise vector. θ (z t ,t,e c ,γ(a,b),α * (a,b)), hence z t-1 It can be obtained using the following formula:

[0036]

[0037] Where σ t It is a hyperparameter, and z′~(0,I) is random noise.

[0038] Furthermore, in S3, when training the attribute-controllable font generation model based on the diffusion model, a combination of multiple loss functions is used to achieve attribute-controllable font generation:

[0039] Generate adversarial loss L G Improve image realism using the cross-entropy function;

[0040]

[0041] Denoising loss L MSE The diffusion process is optimized by using the mean square error between the predicted and target values.

[0042]

[0043] Content loss L content Smooth the pixel differences between the content image vector and the generated image vector using L1 constraints;

[0044]

[0045] Style loss L style VGG network is used to extract feature differences to ensure style consistency;

[0046]

[0047] Attribute loss L attribute Calculate the difference between the generated image and the target image attributes while preserving the source image attributes;

[0048]

[0049] Each loss item is weighted according to a preset weight to form the generator loss.

[0050] L Generator =λ1L G +λ2L MSE +λ3L content +λ4L style +λ5L attribute

[0051] Where λ1, λ2...λ5 are the hyperparameters of the loss function.

[0052] The discriminator loss includes image authenticity cross-entropy loss and attribute prediction smoothing L1 loss, which together optimize the discrimination ability.

[0053] L Discrimintor =L D +L′ attribute

[0054] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0055] The attribute-controllable font generation method based on a diffusion model proposed in this invention fully considers the coupling relationship between content and style in font generation, the consistency requirements of complex character structures, and the generalization challenge of generating unseen styles, making it well applicable to diverse font design scenarios. Compared with existing technologies, the model proposed in this invention can fully utilize the interaction mechanism of content features and attribute information during the iterative denoising process of the diffusion model, solving the problem of maintaining complex character structures from the perspective of multi-scale content aggregation, and overcoming the limitations of style parameter adhesion in traditional methods from the perspective of attribute decoupling. This effectively improves the model's ability to accurately control font attributes and the robustness of complex character generation. Attached Figure Description

[0056] Figure 1 A diagram illustrating the font dataset and its attribute comparison;

[0057] Figure 2 This is a basic schematic diagram of the diffusion model;

[0058] Figure 3 A network framework diagram for generating attribute-controllable fonts;

[0059] Figure 4 This is a diagram of the multi-scale content aggregation module of the present invention;

[0060] Figure 5 This is the attribute attention map of the present invention;

[0061] Figure 6 This is a diagram illustrating the effect of attribute-based font generation according to the present invention. Detailed Implementation

[0062] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0063] This invention discloses a method for generating attribute-controllable fonts based on a diffusion model, see [link to relevant documentation]. Figures 1 to 6 As shown, the attribute-controllable font generation method based on the diffusion model specifically includes the following steps:

[0064] S1: Obtain the original font image and attribute data;

[0065] S2: Based on the above font image and attribute data, preprocessing is performed to obtain a standardized dataset.

[0066] S3: The dataset created based on S2 is used to train the generation model of attribute-controllable fonts based on the diffusion model;

[0067] S4: Given a font content image and a target attribute vector, generate a high-quality font with a specified style using the generative model obtained in S3.

[0068] Reference Figure 1 The specific method for obtaining the original font image and attribute data as described in S1 is as follows:

[0069] S11: Read the original font image data and convert it into NumPy data format;

[0070] S12: Read the font attribute data and convert it to NumPy data format through a format conversion operation.

[0071] The specific method for preprocessing the font image and attribute data as described in S2 is as follows:

[0072] S21: For the NumPy format image data obtained in S11, perform binarization and size normalization operations in sequence to remove redundant image information and unify image size, and build a standardized image dataset.

[0073] S22: Standardize the font attribute NumPy data after conversion in S12, linearly mapping the original attribute data in the range of 0-100 to the range of 0-1, thereby normalizing the attribute data and facilitating subsequent model training and feature extraction.

[0074] The specific steps of the attribute-controllable font generation model based on the diffusion model described in S3 are as follows:

[0075] S31: For the content image and target image preprocessed in S21, the latent diffusion model (LDM) is used as the denoising generator. The images are encoded into the latent space by a pre-trained variational autoencoder (VAE) to obtain the content latent vector and the target latent vector.

[0076] Reference Figure 3 The pre-trained variational autoencoder (VAE) used is trained on a large image dataset (ImageNet), and the parameters of its encoder and decoder are frozen during training. Initially, the encoder transforms the input image (size: H×W) into a four-dimensional latent representation z∈R. 4×H / 8×W / 8 This latent representation serves as the network input for the diffusion and denoising processes in the diffusion model.

[0077] S32: The target potential vector obtained in S31 is fused with the noise information containing noise intensity t to form the input feature, which is then input into the UNet denoising structure for training.

[0078] Reference Figure 2The process of adding noise to an image can be described as a process involving random variables z0, z1, ..., z T The process of forming a Markov chain, in which noise is injected, can be described as follows:

[0079]

[0080] Where t ~ [0,T], ò ∈R 4×H / 8×W / 8 It is the Gaussian noise added at each step, α t =1-β t , β i ~(0,1) is a fixed parameter of variance.

[0081] S33: The content latent vector obtained in S31 is processed by the content encoder to extract global and local content features at different scales;

[0082] Reference Figure 3 The content encoder outputs multi-scale font content features through cascaded downsampling of deep blocks with spectral normalization, combined with self-attention modules at specific levels, using multi-level residual connections. Achieve multi-scale decoupling and semantic preservation of font content features.

[0083] S34: Input the content features extracted in S33 into the multi-scale content aggregation module of UNet to preserve the font content information during the denoising process;

[0084] Reference Figure 3 and Figure 4 S34 will process each content feature from S33 Content features are injected into UNet via the MCA module. Encoder c (MCA) will then extract the content features. Compared with previous UNet block features r i Connection, thereby generating channel information feature I c To improve the capability of adaptive selective channel fusion, in I... c Channel attention is applied, using average pooling, two 1×1 convolutions, and an activation function. This attention produces a global channel-aware vector W. c Used to perform channel information feature I through channel multiplication c We then perform weighting. After the residual connections, we use 1×1 convolutions to reduce IL. c The number of channels is used to obtain the output I. CO .

[0085] S35: Subtract the target font attribute and the content font attribute from the preprocessed S22 to obtain the difference attribute. The difference attribute is processed by the attribute attention module to generate the attribute feature difference vector and attribute channel attention feature maps of different scales.

[0086] Reference Figure 3 and Figure 5 The attribute attention module in S35 is used to achieve controllable adjustment of the attributes of the generated font image. Specifically, it embeds the attribute subscript index. As a more specific representation of each attribute feature, where N α N represents the number of attributes. e This indicates the dimension of the attribute embedding. For convenience, we will then perform channel attention on the attributes and their corresponding features within the same dimension. This indicates that a new dimension has been inserted into the attribute embedding 'e', ​​forming a new tensor shape. This can be represented as the transpose of the last two dimensions of e. This represents the multiplication of the matrix in the last two dimensions. The feature vector of each attribute index is mapped to a two-dimensional feature. The specific method is as follows:

[0087]

[0088] The value corresponding to each attribute can be regarded as the weight coefficient of attribute embedding. Then, the correlation between the attributes of the target image b and the source image a, and the association between the font image and the attribute, that is, the attribute feature difference, can be expressed as:

[0089]

[0090] Here, ⊙ represents element-wise multiplication of attribute differences and attribute embeddings. It is a tiling of α(b) and α(a), from which we can obtain a specific representation of the feature difference between the two attributes, β(a,b). This is beneficial for the next stage of channel attention operation, which is used to explore the correlation between different attributes and help our model focus on more important attributes.

[0091] γ(a,b)=(α t (b)-α t (a))0⊙e

[0092] in, This means that a new dimension has been inserted into the attribute difference feature and multiplied with the attribute embedding e by weights. In this way, we obtain global information about the feature difference between the two attributes. This is used in subsequent UNet models to represent the association between font / signature images and attributes using cross-attention.

[0093] α * (a,b)=F CA (β(a,b))

[0094] F CA This indicates that a channel attention operation is performed on β(a,b), generating a channel attention map by utilizing the inter-channel relationships of the features. The specific operation is shown in the figure above: the feature maps in β(a,b) are first aggregated through an average pooling layer, then sent to two convolutional layers, where the channel attention map is output through channel compression and stretching. Then, channel multiplication is performed between the attention map M(a,b) and γ(a,b) to obtain α. * (a,b). Simultaneously, through multiple convolution operations, the model is scaled to the scale required for each stage in the UNet network model.

[0095] S36: Input the difference vector and feature map generated in S35 into the denoising UNet structure to provide style attribute information for font denoising;

[0096] Reference Figure 3 Among them, the attribute feature difference vector With content vectors through cross-attention mechanism To achieve fusion, it is coupled layer by layer to the middle layer of the UNet network, where...

[0097]

[0098] and These are learnable projection matrices, which, after training, can output images with the corresponding target style while keeping the original content unchanged. Attribute channel attention feature maps at different scales are concatenated with UNet feature blocks of corresponding scales. The concatenated features are then enhanced through two layers of channel attention and a self-attention layer, performing cross-channel and cross-spatial feature interaction. Finally, an activation function outputs multi-scale fused features, achieving hierarchical transfer of attribute style information within the denoising UNet structure.

[0099] S37: The UNet network architecture based on content features and style attribute conditional injection can achieve noise prediction. By performing reverse denoising on the noisy image obtained in step S32 using the noise predicted by the UNet network, the target font image can be generated.

[0100] Reference Figure 1 For the inverse denoising process of the diffusion model, the UNet model can predict the noise vector. θ (z t ,t,e c ,γ(a,b),α * (a,b)), hence z t-1It can be obtained using the following formula:

[0101]

[0102] Where σ t It is a hyperparameter, and z′~(0,I) is random noise.

[0103] S38: To adapt the font style to the font content, a discriminator based on a CNN network is added during network training. The discriminator outputs two classification heads: one is used to distinguish between real and fake images (the discriminator uses the rasterized image of the given font as the real sample and the output of the denoising generator as the fake sample), and the other is used to predict font attributes.

[0104] Reference Figure 3 In step S3, when training the generation model for attribute-controllable fonts based on a diffusion model, a combination of multiple loss functions is used to achieve attribute-controllable font generation.

[0105] Generate adversarial loss L G Improve image realism using the cross-entropy function;

[0106] L G =-logp(y d =1|z'0)

[0107] Denoising loss L MSE The diffusion process is optimized by using the mean square error between the predicted and target values.

[0108]

[0109] Content loss L content Smooth the pixel differences between the content image vector and the generated image vector using L1 constraints;

[0110]

[0111] Style loss L style VGG network is used to extract feature differences to ensure style consistency;

[0112]

[0113] Attribute loss L attribute Calculate the difference between the generated image and the target image attributes while preserving the source image attributes;

[0114]

[0115] Each loss item is weighted according to a preset weight to form the generator loss.

[0116] L Generator =λ1L G +λ2LMSE +λ3L content +λ4L style +λ5L attribute

[0117] Where λ1, λ2...λ5 are the hyperparameters of the loss function.

[0118] The discriminator loss includes image authenticity cross-entropy loss and attribute prediction smoothing L1 loss, which together optimize the discrimination ability.

[0119] L Discrimintor =L D +L′ attribute

[0120] The image authenticity cross-entropy loss is:

[0121] L D =-logp(y d =1|z0)-logp(y d =0|z′0).

[0122] The attribute prediction smoothing L1 loss is:

[0123]

[0124] Implementation Cases

[0125] Using the trained model weights Figure 6 The original image and target attribute vector of the font content shown Figure 6 The image below shows a schematic diagram of the font generation effect after modifying a single target attribute value. During the generation operation, the attribute predictor within the model first predicts the attribute values ​​of the original image, subtracts them from the target attribute value, and inputs the result into the attribute attention module. This yields the attribute feature difference vector and attribute channel attention feature maps at different scales. Simultaneously, the original image is input into the content encoder to extract content information. The style attribute information and content information are then integrated into the UNet structure, ultimately generating the target font image based on a randomly sampled Gaussian noise image.

[0126] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be used to limit the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for generating attribute-controllable fonts based on a diffusion model, characterized in that, Includes the following steps: S1: Obtain the original font image and attribute data; S2: Based on the above font images and attribute data, preprocess the data to obtain a standardized dataset; S3: The dataset created based on S2 is used to train the generation model of attribute-controllable fonts based on the diffusion model; S4: Given a font content image and a target attribute vector, generate a high-quality font with a specified style using the generative model obtained in S3.

2. The method for generating attribute-controllable fonts based on a diffusion model according to claim 1, characterized in that, The specific method for obtaining the original font image and attribute data in S1 is as follows: S11: Read the original font image data and convert it into NumPy data format; S12: Read the font attribute data and convert it to NumPy data format through a format conversion operation.

3. The method for generating attribute-controllable fonts based on a diffusion model according to claim 1, characterized in that, The specific method for preprocessing the original font image and font attribute data in S2 is as follows: S21: For the NumPy format image data obtained in S11, perform binarization and size normalization operations in sequence to remove redundant image information and unify image size, and build a standardized image dataset. S22: Standardize the font attribute NumPy data after conversion in S12, linearly mapping the original attribute data in the range of 0-100 to the range of 0-1, thereby normalizing the attribute data to facilitate subsequent model training and feature extraction.

4. The method for generating attribute-controllable fonts based on a diffusion model according to claim 1, characterized in that, The generation model for attribute-controllable fonts based on a diffusion model used in S3 includes the following steps: S31: For the content image and target image preprocessed by S21, the latent diffusion model is used as the denoising generator. The images are encoded into the latent space by a pre-trained variational autoencoder to obtain the content latent vector and the target latent vector. S32: The target potential vector obtained in S31 is fused with the noise information containing noise intensity t to form the input feature, which is then input into the UNet denoising structure for training. S33: The content latent vector obtained in S31 is processed by the content encoder to extract global and local content features at different scales; S34: Input the content features extracted in S33 into the multi-scale content aggregation module of UNet to preserve the font content information during the denoising process; S35: Subtract the target font attribute and the content font attribute from the preprocessed S22 to obtain the difference attribute. The difference attribute is processed by the attribute attention module to generate the attribute feature difference vector and attribute channel attention feature maps of different scales. S36: Input the difference vector and feature map generated in S35 into the denoising UNet structure to provide style attribute information for font denoising; S37: The UNet network architecture based on content features and style attribute conditional injection can achieve noise prediction. By performing reverse denoising on the noisy image obtained in step S32 using the noise predicted by the UNet network, the target font image can be generated. S38: To adapt the font style to the font content, a discriminator based on a CNN network is added during network training. The discriminator outputs two classification heads: one is used to distinguish between real and fake images (the discriminator uses the rasterized image of the given font as the real sample and the output of the denoising generator as the fake sample), and the other is used to predict font attributes.

5. The method for generating attribute-controllable fonts based on a diffusion model according to claim 1, characterized in that, The pre-trained variational autoencoder used in S31 is trained on a large image dataset, and the parameters of its encoder and decoder remain frozen during training; initially, the encoder transforms the input image into a four-dimensional latent representation z∈R. 4×H / 8×W / 8 This latent representation serves as the network input for the diffusion and denoising processes in the diffusion model.

6. The method for generating attribute-controllable fonts based on a diffusion model according to claim 1, characterized in that, The image noise addition process in S32 can be described as a process involving random variables z0, z1, ..., z T The process of forming a Markov chain, in which noise is injected, can be described as follows: Where t ~ [0,T], ò ∈R 4×H / 8×W / 8 It is the Gaussian noise added at each step, α t =1-β t , β i ~(0,1) is a fixed parameter of variance.

7. The method for generating attribute-controllable fonts based on a diffusion model according to claim 1, characterized in that, The content encoder in S33 uses spectral normalized deep block cascaded downsampling, combined with a self-attention module at a specific level, to output multi-scale font content features through multi-level residual connections, thereby achieving multi-scale decoupling and semantic preservation of font content features.

8. The method for generating attribute-controllable fonts based on a diffusion model according to claim 1, characterized in that, The multi-scale content aggregation module of S34 concatenates content features at different scales with corresponding UNet block features to generate channel information features. It generates a global channel awareness vector through channel attention and weights it. After residual connection, it compresses the channels by 1×1 convolution to generate an intermediate fusion vector e. c This enables cross-scale content aggregation during the noise reduction process.

9. The method for generating attribute-controllable fonts based on a diffusion model according to claim 1, characterized in that, The attribute attention module in S35 is used to achieve controllable adjustment of attributes for generating skilled forged signatures. Specifically, it converts the attribute subscript index embedding into a two-dimensional feature map, generates an attribute feature difference vector γ(a,b) through element-wise operations of attribute differences and attribute embeddings, generates global information through dimensionality insertion and weight multiplication, generates an attention map through channel attention and multiplies it with the global information, and finally scales it to various scales of UNet through convolution to obtain the attribute channel attention feature map α. * (a,b) is used to achieve personalized adjustment of attributes such as stroke thickness and slant, and to focus on important attributes.

10. The method for generating attribute-controllable fonts based on a diffusion model according to claim 1, characterized in that, In S36, the attribute feature difference vector and the UNet feature block are fused through a cross-attention mechanism. The attribute channel attention feature maps of different scales are concatenated with the corresponding UNet feature blocks. The concatenated features are then enhanced through two layers of channel attention and self-attention layers for cross-channel and cross-space feature interaction. Finally, the multi-scale fused features are output through the activation function, realizing the hierarchical transfer of attribute style information in the denoising UNet structure.

11. The method for generating attribute-controllable fonts based on a diffusion model according to claim 1, characterized in that, In S37, during the reverse denoising or sampling process, the UNet model can predict the noise vector. θ (z t ,t,e c ,γ(a,b),α * (a,b)), hence z t-1 It can be obtained using the following formula: Where σ t It is a hyperparameter, and z′~(0,I) is random noise.

12. The method for generating attribute-controllable fonts based on a diffusion model according to claim 1, characterized in that, In S3, when training the attribute-controllable font generation model based on the diffusion model, multiple loss functions are combined to achieve attribute-controllable font generation: the basic denoising loss optimizes the diffusion process through the mean square error between the predicted and target values; the generative adversarial loss improves image realism with the cross-entropy function; the style loss extracts feature differences based on the VGG network to ensure style consistency; the content loss constrains pixel differences with smoothed L1; the attribute loss calculates the difference between the generated image and the target attribute while maintaining the attribute of the source image; each loss term is weighted according to preset weights to form the generator loss; the discriminator loss includes image realism cross-entropy loss and attribute prediction smoothed L1 loss, which together optimize the discrimination ability.

Images