Font generation method for solving font structure distortion based on frequency domain structure enhancement

Abstract

The invention discloses a structure enhancement type diffusion denoising network and a font generation method for solving font structure distortion. Low-frequency structure information of a content image and low-frequency components of a noise image are subjected to weighted fusion through a structure enhancement unit based on two-dimensional discrete wavelet transform; therefore, the reconstruction of the glyph basic structure is explicitly guided on the frequency domain level, and the structural distortion problems of stroke missing, fracture or dislocation and the like in the generated result are effectively relieved. According to the method, low-frequency structure information of a content image is directly mined in a frequency domain through discrete wavelet transform for guidance, so that effective maintenance of a character geometric structure is realized on the premise of not introducing any manual annotation prior. In each time step of a diffusion denoising process, low-frequency structure information of a content image and low-frequency components of a noise image are subjected to weighted fusion, and high-frequency details are reserved, so that reconstruction of a font basic structure is explicitly guided on a frequency domain level.

Description

Technical Field

[0001] This invention belongs to the field of image technology, specifically relating to a font generation method based on frequency domain structure enhancement to solve glyph structure distortion. Background Technology

[0002] Digital fonts are now widely used. Font designers save the glyphs they create by hand or with the help of design software as font files, and then install and use them directly on electronic devices.

[0003] Nearly a quarter of the existing ancient calligraphy treasures suffer from varying degrees of physical damage. Traditional methods of manual restoration and font replenishment have significant limitations; manually designing a complete Chinese character font library often takes months or even years. This high-cost, low-efficiency traditional model can no longer meet the widespread application of modern font design and the urgent need for the protection of calligraphy cultural heritage.

[0004] Existing methods lack effective supervision of character geometry during diffusion, making it difficult for the denoising network to accurately predict noise residuals consistent with the real character structure in each backward iteration. As a result, the generated characters still have a large degree of structural distortion, such as missing strokes, redundancy, misalignment, and breakage. Summary of the Invention

[0005] The purpose of this invention is to provide a structure-enhanced diffusion denoising network and a font generation method for solving glyph structure distortion, so as to solve the problem of structural distortion in existing font generation methods.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: A structure-enhanced diffusion denoising network includes a content image encoder, a style image encoder, a structure enhancement unit, and a U-Net backbone network; The content image encoder is used to extract content features from the content image; The style image encoder is used to extract style features from style images; The structure enhancement unit is used to enhance the noise image at each time step in the denoising process. Specifically, it fuses the low-frequency information of the content image and the low-frequency information of the noise image to obtain the low-frequency information of the structure enhancement; then it converts the low-frequency information of the structure enhancement and the high-frequency information of the noise image into the structure-enhanced noise image. The U-Net backbone network is used to receive content features, style features, and structurally enhanced noisy images, and output generated images that conform to the specified content and style.

[0007] The present invention also has the following features: Furthermore, the content image encoder adopts a multi-scale coding structure; the content image encoder includes three-level downsampled residual convolutional blocks, and the content features include shallow image features, mid-level image features, and deep image features; The style image encoder includes four consecutive downsampling residual units, each containing two 3×3 convolutional layers with spectral normalization, achieving 2x spatial downsampling through average pooling; The U-Net backbone network includes an encoder, an intermediate bottleneck layer, and a decoder; the encoder is composed of multiple downsampling blocks stacked together, including multi-scale content feature aggregation downsampling blocks and ordinary downsampling blocks; the decoder is composed of multiple upsampling blocks stacked together, including style aggregation upsampling blocks and ordinary upsampling blocks.

[0008] Furthermore, the structure enhancement unit outputs the enhanced noise image through the following steps: Step a1: Perform a two-dimensional discrete wavelet transform on the content image, decomposing it into a low-frequency approximate sub-band and three high-frequency detail sub-bands, as shown in the following equation:

[0009] in, This represents the input content image; This represents a two-dimensional discrete wavelet transform operation; Indicated by the scaling function The generated low-frequency approximate subband; Subscript Indicates the direction type, and its value set is: ,in Indicates the horizontal direction. Indicates the vertical direction. Indicates diagonal direction; Wavelet function representing the corresponding direction The generated high-frequency detail subbands, specifically: hour, Indicates the horizontal high-frequency sub-band; hour, Indicates the vertical high-frequency sub-band; hour, Indicates diagonal high-frequency sub-bands; symbol This represents the set union operation; Step a2: For each time step of the diffusion denoising process, acquire the current noise image to be processed, and perform a two-dimensional discrete wavelet transform on the noise image to decompose it into a low-frequency approximate sub-band and three high-frequency detail sub-bands, as shown in the following equation:

[0010] in, This represents the noisy image currently being processed. Step a3 involves weighted fusion of the low-frequency approximate subbands of the content image and the low-frequency approximate subbands of the noise image to obtain the fused low-frequency information with enhanced structure, as shown in the following formula:

[0011] in, Low-frequency information indicating structural enhancement; Content image Low-frequency information obtained through two-dimensional discrete wavelet transform; Representing a noisy image Low-frequency information obtained through two-dimensional discrete wavelet transform; Represents the learnable global fusion coefficients; w represents a tensor with the same shape as the low-frequency subband, such that all spatial locations and channels share the same fusion weight; Step a4: Using the low-frequency information of the structure enhancement and the high-frequency detail information of the noise image to be processed, the structure-enhanced noise image is reconstructed through inverse two-dimensional discrete wavelet transform, as shown in the following equation:

[0012] in, A noisy image representing structural enhancement; This represents the inverse discrete wavelet transform.

[0013] Furthermore, the U-Net backbone network uses the following steps to output generated images that conform to the specified content and style: Step b1: In the U-Net backbone network, the multi-scale content feature aggregation downsampling block first concatenates the output of the previous downsampling block with the content features of the corresponding level along the channel dimension. The spliced ​​features are weighted to form a two-level fused feature of intermediate layer content with residual connections; Secondly, the number of feature channels is compressed to the target dimension through optional convolution operations; Finally, the style features extracted by the style image encoder are injected into the two-level fusion features using the cross-attention mechanism, resulting in the three-level fusion features, as shown below:

[0014] in, This represents the input of the current downsampling block; This indicates the output characteristics of the current downsampling block; This indicates splicing along the channel dimension; This represents a lightweight feature map that includes normalization, non-linear activation, and 1×1 convolution; This indicates the channel attention module; This means compressing the spliced ​​high-dimensional features to the target number of channels; This indicates that the global style vector Flatten out into a sequence of context tokens; Indicated by For query, , This is a cross-attention mechanism for key-value pairs; The ordinary downsampling block receives the intermediate layer features output from the previous downsampling block and performs feature extraction. Optionally, it performs convolutional downsampling to reduce spatial resolution, as shown in the following equation:

[0015] in: This represents the input of the current downsampling block; This indicates the output characteristics of the current downsampling block; This represents a standard residual convolutional block, which includes normalization, non-linear activation, and convolution operations. This indicates an optional convolutional downsampling operation; Step b2: The intermediate bottleneck layer performs a refined fusion of the deepest features, content features, and style features; Step b3: The upsampled block of each style aggregation fuses the output features from the previous decoder, the skip connection features from the corresponding level of the encoder, and the global style features to obtain new U-Net intermediate layer features, as shown below:

[0016] In the formula: This indicates that the current upsampled block comes from the input features of the previous level decoder; This represents the skip connection features from the corresponding level of the encoder; This indicates that the global style vector Flatten out into a sequence of context tokens; The standard upsampling block concatenates the output features from the previous decoder with the skip connection features from the corresponding level of the encoder, and performs feature fusion through a standard residual convolutional unit. Finally, it optionally performs a convolutional upsampling operation to improve spatial resolution. The top-level output features of the decoder are mapped to noise prediction values ​​with the same number of channels as the input image through a convolutional layer. This is used for subsequent iterations of noise reduction; Step b4: Input the current noisy image into the structure enhancement unit to generate a structure-enhanced noisy image; Input the U-Net backbone network and combine it with the embedding t at the current time step and multi-scale content features. With global style feature vector Output the predicted value of the current noise. Finally, the image is updated according to the backsampling formula of the diffusion model to obtain the noisy image at the next time step. As shown in the following formula:

[0017] In the formula: Represents the noise image at step t; This represents a noisy image after structural enhancement. This represents the image obtained at step t-1 after one step of denoising; , ,in These are preset variance scheduling hyperparameters; This represents the noise residual predicted by the U-Net backbone network; Represents the sampling noise figure. ; For standard Gaussian noise, when When used to introduce randomness, when At that time, set ; After multiple iterations, a generated image that conforms to the specified content and style is obtained.

[0018] A font generation method based on frequency domain structure enhancement to solve glyph structure distortion, the method using the aforementioned structure-enhanced diffusion denoising network, includes the following steps: Step 1: Construct a character image dataset; Step 2: Construct the structure-enhanced diffusion denoising network as described in claim 4; Step 3: Use the character image dataset as the training set to train the structure-enhanced diffusion denoising network. Use the trained structure-enhanced diffusion denoising network as the font generation model to achieve single-sample font generation that solves the problem of character structure distortion.

[0019] Furthermore, step 1 includes the following sub-steps: Step 11: Select one font from the public database as the content font, and select several other fonts as style fonts; Step 12: Convert the multiple characters corresponding to the content font into images, which will be used as content images; Convert multiple characters corresponding to the style font into images, which are then used as style images; Convert multiple characters of the corresponding content font of the style font into images, which are then used as real images; All content images, style images, and real images are compiled into a glyph image dataset.

[0020] Compared with the prior art, the present invention has the following technical effects: (I) The structure-enhanced diffusion denoising network and font generation method for solving glyph structure distortion of the present invention, in each time step of the diffusion model denoising process, uses a structure enhancement unit based on two-dimensional discrete wavelet transform to weightedly fuse the low-frequency structural information of the content image with the low-frequency components of the noise image, while preserving its high-frequency details. This explicitly guides the reconstruction of the glyph basic structure at the frequency domain level, effectively alleviating structural distortion problems such as missing strokes, breaks, or misalignments in the generated results. Unlike traditional structure-guided font generation methods that rely on additional supervisory signals such as stroke segmentation maps, component category labels, or skeleton extraction, this method directly mines the low-frequency structural information of the content image itself in the frequency domain through discrete wavelet transform for guidance. Thus, without introducing any manual annotation priors, it effectively preserves the geometric structure of characters, significantly improving the generalization ability and deployment convenience of the method on any new font.

[0021] (II) When enhancing the character structure, this invention does not rely on additional supervisory signals such as stroke segmentation maps, component category labels or skeleton extraction as traditional structure-guided font generation methods. Instead, it directly mines the low-frequency structural information of the content image itself in the frequency domain through discrete wavelet transform for guidance. Thus, without introducing any artificial annotation priors, it achieves effective preservation of the character geometric structure, significantly improving the generalization ability and deployment convenience of the method on any new font (including multilingual characters).

[0022] (III) This invention is applicable to single-sample font generation scenarios. It can achieve high-quality style transfer with only one example image of the target style, while maintaining the consistency of character semantic structure. The method has been verified to be effective on multiple font styles and has good generalization ability and engineering application value. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the structure of the structure-enhanced diffusion denoising network of the method of the present invention.

[0024] Figure 2 This diagram illustrates a comparison of the generation results of the method of this invention with those of other algorithms on a test set of visible fonts but invisible characters.

[0025] Figure 3 This diagram illustrates a comparison of the generation results of the method of the present invention on a test set of visible characters in invisible fonts with those of other algorithms.

[0026] Figure 4 This diagram illustrates a comparison of the generation results of the method of this invention with those of other algorithms on a test set of invisible fonts and invisible characters.

[0027] Figure 5 This diagram illustrates a comparison of the generation results of the method of this invention with those of other algorithms in a zero-sample cross-language generation task. Detailed Implementation

[0028] The following are specific embodiments of the present invention. It should be noted that the present invention is not limited to the following specific embodiments. All equivalent modifications made based on the technical solutions of this application fall within the protection scope of the present invention.

[0029] This embodiment presents a font generation method to solve the problem of glyph distortion, based on the application of a structure enhancement unit in a font generation diffusion model using two-dimensional discrete wavelet transform; a specific implementation method in which wavelet decomposition is performed on the content image and the noise image respectively during the denoising process, and their low-frequency components are fused to guide structure reconstruction; and a single-sample font generation architecture that preserves glyph structure without the need for external structure annotation and relies only on the frequency domain information of the original content image.

[0030] Specifically, a structure-enhanced diffusion denoising network is characterized in that the structure-enhanced diffusion denoising network includes a content image encoder, a style image encoder, a structure enhancement unit, and a U-Net backbone network; Content image encoders are used to extract content features from content images; A style image encoder is used to extract style features from a style image; The structure enhancement unit is used to convert the noisy image to be processed into a structure-enhanced noisy image; The U-Net backbone network receives noisy images with content features, style features, and structural enhancements, and outputs generated images that conform to the specified content and style.

[0031] Specifically, the content image encoder adopts a multi-scale coding structure, including a residual convolutional block with progressive downsampling; the residual convolutional block contains three downsampling stages; the three downsampling stages output shallow image features, mid-level image features and deep image features, respectively, as the content features of the content image; In this embodiment, the content image encoder uses a single content image from the glyph image dataset. As input, the image resolution is 96×96 pixels, and B represents batch size, which is 16 here.

[0032] The encoder employs a multi-scale coding structure, consisting of a series of progressively downsampled residual convolutional blocks stacked together. It comprises three downsampling stages, each using a downsampled residual unit to achieve feature extraction and spatial compression. Specifically, in the first downsampling stage, after processing the input image through convolution and residual blocks, the output is a feature map with a resolution of 48×48 and 64 channels, denoted as the shallow image feature map. Its high spatial resolution effectively preserves the local stroke details of characters; in the second downsampling stage, the feature map is further downsampled to 24×24 resolution, and the number of channels increases to 128, denoted as the mid-level image feature. This is used to characterize the relative positional relationships between components within a character; in the third downsampling stage, the feature map is compressed to a 12×12 resolution, and the number of channels is expanded to 256, denoted as deep image features. The overall layout and global semantic structure of the main encoded characters. These three layers of features together constitute a set of multi-scale content features. These correspond to structural information at different spatial granularities and will be fed into the corresponding layers of the U-Net backbone network for cross-scale feature fusion and structure-guided glyph reconstruction.

[0033] In this embodiment, the style image encoder is composed of four consecutive downsampled residual units stacked together, each of which contains two 3×3 convolutional layers with spectral normalization. The style image encoder uses a style image from the glyph image dataset. As input, the image resolution is 96×96 pixels, the number of channels is 3, and B represents the batch size, which is 16 here. The encoder adopts a progressively downsampled residual convolutional structure, which is composed of four consecutive downsampled residual units stacked together. Each unit contains two 3×3 convolutional layers with spectral normalization, and 2x spatial downsampling is achieved through average pooling. Specifically, the input image goes through the following processing stages in sequence: (1) Output resolution is 48×48, number of channels is 64; (2) Output resolution is 24×24, number of channels is 128; (3) Output resolution is 12×12, number of channels is 256; (4) Output resolution is 6×6, number of channels is 512; (5) The feature map is processed by a post-processing module consisting of a 1×1 convolutional layer and InstanceNorm2d and ReLU activation functions to obtain the final deep feature map (size is 6×6×1024). To generate a single style representation, global adaptive average pooling is performed on the feature map, compressing it into a 1×1×1024 tensor and flattening it into a vector of length 1024. This vector is the style feature vector, which is subsequently input into the U-Net backbone network as a style guiding signal in the glyph generation process.

[0034] The U-Net backbone network adopts an encoder-decoder architecture, including an encoder, an intermediate bottleneck layer, and a decoder. The encoder is composed of multiple downsampling blocks stacked together, including multi-scale content feature aggregation downsampling blocks and ordinary downsampling blocks. The decoder is composed of multiple upsampling blocks stacked together, including style aggregation upsampling blocks and ordinary upsampling blocks.

[0035] In this embodiment, the U-Net backbone network serves as the core generative model for inverse denoising using the diffusion model. It receives the structurally enhanced noisy image, time step, multi-scale content features, and style feature vectors output by the structural enhancement unit as input. The time step, multi-scale content features, and style feature vectors act as conditional information to guide the structurally enhanced noisy image denoising process, gradually restoring the glyph image of the target style. This network employs an encoder-decoder architecture, consisting of an encoder, an intermediate bottleneck layer, and a decoder.

[0036] The encoder consists of a series of downsampling blocks stacked together, including multi-scale content feature aggregation downsampling blocks and ordinary downsampling blocks.

[0037] The input to the multi-scale content feature aggregation downsampling block is the intermediate layer feature from the previous U-Net layer. First, it is concatenated with the same-scale content features from the content encoder. The concatenated feature then undergoes group normalization (32 groups), a Swish activation function, and a 1×1 convolution to compress the number of channels back to the original number of channels in the U-Net intermediate layer feature. Further weighting is performed using a channel attention mechanism, which extracts channel statistics using global average pooling, generates channel weights through two fully connected layers and a Sigmoid activation, and finally multiplies them with the original feature and adds residual connections to form an intermediate representation for content enhancement. This feature is then fed into the temporal conditional residual. The convolutional unit includes group normalization, Swish activation, convolution operations, and fuses temporal embedding vectors to output intermediate-content two-level fused features. Next, the global style features are flattened along their spatial dimensions and converted into a style token sequence, with a sequence length equal to the total number of spatial locations of the style features. Using the intermediate-content two-level fused features as queries and the style token sequence as keys and values, style semantics are injected into the feature map through a cross-attention mechanism, resulting in intermediate-content-style three-level fused features. If the current block is configured with downsampling, a 3×3 convolution (stride 2, padding 1) halves the spatial resolution. In contrast, the ordinary downsampling block does not introduce any external guiding signals, only containing the aforementioned temporal conditional residual convolutional unit and an optional 3×3 downsampling convolution. The outputs of both types of downsampling blocks are stored as skip connection signals for use by the decoder. The encoder in this example consists of: an ordinary downsampling block, a multi-scale content aggregation downsampling block, a multi-scale content aggregation downsampling block, and an ordinary downsampling block.

[0038] The structure enhancement unit is used to enhance the noise image at each time step in the denoising process. It fuses the low-frequency information of the content image and the low-frequency information of the noise image to obtain the structure-enhanced low-frequency information. Then, it converts the structure-enhanced low-frequency information and the high-frequency information of the original noise image into a structure-enhanced noise image. The U-Net backbone network is used to receive the structure-enhanced noisy image, the current time step embedding, content features, and style features, and based on this, predict the noise added at the current time step, and gradually generate the target domain glyph image through iterative denoising. Specifically, the structure enhancement unit converts the noise image to be processed into a structure-enhanced noise image through the following steps: Content Image A two-dimensional discrete wavelet transform is performed, and the wavelet is decomposed into a low-frequency approximate subband by a one-layer wavelet decomposition (decomposition scale J=1) based on the Daubechies-4 (db4) wavelet basis and symmetric boundary extension mode. And three high-frequency detail sub-bands, corresponding to the horizontal direction respectively. Vertical direction and diagonal direction Since first-order wavelet decomposition compresses each spatial dimension by a factor of 2, the resolution of each sub-band is 1. The number of channels remains at 3. This embodiment only retains the low-frequency subband. As a structural guidance signal, it is used for subsequent fusion; Step a1: Perform a two-dimensional discrete wavelet transform on the content image, decomposing it into one low-frequency approximate sub-band and three high-frequency detail sub-bands, as shown in the following equation:

[0039] in, This represents the input content image; This represents a two-dimensional discrete wavelet transform operation; Indicated by the scaling function The generated low-frequency approximate subband; Subscript Indicates the direction type, and its value set is: ,in Indicates the horizontal direction. Indicates the vertical direction. Indicates diagonal direction; Wavelet function representing the corresponding direction The generated high-frequency detail subbands, specifically: hour, Indicates the horizontal high-frequency sub-band; hour, Indicates the vertical high-frequency sub-band; hour, Indicates diagonal high-frequency sub-bands; symbol This represents the set union operation; At each time step t in the diffusion denoising process, the current noise image to be processed is acquired. The noisy image is also subjected to a single two-dimensional discrete wavelet transform, using the same method as the decomposed content image. Wavelet basis and symmetric boundary mode. It is decomposed into a low-frequency approximate subband. And three high-frequency detail subbands: , , The low-frequency subband This will be used for subsequent weighted fusion with the low frequencies of the content image, while the three high-frequency sub-bands will retain the local texture and detail features of the original noisy image for use in the inverse wavelet transform stage. Step a2: At each time step of the diffusion denoising process, acquire the current noise image to be processed, and perform a two-dimensional discrete wavelet transform on the noise image; decompose the noise image into a low-frequency approximate sub-band and three high-frequency detail sub-bands, as shown in the following equation:

[0040] in, Represents a content image; After obtaining the low-frequency subbands of the content image and the noise image, this embodiment performs a weighted fusion of the two to generate a structure-enhanced low-frequency representation. A globally learnable scalar weight parameter is introduced. Its initial value was set to 0.5, although While it is a single scalar, in actual tensor operations, it is automatically broadcast to have the exact same shape as the low-frequency subband, ensuring that all channels and all spatial locations share the same fusion weights. These weights are then automatically optimized as model parameters during training via backpropagation, resulting in the fused structure-enhanced low-frequency information. ; Step a3 involves weighted fusion of the low-frequency subband of the content image and the low-frequency approximate subband of the noisy image to obtain the fused structurally enhanced low-frequency information, as shown in the following formula:

[0041] in, Low-frequency information indicating structural enhancement; Content image Low-frequency information obtained through two-dimensional discrete wavelet transform; Representing a noisy image Low-frequency information obtained through two-dimensional discrete wavelet transform; Represents the learnable global fusion coefficients; w represents a tensor with the same shape as the low-frequency subband, such that all spatial locations and channels share the same fusion weight; After weighted fusion of the low-frequency subbands, this embodiment utilizes the structure-enhanced low-frequency information and the high-frequency detail information of the original noise image to reconstruct the structure-enhanced noise image through inverse two-dimensional discrete wavelet transform. Step a4: Utilizing the structure-enhanced low-frequency information and the high-frequency detail information of the noise image, the structure-enhanced noise image is reconstructed through inverse two-dimensional discrete wavelet transform, as shown in the following equation:

[0042] in, A noisy image representing structural enhancement; This represents the inverse discrete wavelet transform.

[0043] Furthermore, the U-Net backbone network, as the core generative model for inverse denoising using the diffusion model, receives the structurally enhanced noisy image, time step, multi-scale content features, and style feature vectors output by the structural enhancement unit as input. The time step, multi-scale content features, and style feature vectors serve as conditional information to guide the structurally enhanced noisy image denoising process, progressively recovering the glyph image of the target style. The U-Net backbone network consists of an encoder, an intermediate bottleneck layer, and a decoder. The encoder is composed of a series of stacked downsampling blocks, including multi-scale content feature aggregation downsampling blocks and ordinary downsampling blocks. The decoder is composed of a series of stacked upsampling blocks, including style aggregation upsampling blocks and ordinary upsampling blocks.

[0044] The U-Net backbone network uses the following steps to output generated images that conform to the specified content and style: The input to the multi-scale content feature aggregation downsampling block is the intermediate layer feature from the previous U-Net layer. First, it is concatenated with the same-scale content features from the content encoder. The concatenated feature then undergoes group normalization (32 groups), a Swish activation function, and a 1×1 convolution to compress the number of channels back to the original number of channels in the U-Net intermediate layer feature. Further weighting is performed using a channel attention mechanism, which extracts channel statistics using global average pooling, generates channel weights through two fully connected layers and a Sigmoid activation, and finally multiplies them with the original feature and adds residual connections to form an intermediate representation for content enhancement. This feature is then fed into a temporal conditional residual function. The differential convolutional unit includes group normalization, Swish activation, convolution operations, and fuses temporal embedding vectors to output intermediate-content two-level fused features. Next, the global style features are flattened along their spatial dimensions and converted into a style token sequence, with a sequence length equal to the total number of spatial locations of the style features. Using the intermediate-content two-level fused features as queries and the style token sequence as keys and values, style semantics are injected into the feature map through a cross-attention mechanism, resulting in intermediate-content-style three-level fused features. If the current block is configured for downsampling, a 3×3 convolution (stride 2, padding 1) halves the spatial resolution. In contrast, the ordinary downsampling block does not introduce any external guiding signals and only contains the aforementioned temporal conditional residual convolutional unit and an optional 3×3 downsampling convolution. The outputs of both types of downsampling blocks are stored as skip connection signals for use by the decoder. The encoder in this example consists of: an ordinary downsampling block, a multi-scale content aggregation downsampling block, a multi-scale content aggregation downsampling block, and an ordinary downsampling block.

[0045] Specifically, in step b1, multi-scale content feature aggregation downsampling blocks are used to fuse content features. The output of the previous downsampling block is concatenated with the content features of the corresponding level in the channel dimension. The concatenated features are weighted to form a two-level fused feature of intermediate layer-content with residual connection. Secondly, the number of feature channels is compressed to the target dimension through optional convolution operations; Finally, the style feature vector is injected into the two-level fusion feature using the cross-attention mechanism, thus obtaining the intermediate layer-content-style three-level fusion feature, as shown in the following formula:

[0046] in, This represents the input of the current downsampling block; This indicates the output characteristics of the current downsampling block; This indicates splicing along the channel dimension; This represents a lightweight feature map that includes normalization, non-linear activation, and 1×1 convolution; This indicates the channel attention module; This means compressing the spliced ​​high-dimensional features to the target number of channels; This indicates that the global style vector Flatten out into a sequence of context tokens; Indicated by For query, , This is a cross-attention mechanism for key-value pairs; A regular downsampling block receives the intermediate layer features output from the previous downsampling block, performs feature extraction, and executes convolutional downsampling operations, as shown in the following equation:

[0047] in: This represents the input of the current downsampling block; This indicates the output characteristics of the current downsampling block; This represents a standard residual convolutional block, which includes normalization, non-linear activation, and convolution operations. This indicates an optional convolutional downsampling operation; Step b2 involves the intermediate bottleneck layer refining the fusion of the deepest features, content features, and style features. In this step, the intermediate bottleneck layer receives the deepest U-Net features output from the encoder. First, it processes these features through a temporal conditional residual convolutional block, which includes group normalization, Swish activation, and convolution operations, and fuses the temporal embedding vector. Then, the current feature is concatenated with the content features output by the content encoder at the corresponding deepest scale along the channel dimension. The content features have 256 channels, and their spatial dimensions are consistent with the current feature. The concatenated features are then processed sequentially through group normalization, Swish activation, 1×1 convolution, and channel attention mechanisms to generate intermediate-content two-level fused features. These features are then input into another temporal conditional residual convolutional block for transformation. Finally, the global style features are flattened along the spatial dimension into a style token sequence, and injected into the current feature through a cross-attention mechanism.

[0048] In step b3, the style aggregation upsampling block first concatenates the output features from the previous level decoder with the skip connection features from the corresponding level of the encoder along the channel dimension. The concatenated features are then fed into a standard temporal conditional residual convolutional unit, which includes group normalization (32 groups), a Swish activation function, convolution operations, and fused temporal embedding vectors. Subsequently, the global style features are flattened along the spatial dimension into a style token sequence, and style semantics are injected into the feature map using the current feature map as the query and the style token sequence as the key and value, through a single-head cross-attention mechanism, resulting in an enhanced representation of fused style information. Finally, optionally, a convolutional upsampling layer (using a learnable convolutional kernel with a stride of 2) is used to double the spatial resolution. In contrast, the ordinary upsampling block does not introduce global style features or include a cross-attention mechanism; it only concatenates the decoded features and skip connection features along the channel dimension and fuses them through a temporal conditional residual convolutional unit with the same structure, and then optionally performs a convolutional upsampling operation. The outputs of both types of upsampling blocks serve as the input to the next level decoder. The decoder in this example consists of the following components in sequence: a normal upsampling block, a style aggregation upsampling block, a style aggregation upsampling block, and a normal upsampling block.

[0049] Each style aggregation upsampling block jointly fuses the output features from the previous level decoder, the skip connection features from the corresponding level of the encoder, and the global style features to obtain new U-Net intermediate layer features, as shown in the following equation:

[0050] In the formula: This indicates that the current upsampled block comes from the input features of the previous level decoder; This represents the skip connection features from the corresponding level of the encoder; This represents a standard residual convolutional block, which includes normalization, nonlinear activation, and convolution operations, and has identity residual connections. This indicates that the global style vector Flatten out into a sequence of context tokens; The standard upsampling block concatenates the output features from the previous decoder with the skip connection features from the corresponding level of the encoder, and performs feature fusion through a standard residual convolutional unit. Finally, it optionally performs a convolutional upsampling operation to improve spatial resolution. The top-level output features of the decoder are mapped by the convolutional layer to the noise prediction value of the input image channel number. This is used for subsequent iterations of noise reduction; Step b4: In this step, the diffusion model reverse denoising process is used to progressively recover the target domain glyph image from the noisy image based on the aforementioned modules. Specifically, given an initial Gaussian noise image... Starting from time step t=T and decreasing to t=1, perform the following iterative operations: First, process the current noisy image... The input is fed into the structure enhancement unit to generate the structure-enhanced image. ; then, The U-Net backbone network is input along with the current time-step embedding *t*, multi-scale content features, and a global style feature vector. The decoder outputs a predicted value for the current noise level. This prediction employs a classifier-free guidance strategy with a guidance scale of 7.5, meaning the network simultaneously computes conditional and unconditional noise estimates and performs a weighted fusion. Finally, the image is updated using the DPM-Solver++ backsampling algorithm to obtain the noisy image for the next time step. The algorithm introduces random perturbations when t>1, and sets the noise term to zero when t=1 to output a deterministic result. In this example, the total number of denoising steps T is set to 20, and the input image resolution is 96×96. After 20 iterations, the final output is a clear, structurally complete generated glyph image that conforms to the specified content and style.

[0051] Current noisy image The input is fed into the structure enhancement unit to generate a structure-enhanced noisy image. ;Will Input the U-Net backbone network and combine it with the embedding t at the current time step and multi-scale content features. With global style feature vector Output the predicted value of the current noise. Finally, the image is updated according to the backsampling formula of the diffusion model to obtain the noisy image at the next time step. As shown in the following formula:

[0052] In the formula: Represents the noise image at step t; This represents the image obtained at step t-1 after one step of denoising; , ,in These are preset variance scheduling hyperparameters; This represents the noise residual predicted by the U-Net backbone network; Represents the sampling noise figure. ; is standard Gaussian noise, which is used to introduce randomness when and is set to when ; After T iterations, a clear, structurally complete generated image that conforms to the specified content and style is obtained.

[0053] A font generation method for solving glyph structure distortion based on a structure-enhanced diffusion denoising network, comprising the following steps: Step 1, construct a glyph image dataset; Step 2, construct the structure-enhanced diffusion denoising network of claim 2; Step 3, use the glyph image dataset as a training set to train the structure-enhanced diffusion denoising network, and use the trained structure-enhanced diffusion denoising network as a font generation model to implement single-sample font generation for solving glyph structure distortion.

[0054] Specifically, step 1 includes the following sub-steps: Step 11, select a font as the content font from a public database, and select multiple other fonts as style fonts; In this embodiment, specifically, TTF files are collected. The Song font file is downloaded from Founder Typeface Library (https: / / www.foundertype.com / ) as the content font, and 400 font files of other styles are used as the style fonts of the training set, and 13 font files of other styles are used as the style fonts of part of the test set, all of which are of the TTF type.

[0055] Step 12, convert multiple characters corresponding to the content font into images as content images; Convert multiple characters corresponding to the style font into images as style images; Convert multiple characters of the content font corresponding to the style font into images as real images; Use all content images, style images, and real images as the glyph image dataset.

[0056] In this step, in this embodiment, a TTF-to-glyph image script is written, the target font file is read and initialized as a corresponding font object, 799 training characters and 300 test characters are written into two txt files, the size of the converted glyph image is set to 96*96 pixels, and the glyphs are rendered in a centered manner on a blank canvas, and the rendered glyph images are named according to a unified rule. For example, for a font with the style name "FZBGDT", each glyph image is named in the format of "style name + content character.jpg". For example, "FZBGDT+张.jpg" represents the image of the character "张" in the FZBGDT style.

[0057] The final training set contains 799 images across 400 styles. For each style, one glyph image is randomly selected as the style image, along with the corresponding ground truth images of the 799 characters from each of the 400 styles. The test set consists of three parts: first, visible fonts with invisible characters, including style images of 100 styles randomly selected from the training set, content images of 300 test characters, and the corresponding ground truth images; second, invisible fonts with visible characters, including style images of 13 styles not seen during training, content images of the 799 characters used for training, and the corresponding ground truth images; and third, invisible fonts with invisible characters, including style images of 13 styles not seen during training, content images of 300 test characters, and the corresponding ground truth images. The content and style images in the test set serve as input to the Structure-Enhanced Diffusion Denoising Network (SEDN) in step 2. The ground truth images are used for subjective and objective comparison with the generated results of the SEDN to verify the effectiveness of the proposed method.

[0058] Each content image, one style image for each style, and a real image containing the content glyph in the glyph image dataset are used as the training set to train the structure-enhanced diffusion denoising network. The trained structure-enhanced diffusion denoising network is used as the font generation model to achieve single-sample font generation that solves the problem of glyph structure distortion.

[0059] In this embodiment, step 3 specifically includes the following steps: Step 31: From the constructed training set, randomly select a batch of samples in each iteration. Each sample consists of three parts: an image with Song typeface content. (One of 799 characters in total) A single-style image of a target style. (Choose any one of 400 training styles, with only one example glyph image required for each style), and the actual target domain glyph image of the corresponding character under that style. The three constitute the training triplet. ,in and As a conditional input As a reconstruction supervision target, the training set contains a total of One valid sample.

[0060] Step 32, for each target domain glyph image First, randomly sample a time step. ,in The total number of diffusion steps during training is used to independently sample noise from a standard Gaussian distribution. Subsequently, the cumulative signal-to-noise ratio is calculated based on the preset scaled linear noise scheduler. Generate noisy images ; Step 33: Employ a classifier-free guidance training strategy. Specifically, conditional information is randomly "dropped" with a preset probability of drop_prob=0.1: for each sample, the content image is independently dropped with a 10% probability. or style image Replace with a blank image with a grayscale value of 1.

[0061] Step 34, extract the noisy image. Input the structural reinforcement unit to obtain the reinforced input. Then embed it along with the time step. Original content image and style images They are fed into the U-Net backbone network together, and the noise prediction value is output. .

[0062] Step 35: The noise prediction value output by the structure-enhanced diffusion denoising network is... Compared with real Gaussian noise Calculate the mean square error (MSE) as the diffusion loss. As shown in the following formula:

[0063] Simultaneously, the backsampling formula of the diffusion model is used to predict noise. Noise images with structure enhancement Reconstructed image and compared it with the real target glyph image. Input the pre-trained VGG19 network and calculate the sum of the L1 distances between the two networks in the feature space of the first L=3 layers as the perceptual loss. As shown in the following formula:

[0064] In the formula: This indicates the VGG19 network's [number missing]th ... Feature map output of the layer; L=3 corresponds to selecting the three levels of relu2_2, relu3_3, and relu4_3 in VGG19 for feature alignment; Ultimately, the total loss function consists only of the diffusion loss and the perception loss, and the weighted sum of these two items is defined as follows:

[0065] In the formula: This represents the weighting coefficient of the perceived loss; The training parameters for this example are configured as follows, and the model uses the AdamW optimizer. , The learning rate is set to Use a linear learning rate for warm-up. The batch size was 16, and a total of 780,000 training steps were performed. The subjective evaluation used in this embodiment is a comparison between the results generated by the present invention and real images and other methods; The objective evaluation criteria used in this embodiment are as follows: RMSE: For each pair of generated images With real images Calculate the root mean square error using NumPy: SSIM: Calculated using the skimage.metrics.structural_similarity function from the scikit-image library, with each pair of generated images as input. With real images ; MS-SSIM: Calculated using the ms_ssim function in the PyTorch library pytorch_msssim, with each pair of generated images as input. With real images ; LPIPS: will generate images With real images Linearly map to [−1,1]; then call the pre-trained model based on AlexNet from the official LPIPS library to calculate the perceptual distance between them using lpips.LPIPS; Performance testing: First, the task of generating Chinese characters: To evaluate the font generation method for solving glyph distortion proposed in this invention, experiments were conducted in three typical generalization scenarios described in step 1: (1) Seen Font Unseen Character (SFUC); (2) Unseen Font Seen Character (UFSC); and (3) Unseen Font Unseen Character (UFUC). In each test setting, the method of this invention was compared with six representative baseline methods, and quantitative evaluation was performed using unified evaluation metrics (including RMSE, SSIM, MS-SSIM, and LPIPS). The experimental results are shown in Tables 1, 2, and 3. The experiments show that the method of this invention significantly outperforms existing baselines in all three test scenarios. Regardless of whether the target style appears in training or whether the target character has been seen by the model, this method can generate glyph images with complete structure, consistent style, and clear details. Especially in the most challenging UFUC scenario (where neither the font nor the character has been seen), this method still maintains the best performance. Figure 2 , Figure 3 , Figure 4 In the subjective demonstration, the character distortions that appeared have been marked with boxes. The subjective results have also fully verified the effectiveness of the method of the present invention in alleviating structural distortion problems such as missing strokes, misalignment, and breakage. The following are the evaluation indicators of this embodiment and other algorithms on different test sets, as shown in Tables 1-3.

[0066] Table 1. Evaluation metrics of this embodiment and other algorithms on the SFUC test set.

[0067] Table 2. Evaluation metrics of this embodiment and other algorithms on the UFSC test set.

[0068] Table 3 Evaluation metrics of this embodiment and other algorithms on the UFUC test set

[0069] Second, zero-shot cross-language generation task: To verify the generalization ability of the method in this embodiment in cross-language scenarios, a zero-shot cross-language generation experiment was further conducted. Specifically, without any fine-tuning or retraining, the model trained only on Chinese data was directly applied to character generation tasks in other languages ​​(such as Korean and Latin). The experiment covered two types of scenarios: (1) language transfer within the same writing system (e.g., Chinese → Korean); (2) language transfer across writing systems (e.g., Chinese → Latin). Figure 5 As shown, existing methods generally suffer from problems such as stroke breakage, structural collapse, or style inconsistency under zero-sample cross-language settings. However, the method in this embodiment can still stably generate target language glyphs with complete structure, consistent style, and high visual quality. This result shows that the method in this embodiment can effectively support zero-sample font generation across languages, demonstrating strong adaptability and robustness to different writing systems.

Claims

1. A structure-enhanced diffusion denoising network, characterized in that, It includes a content image encoder, a style image encoder, a structure enhancement unit, and a U-Net backbone network; The content image encoder is used to extract content features from the content image; The style image encoder is used to extract style features from style images; The structure enhancement unit is used to enhance the noise image at each time step in the denoising process. Specifically, it fuses the low-frequency information of the content image and the low-frequency information of the noise image to obtain the low-frequency information of the structure enhancement; then it converts the low-frequency information of the structure enhancement and the high-frequency information of the noise image into the structure-enhanced noise image. The U-Net backbone network is used to receive content features, style features, and structurally enhanced noisy images, and output generated images that conform to the specified content and style.

2. The structure-enhanced diffusion denoising network as described in claim 1, characterized in that, The content image encoder adopts a multi-scale coding structure; the content image encoder includes three levels of downsampling residual convolutional blocks, and the content features include shallow image features, mid-level image features, and deep image features; The style image encoder includes four consecutive downsampling residual units, each containing two 3×3 convolutional layers with spectral normalization, achieving 2x spatial downsampling through average pooling; The U-Net backbone network includes an encoder, an intermediate bottleneck layer, and a decoder; the encoder is composed of multiple downsampling blocks stacked together, including multi-scale content feature aggregation downsampling blocks and ordinary downsampling blocks; the decoder is composed of multiple upsampling blocks stacked together, including style aggregation upsampling blocks and ordinary upsampling blocks.

3. The structure-enhanced diffusion denoising network as described in claim 2, characterized in that, The structure enhancement unit outputs a structure-enhanced noise image through the following steps: Step a1: Perform a two-dimensional discrete wavelet transform on the content image, decomposing it into a low-frequency approximate sub-band and three high-frequency detail sub-bands, as shown in the following equation: in, This represents the input content image; This represents a two-dimensional discrete wavelet transform operation; Indicated by the scaling function The generated low-frequency approximate subband; Subscript Indicates the direction type, and its value set is: ,in Indicates the horizontal direction. Indicates the vertical direction. Indicates diagonal direction; Wavelet function representing the corresponding direction The generated high-frequency detail subbands, specifically: hour, Indicates the horizontal high-frequency sub-band; hour, Indicates the vertical high-frequency sub-band; hour, Indicates the diagonal high-frequency sub-band; symbol This represents the set union operation; Step a2: For each time step of the diffusion denoising process, acquire the current noise image to be processed, and perform a two-dimensional discrete wavelet transform on the noise image to decompose it into a low-frequency approximate sub-band and three high-frequency detail sub-bands, as shown in the following equation: in, This represents the noisy image currently being processed. Step a3 involves weighted fusion of the low-frequency approximate subbands of the content image and the low-frequency approximate subbands of the noise image to obtain the fused low-frequency information with enhanced structure, as shown in the following formula: in, Low-frequency information indicating structural enhancement; Content image Low-frequency information obtained through two-dimensional discrete wavelet transform; Representing a noisy image Low-frequency information obtained through two-dimensional discrete wavelet transform; Represents the learnable global fusion coefficients; w represents a tensor with the same shape as the low-frequency subband, such that all spatial locations and channels share the same fusion weight; Step a4: Using the low-frequency information of the structure enhancement and the high-frequency detail information of the noise image to be processed, the structure-enhanced noise image is reconstructed through inverse two-dimensional discrete wavelet transform, as shown in the following equation: in, A noisy image representing structural enhancement; This represents the inverse discrete wavelet transform.

4. The structure-enhanced diffusion denoising network as described in claim 3, characterized in that, The U-Net backbone network uses the following steps to output generated images that conform to the specified content and style: Step b1: In the U-Net backbone network, the multi-scale content feature aggregation downsampling block first concatenates the output of the previous downsampling block with the content features of the corresponding level along the channel dimension. The spliced ​​features are weighted to form a two-level fused feature of intermediate layer content with residual connections; Secondly, the number of feature channels is compressed to the target dimension through optional convolution operations; Finally, the style features extracted by the style image encoder are injected into the two-level fusion features using the cross-attention mechanism, resulting in the three-level fusion features, as shown below: in, This represents the input of the current downsampling block; This indicates the output characteristics of the current downsampling block; This indicates splicing along the channel dimension; This represents a lightweight feature map that includes normalization, non-linear activation, and 1×1 convolution; This indicates the channel attention module; This means compressing the spliced ​​high-dimensional features to the target number of channels; This indicates that the global style vector Flatten out into a sequence of context tokens; Indicated by For query, , This is a cross-attention mechanism for key-value pairs; The ordinary downsampling block receives the intermediate layer features output from the previous downsampling block and performs feature extraction. Optionally, it performs convolutional downsampling to reduce spatial resolution, as shown in the following equation: in: This represents the input of the current downsampling block; This indicates the output characteristics of the current downsampling block; This represents a standard residual convolutional block, which includes normalization, non-linear activation, and convolution operations. This indicates an optional convolutional downsampling operation; Step b2: The intermediate bottleneck layer performs a refined fusion of the deepest features, content features, and style features; Step b3: The upsampling block of each style aggregation fuses the output features from the previous decoder, the skip connection features from the corresponding encoder layer, and the global style features to obtain new U-Net intermediate layer features, as shown below: In the formula: This indicates that the current upsampled block comes from the input features of the previous level decoder; This represents the skip connection features from the corresponding level of the encoder; This indicates that the global style vector Flatten out into a sequence of context tokens; The standard upsampling block concatenates the output features from the previous decoder with the skip connection features from the corresponding level of the encoder, and performs feature fusion through a standard residual convolutional unit. Finally, it optionally performs a convolutional upsampling operation to improve spatial resolution. The top-level output features of the decoder are mapped to noise prediction values ​​with the same number of channels as the input image through a convolutional layer. This is used for subsequent iterations of noise reduction; Step b4: Input the current noisy image into the structure enhancement unit to generate a structure-enhanced noisy image; Input the U-Net backbone network and combine it with the embedding t at the current time step and multi-scale content features. With global style feature vector Output the predicted value of the current noise. Finally, the image is updated according to the backsampling formula of the diffusion model to obtain the noisy image at the next time step. As shown in the following formula: In the formula: Represents the noise image at step t; This represents a noisy image after structural enhancement. This represents the image obtained at step t-1 after one step of denoising; , ,in These are preset variance scheduling hyperparameters; This represents the noise residual predicted by the U-Net backbone network; Represents the sampling noise figure. ; For standard Gaussian noise, when When used to introduce randomness, when At that time, set ; After multiple iterations, a generated image that conforms to the specified content and style is obtained.

5. A font generation method for solving glyph structure distortion based on frequency domain structure enhancement, the method being based on the structure-enhanced diffusion denoising network described in claim 4, characterized in that, Includes the following steps: Step 1: Construct a character image dataset; Step 2: Construct the structure-enhanced diffusion denoising network as described in claim 4; Step 3: Use the character image dataset as the training set to train the structure-enhanced diffusion denoising network. Use the trained structure-enhanced diffusion denoising network as the font generation model to achieve single-sample font generation that solves the problem of character structure distortion.

6. The font generation method for solving glyph structure distortion based on frequency domain structure enhancement as described in claim 5, characterized in that, Step 1 includes the following sub-steps: Step 11: Select one font from the public database as the content font, and select several other fonts as style fonts; Step 12: Convert the multiple characters corresponding to the content font into images, which will serve as the content image; Convert multiple characters corresponding to the style font into images, which are then used as style images; Convert multiple characters of the corresponding content font of the style font into images, which are then used as real images; All content images, style images, and real images are compiled into a glyph image dataset.

Images