Small sample font generation method and system, computer equipment and storage medium

Abstract

The invention discloses a small sample font generation method and system, computer equipment and a storage medium, and is applied to the technical field of font generation, and the method comprises the steps: obtaining a source font image and a target font style reference video; processing the source font image and the target font style reference video to form a visual condition; inputting the visual condition into a pre-constructed font video generation framework, and outputting a video sequence transited from a source font image to a target font image; extracting the last frame of the optimized video sequence as a generated target font image; according to the method, the font generation task is modeled into the video generation task, and the diffusion model and the condition alignment mechanism are combined, so that the defects of structure fidelity, style consistency, training stability and the like in font generation in the prior art are successfully overcome, and high-quality and high-consistency font image generation is realized.

Description

Technical Field

[0001] This invention relates to the field of font generation technology, and in particular to a method, system, computer device, and storage medium for generating small sample fonts. Background Technology

[0002] In recent years, few-shot font generation (FFG) has become a research hotspot. Its core lies in generating new font character images using only a very small number of reference samples, that is, transferring the stylistic features of the reference font while ensuring the content of the source characters remains unchanged. This type of technology has broad commercial value and application prospects, and is particularly suitable for writing systems with numerous glyphs.

[0003] Currently, most existing research is based on the idea of ​​decoupling content and style. Specifically, it uses independent content encoders and style encoders to extract structural information of source characters and style features of reference samples, respectively, and then fuses them to generate the target character image. Early methods modeled content and style features as global vectors, but this approach struggles to capture fine-grained local changes in fonts, resulting in unsatisfactory generation results. Therefore, subsequent research has proposed more refined representation methods, such as breaking down characters into components or strokes and encoding them locally, which has improved the accuracy and diversity of the generated images to some extent.

[0004] However, existing methods still fall short in handling complex character structures and preserving stylistic details. The generated results often suffer from missing strokes, structural breaks, blurred details, and insufficient stylistic consistency, performing even worse when faced with unfamiliar fonts and new character combinations. To address these issues, researchers have explored two approaches. One is to improve the representation of content and style, for example, by constructing more stable content features through a base font space, or by combining contrastive learning and diffusion models to obtain more robust style representations. The second is to model font generation as a deformation process, that is, a gradual transformation from the source font to the target font, thereby better preserving the overall character structure. However, existing deformation methods generally have limitations: some methods only perform interpolation in the latent space, lacking explicit supervision of the deformation process, making it difficult to accurately capture the true laws of font evolution. Overall, how to simultaneously ensure structural integrity and stylistic consistency remains a key challenge that urgently needs to be overcome in current technologies.

[0005] To overcome these shortcomings, this application proposes a method, system, computer device, and storage medium for generating small sample fonts. Summary of the Invention

[0006] The purpose of this application is to provide a method, system, computer device, and storage medium for generating small-sample fonts, aiming to solve the problems of insufficient structural fidelity and style consistency in existing small-sample font generation methods.

[0007] To achieve the above objectives, this application provides the following technical solution:

[0008] Firstly, this application provides a method for generating small sample fonts, the steps of which include:

[0009] Obtain the source font image and the target font style reference video;

[0010] The source font image and the target font style reference video are processed to form visual conditions;

[0011] The visual conditions are input into a pre-constructed font video generation framework, and a video sequence transitioning from the source font image to the target font image is output; wherein, during the generation of the video sequence, a conditional alignment mechanism is used to optimize the generation process of the last frame;

[0012] The last frame of the optimized video sequence is extracted as the target font image for generation.

[0013] Secondly, this application provides a small-sample font generation system, specifically including:

[0014] Acquisition module: Acquires source font images and target font style reference videos;

[0015] Conditional fusion module: processes the source font image and the target font style reference video to form visual conditions;

[0016] Video generation module: Inputs the visual conditions into a pre-constructed font video generation framework and outputs a video sequence transitioning from the source font image to the target font image; wherein, during the video sequence generation process, a conditional alignment mechanism is used to optimize the generation process of the last frame;

[0017] Image output module: Extracts the last frame of the optimized video sequence as the target font image for generation.

[0018] Thirdly, this application provides a computer device, the computer device including a processor and a memory coupled to the processor, wherein the memory stores program instructions for implementing a small-sample font generation method; the processor is used to execute the program instructions stored in the memory to implement a small-sample font generation.

[0019] Fourthly, this application provides a storage medium storing processor-executable program instructions for executing a small-sample font generation method.

[0020] This application provides a method, system, computer device, and storage medium for generating small sample fonts, which has the following beneficial effects:

[0021] (1) By innovatively modeling the font generation task as a video generation process, the model fully utilizes the inter-frame continuity to ensure the integrity of the overall character structure, effectively avoiding the common problems of missing strokes, breaks and structural deformation in traditional methods, significantly improving the accuracy of complex font generation, and performing particularly well in the processing of fonts with rich details; at the same time, relying on the inter-frame self-attention mechanism, the model can incorporate information from other frames in each frame generation, accurately simulating the frame-by-frame evolution process from the source font to the target font, achieving efficient and natural style transition, and completely solving the problems of style abrupt changes and inconsistent connections in the transition stage in traditional methods;

[0022] (2) The test-time conditional alignment mechanism and noise inversion mechanism introduced in this application can accurately adjust the generation process, effectively eliminate generation deviations between samples, and significantly enhance the style consistency and content accuracy of the generated fonts. This mechanism is particularly advantageous when facing unseen fonts or new character combinations, and successfully solves the pain points of inconsistent styles and distorted content in the processing of small samples and unseen samples in traditional methods. In addition, based on the synergistic effect of the video generation framework and diffusion model, it can better adapt to diverse font styles and character combinations. Whether it is known fonts or unknown fonts, it can stably output high-quality results, greatly improving the diversity and adaptability of the technology.

[0023] (3) This application constructs a font generation framework with the diffusion model as the core and adopts the image-to-video generation approach, which fundamentally avoids the instability and mode collapse problems that are prone to occur in the training of generative adversarial networks. The diffusion model, through the generation method of progressive denoising, not only ensures the smooth convergence of the training process, but also ensures the high quality of the generation results. This advantage is particularly important in the scenario of small sample font generation, enabling the model to continuously and stably output high-quality font images that meet expectations under limited reference samples, providing a reliable guarantee for the actual implementation of the technology. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating a small-sample font generation method according to Embodiment 1 of this application;

[0025] Figure 2 This is a schematic diagram of the font image generation framework based on the diffusion model in Embodiment 1 of this application;

[0026] Figure 3 This is a schematic diagram of the font video generation framework based on the diffusion model in Embodiment 1 of this application;

[0027] Figure 4 This is a schematic diagram of the structure of a small-sample font generation system according to Embodiment 2 of this application;

[0028] Figure 5This is a schematic diagram of the computer device structure according to Embodiment 3 of this application;

[0029] Figure 6 This is a schematic diagram of the storage medium structure of Embodiment 4 of this application. Detailed Implementation

[0030] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0031] The following analysis, based on relevant technologies, examines existing solutions.

[0032] Existing few-shot font generation (FFG) methods suffer from two core drawbacks: incomplete character structure and poor style consistency. First, existing methods typically decouple content and style, extracting and combining the content of the source characters and the style features of reference samples to generate the target font. However, this approach often results in incomplete character structure when generating complex fonts, particularly in preserving stroke details. The font deformation process is often not effectively modeled, leading to a lack of precise structural and morphological details in the generated characters. Second, existing methods exhibit poor style consistency, especially when generating unseen fonts or new character combinations, often resulting in stylistic instability. This is because many existing techniques employ localized style representation, failing to adequately consider maintaining style consistency across multiple frames, leading to excessive style variations or a lack of coordination in local areas.

[0033] Therefore, this application proposes a few-sample font generation method and system. By modeling the font generation task as a video generation process, the transition between the source and target fonts is decomposed into a series of frames. The continuity between frames ensures the integrity of the font structure, thus avoiding missing or broken strokes during generation. Simultaneously, by introducing a conditional alignment mechanism, style consistency during the generation process is optimized, ensuring that the generated font retains the content of the source characters while accurately representing the stylistic features of the reference font.

[0034] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0035] Example 1

[0036] Please see Figure 1 This is a flowchart illustrating a small-sample font generation method according to Embodiment 1 of this application; the steps include:

[0037] S1: Obtain the source font image and the target font style reference video;

[0038] S2: Process the source font image and the target font style reference video to form visual conditions;

[0039] S3: Input the visual conditions into a pre-constructed font video generation framework, and output a video sequence transitioning from the source font image to the target font image; wherein, during the generation of the video sequence, a conditional alignment mechanism is used to optimize the generation process of the last frame;

[0040] S4: Extract the last frame of the optimized video sequence as the target font image for generation.

[0041] In this embodiment, character images are first acquired as source font images to provide the core structural information of the font to be generated. A reference video of the target font style is also acquired; the video must cover the typical stroke features of the target style so that the style encoder can extract complete temporal style features.

[0042] The source font image is input into the content encoder to extract content features. The target font style reference video is decomposed and input frame by frame into the style encoder. The style encoder extracts style feature vectors from each frame and generates temporal style features through average pooling. The content features and temporal style features are concatenated to form an initial feature sequence. Simultaneously, the source font image and reference video frames are projected into the latent space through the VAE encoder to generate noisy latent features. The initial feature sequence is combined with the latent space noise features, and time step information is incorporated to finally form the visual conditions, which serve as the input to the font video generation framework.

[0043] The font video generation framework proposed in this application is based on a denoising neural network with a transformer. It adds a temporal transformer to the traditional image denoising block model. Specifically, it includes: ignoring the time axis, reshaping latent spatial features to the batch axis, and independently processing the spatial details of each frame through self-attention and feedforward networks. The temporal transformer contains a temporal self-attention layer, a temporal feedforward layer, and an adaptive normalization module. The temporal self-attention layer models inter-frame correlations, enabling the generation of the current frame to integrate structural and stylistic information from other frames. The frame index is input into the adaptive normalization module to ensure that the model generates transition frames from the source font to the target font sequentially.

[0044] Backdiffusion generation is performed using a DDIM (Denoising Diffusion Implicit Model) sampler. Specifically, this involves sampling the initial noise latent feature z from a standard normal distribution N(0,I). TIteratively denoise according to the time step \(T\rightarrow1\). At each step, combine the visual conditions and the frame index information, and predict the noise through the denoising neural network to update the latent features. Finally, generate a video sequence.

[0045] To address the generation bias existing for unseen font styles, optimize the generation process through a noise inversion mechanism.

[0046] Based on the DDIM iteration formula Fix the network parameters and the features of the previous frames, and only adjust the noise adopted corresponding to the last frame.

[0047] The formula is expressed as:

[0048]

[0049] where is the last frame of the font video generated at the time step \(t\); is the L2 loss function. Minimize the L2 loss through gradient descent to update the noise map.

[0050] Input the last frame of the optimized video sequence into the VAE decoder \(D\). Through the recovery operation from the latent space to the image space, generate a target font image with the resolution. After post-processing this image, it can be used for subsequent application scenarios.

[0051] Optionally, in the step of processing the source font image and the target font style reference video to form visual conditions, it specifically includes:

[0052] Extract the content features of the source font image through a content encoder;

[0053] Extract the features of each frame of the target font style reference video through a style encoder to obtain temporal style features;

[0054] Concatenate the content features and the temporal style features to form a visual condition vector.

[0055] In one embodiment, the content encoder adopts a 6-layer Transformer architecture, including a Patch Embed layer, multiple multi-head self-attention modules, and a point-wise feed-forward network. Input a single-channel source font image with a resolution of \(256\times256\) (in this embodiment, it is the Song typeface character "书"). First, the Patch Embed layer disassembles the image into a patch sequence of \(16\times16\), and converts it into a feature sequence with a dimension of \(256\times768\). The self-attention module captures the spatial correlations between the patches (such as the positional relationship between the horizontal and vertical strokes of the character "书"), and then through the feed-forward network, perform non-linear feature transformation. Finally, output a content feature vector with a dimension of 512, which only retains the character structure information and eliminates style interference.

[0056] The target font style reference video is a transition video of the Thin Gold style font, with a frame rate of 10fps and a duration of 1.5 seconds. It contains 15 frames of 256×256 images, covering the typical characteristics of Thin Gold style: "thin horizontal strokes, strong vertical strokes, and sharp, angular strokes." The style encoder adopts the same transformer structure as the content encoder, with an added frame-level feature cache layer. It receives video frames one by one and extracts style features. After each frame is processed by Patch Embed and 4 layers of self-attention, it outputs 512-dimensional single-frame style features. The 15 frame features are arranged in chronological order to form a 15×512 temporal style feature. The feature distribution is standardized through normalization to preserve the temporal correlation of style evolution with frames. The content feature vector is expanded in dimension, reshaped from 512 dimensions to 1×512, and concatenated with the temporal style feature (15×512) in the time dimension to form an initial feature sequence of 16×512. A VAE encoder is introduced to perform latent space projection on the source font image, generating a 1×256 latent space noise feature. This feature is then concatenated with the initial feature sequence along the channel dimension, ultimately forming a 16×768 visual conditional vector, which serves as the input to the font video generation framework. By extracting and concatenating features through a content encoder and a style encoder respectively, the problem of separation between structure and temporal style in traditional feature fusion is solved, providing accurate conditional input for subsequent video generation.

[0057] Optionally, the font video generation framework is constructed based on a denoising network;

[0058] The denoising network includes an image transformer block and a time transformer block. The image transformer block is used to process spatial features, and the time transformer block is used to process temporal correlations.

[0059] In one embodiment, the font video generation framework uses a transformer-based denoising neural network as its core and relies on the DDIM sampler to achieve backdiffusion generation. The network input is a visual conditional vector and initial noise latent features, and the output is a font transition video sequence.

[0060] The denoising network comprises "image transformer block - temporal transformer block" units, each followed by residual connections and layer normalization to ensure the stability of feature propagation. The image transformer block adopts the PixArt-α image transformer structure, including a multi-head self-attention layer and a point-wise feedforward network. Input features are first reshaped, ignoring the temporal dimension and retaining only the spatial dimension. The self-attention layer captures the spatial dependencies of pixels within a single frame, and the feedforward network further enhances the non-linear expression of spatial features. The temporal transformer block is concatenated with the image transformer block, containing a temporal multi-head self-attention layer and a temporal feedforward network. Its core function is to model inter-frame feature correlations. It receives the spatial features output by the image transformer block, reshapes and restores the temporal dimension, and calculates the feature similarity between the current frame and other frames through the temporal self-attention layer. This application's font video generation framework, built based on the denoising network, addresses the problem that traditional frameworks cannot simultaneously maintain structural integrity and inter-frame coherence through the synergistic effect of the image transformer block and the temporal transformer block, respectively processing spatial features and temporal correlations.

[0061] Optionally, the time transformer block includes at least a time self-attention layer and an adaptive layer normalization module;

[0062] The time transformer takes the acquired frame position code as input, and the frame position code is integrated into the forward computation process of the denoising network through the adaptive layer normalization module;

[0063] The temporal self-attention layer is used to calculate the attention weights between the current frame and other frames.

[0064] In one embodiment, the time transformer block consists of a temporal self-attention layer, an adaptive layer normalization module (adaLN), and a temporal feedforward network. The input is the feature tensor output by the image transformer block, and the output is a feature tensor of the same dimension with fused temporal correlations. Frame position codes are generated according to the video sequence frame number: a sinusoidal position coding formula is used to generate position vectors, where the code for the i-th frame includes its temporal information within the sequence.

[0065] The temporal style features in the position vector and visual conditional vector are fused and input into the adaLN module. adaLN integrates frame position encoding into the forward computation of the denoising network through learned scaling factor γ and offset factor β, ensuring that the model generates transition frames in a fixed order.

[0066] The calculation of attention weights in the temporal self-attention layer specifically includes: employing a scaled dot product attention mechanism to decompose the input feature tensor into query (Q), key (K), and value (V) matrices along the time dimension. This application achieves frame position information integration and cross-frame attention calculation through the collaboration of the temporal self-attention layer and the adaptive layer normalization module, solving the problems of frame order disorder and weakened correlation in traditional temporal processing.

[0067] Optionally, after extracting features from the target font style reference video, the obtained temporal style features are used as style addition inputs to the denoising network.

[0068] Optionally, the method further includes, during the generation process of the video sequence, merging the time dimension into the batch dimension and processing the spatial features of each frame through the denoising network;

[0069] After the processing is completed, the feature map is reshaped back to the original tensor shape including the time dimension.

[0070] Optionally, in the step of optimizing the generation process of the last frame by using the conditional alignment mechanism, it specifically includes:

[0071] Adjusting the generation process by using the noise inversion mechanism;

[0072] Optimizing the generation result of the last frame by minimizing the difference loss between the generated image and the target condition.

[0073] Please refer to Figure 2 , which is a schematic diagram of the font image generation framework based on the diffusion model in Embodiment 1 of this application. It shows the entire process from the input of the source font image and the reference image to the generation of the target font image. The source font image i c (such as the character "英") is passed through the content encoder E c to extract content features, and the reference image I s (such as "候" and "凝") is passed through the style encoder E s to extract style features, and the two are fused into the visual condition e img . The font image is added noise through the VAE encoder to obtain , which is input into the transformer module containing multi-head self-attention, multi-head cross-attention, and pointwise feed-forward networks after image block embedding. At the same time, combined with time step embedding and scaling and offset operations, after N times of module processing, the generated font image is output through inverse image blockization The entire process realizes the accurate generation from the source font to the target style font through the fusion of content and style features and the gradual denoising of the diffusion model.

[0074] Please refer to Figure 3 , which is a schematic diagram of the font video generation framework based on the diffusion model in Embodiment 1 of this application. It shows the entire process from the input of the source font image and the reference video to the generation of the target font video. The source font image I c (such as the character "英") is passed through the content encoder E c to extract content features, and the reference video V S (a multi-frame image containing styles such as "候" and "凝") is passed through the style encoder E SStyle features are extracted and fused with frame index embedding and temporal step embedding to form a visual conditional e vid Font image sequence Noise added by VAE encoder After image patch embedding, the input consists of a core module composed of an image transformer block (containing multi-head self-attention, multi-head cross-attention, and pointwise feedforward networks to process single-frame spatial features) and a temporal transformer block (containing temporal multi-head attention and temporal feedforward layers to process inter-frame temporal correlations). During this process, scaling and offset operations are incorporated. After N module processing steps, the image is de-patched, ultimately outputting the generated font video. This framework achieves coherent video generation from source font to target style font through spatial-temporal feature collaborative processing, content-style fusion, and diffusion model denoising.

[0075] In summary, Embodiment 1 of this application first acquires a source font image and a target font style reference video. Then, both are processed to form visual conditions. Specifically, a content encoder extracts the content features of the source font image, and a style encoder extracts features from each frame of the target font style reference video to obtain temporal style features. The content features and temporal style features are then concatenated to form a visual condition vector. The visual conditions are then input into a font video generation framework pre-built based on a denoising network. During generation, the temporal dimension is first merged into the batch dimension to process spatial features, and then the tensor containing the temporal dimension is reconstructed. Simultaneously, a conditional alignment mechanism is used to optimize the last frame, outputting a transitional video sequence from the source font to the target font. Finally, the last frame of the optimized video sequence is extracted as the generated target font image. This application successfully solves the shortcomings of existing technologies in font generation, such as structural fidelity, style consistency, and training stability, by modeling the font generation task as a video generation task and combining a diffusion model and a conditional alignment mechanism, achieving high-quality, highly consistent font image generation.

[0076] Example 2

[0077] Please see Figure 4 This is a schematic diagram of the structure of a small-sample font generation system according to Embodiment 2 of this application; the specific content includes:

[0078] Acquisition module: Acquires source font images and target font style reference videos;

[0079] Conditional fusion module: processes the source font image and the target font style reference video to form visual conditions;

[0080] Video generation module: Inputs the visual conditions into a pre-constructed font video generation framework and outputs a video sequence transitioning from the source font image to the target font image; wherein, during the video sequence generation process, a conditional alignment mechanism is used to optimize the generation process of the last frame;

[0081] Image output module: Extracts the last frame of the optimized video sequence as the target font image for generation.

[0082] In this embodiment, the acquisition module receives the source font image and the target font style reference video uploaded by the user, performs format normalization processing through OpenCV, and outputs a single-channel image and a video frame sequence of the same resolution.

[0083] The conditional fusion module integrates the content encoder and style encoder, extracts content features and temporal style features, generates visual conditional vectors through tensor splicing and latent space projection, and outputs them to the video generation module.

[0084] The video generation module includes a font video generation framework based on a denoising network, a DDIM sampler, and a conditional alignment optimization unit. It integrates temporal and batch-dimensional spatial features, models inter-frame relationships through a temporal transformer block, inputs temporal style features across attention layers, and finally optimizes the last frame using a noise inversion mechanism, outputting an optimized video sequence.

[0085] The image output module includes a VAE decoder (D) and an image post-processing unit. It extracts the latent features of the last frame of the video sequence, reconstructs it into a 256×256 image using the VAE decoder, enhances its clarity through post-processing, and outputs the target font image, which can be downloaded by the user or directly integrated with typesetting software.

[0086] In summary, Embodiment 2 of this application acquires a source font image and a target font style reference video through an acquisition module. Subsequently, a conditional fusion module processes both to form visual conditions. Specifically, a content encoder extracts content features from the source font image, and a style encoder extracts features from each frame of the target font style reference video to obtain temporal style features. The content features and temporal style features are then concatenated to form a visual conditional vector. The visual conditions are input into a font video generation framework pre-built based on a denoising network. This denoising network includes an image transformer block for processing spatial features and a temporal transformer block for processing temporal correlations. The temporal transformer block contains at least a temporal self-attention layer and an adaptive layer normalization module. Frame position encoding is integrated into the forward computation through the adaptive layer normalization module, and the temporal self-attention layer calculates the attention weights between the current frame and other frames. Simultaneously, the temporal style features are added as style input to the denoising network. During the generation process, the temporal dimension is first merged into the batch dimension. After processing the spatial features of each frame through a denoising network, the feature map is reshaped back into the original tensor shape containing the temporal dimension. A conditional alignment mechanism is used to optimize the last frame; that is, a noise inversion mechanism is used to adjust the generation process to minimize the difference between the generated image and the target condition. Finally, the image output module extracts the last frame of the optimized video sequence as the generated target font image.

[0087] Example 3

[0088] Please see Figure 5 This is a schematic diagram of the computer device structure according to Embodiment 3 of this application. The computer device 50 includes a processor 51 and a memory 52 coupled to the processor 51.

[0089] The memory 52 stores program instructions for implementing the above-described method for generating small sample fonts.

[0090] The processor 51 is used to execute program instructions stored in the memory 52 to implement a small sample font generation.

[0091] The processor 51 can also be referred to as a CPU (Central Processing Unit).

[0092] Processor 51 may be an integrated circuit chip with signal processing capabilities. Processor 51 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0093] Example 4

[0094] Please see Figure 6 This is a schematic diagram of the storage medium in Embodiment 4 of this application. The storage medium in this embodiment stores a program file 61 capable of implementing all the above methods. This program file 61 can be stored in the storage medium in the form of a software product, including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or devices such as computers, servers, mobile phones, and tablets.

[0095] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0096] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

[0097] Although embodiments of this application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the appended claims and their equivalents.

[0098] Of course, the present invention may have many other embodiments. Based on this embodiment, other embodiments obtained by those skilled in the art without any creative effort are all within the scope of protection of the present invention.

Claims

1. A method for generating small-sample fonts, characterized in that, include: Obtain the source font image and the target font style reference video; The source font image and the target font style reference video are processed to form visual conditions; The visual conditions are input into a pre-constructed font video generation framework, and a video sequence transitioning from the source font image to the target font image is output; wherein, during the generation of the video sequence, a conditional alignment mechanism is used to optimize the generation process of the last frame; The last frame of the optimized video sequence is extracted as the target font image for generation.

2. The method for generating small sample fonts according to claim 1, characterized in that, The step of processing the source font image and the target font style reference video to form visual conditions specifically includes: The content features of the source font image are extracted using a content encoder; Temporal style features are obtained by extracting features from each frame of the target font style reference video using a style encoder. The content features and the temporal style features are concatenated to form a visual condition vector.

3. The method for generating small sample fonts according to claim 2, characterized in that, The font video generation framework is constructed based on a denoising network. The denoising network includes an image transformer block and a time transformer block. The image transformer block is used to process spatial features, and the time transformer block is used to process temporal correlations.

4. The method for generating small sample fonts according to claim 3, characterized in that, The time transformer block includes at least a time self-attention layer and an adaptive layer normalization module; The time transformer takes the acquired frame position code as input, and the frame position code is integrated into the forward computation process of the denoising network through the adaptive layer normalization module; The temporal self-attention layer is used to calculate the attention weights between the current frame and other frames.

5. The method for generating small sample fonts according to claim 4, characterized in that, After extracting features from the target font style reference video, the obtained temporal style features are used as style addition inputs to the denoising network.

6. The method for generating small sample fonts according to claim 1, characterized in that, The method further includes merging the time dimension into the batch dimension during the video sequence generation process, and processing the spatial features of each frame through the denoising network; After processing, the feature map is reshaped back into the original tensor shape that includes the time dimension.

7. The method for generating small sample fonts according to claim 1, characterized in that, The steps for optimizing the generation process of the last frame using a conditional alignment mechanism specifically include: The generation process is adjusted using a noise inversion mechanism; The generation result of the last frame is optimized by minimizing the difference loss between the generated image and the target conditions.

8. A small-sample font generation system, characterized in that, The few-sample font generation system is used to perform the few-sample font generation method according to any one of claims 1 to 7, and comprises: Acquisition module: Acquires source font images and target font style reference videos; Conditional fusion module: processes the source font image and the target font style reference video to form visual conditions; Video generation module: Inputs the visual conditions into a pre-constructed font video generation framework and outputs a video sequence transitioning from the source font image to the target font image; wherein, during the video sequence generation process, a conditional alignment mechanism is used to optimize the generation process of the last frame; Image output module: Extracts the last frame of the optimized video sequence as the target font image for generation.

9. A computer device, characterized in that, The computer device includes a processor and a memory coupled to the processor, wherein the memory stores program instructions for implementing the small-sample font generation method according to any one of claims 1-7; the processor is used to execute the program instructions stored in the memory to implement small-sample font generation.

10. A storage medium, characterized in that, The device stores processor-executable program instructions for performing the small sample font generation method according to any one of claims 1-7.

Images