Hair body generation method and system based on joint diffusion-adversarial training model

Abstract

The present application provides a kind of based on joint diffusion-adversarial training model's hair body generation method and system, the method includes the following steps: the hair body sample data set obtained is preprocessed, to obtain hair body feature image;The hair body feature image is input to the joint diffusion-adversarial training model constructed and is trained, to obtain first hair body feature image;According to the first hair body feature image, the joint diffusion-adversarial training model is optimized;According to the joint diffusion-adversarial training model after optimization, target hair body image is obtained;By this method, the artistic features of hair body can be effectively captured, and the generated hair body character has both structural stability and can reflect the unique pen potential of hair body, such as details features of turning and ending;The system has the same beneficial effects.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing and artificial intelligence, and in particular to a Mao body generation method and system based on a joint diffusion-adversarial training model. BACKGROUND

[0002] In recent years, font generation technology has made significant progress in printed and standard calligraphy fonts such as Kai and Lishu. Printed font generation relies on the stability of character structure and has achieved high maturity through style decoupling and feature migration. Compared with the above-mentioned structured fonts, cursive script faces greater challenges in the generation task. Due to the strong connectedness, rich pen potential, and high coupling of style of cursive script, traditional methods are difficult to effectively decouple the writing style and character content, and the effect of feature migration is restricted by the complex structure and style diversity of cursive script, making it difficult to achieve high-quality style conversion and content fidelity, resulting in distorted generation results and difficulty in restoring the unique artistic charm of cursive script. The bottleneck of cursive script is further intensified in Mao body, which combines the unique penmanship of sudden and continuous turning, forming a revolutionary aesthetic image of wide opening and close closing, with a unique tension and chapter momentum that surpasses the scope of regular cursive script.

[0003] With the development of artificial intelligence and digital technology, Mao body, as the most revolutionary and artistic form of Chinese modern calligraphy, has a unique wide opening and close closing, and a bold and vigorous penmanship style. However, there is still a lack of effective solutions in existing font generation research. Mao body not only has complex stroke structure, but also has strong artistic character, rich and delicate changes in pen potential, which makes it difficult for traditional font generation methods to accurately reproduce its unique artistic style and structural features when generating Mao body characters.

[0004] Existing generation methods, such as diffusion model-based font generation technology, have achieved good results in printed and handwritten font generation, but there are still problems such as stroke distortion, detail loss, and insufficient pen potential changes in Mao body generation.

[0005] Therefore, how to accurately generate Mao body characters by combining the artistic characteristics of Mao body is a technical problem that needs to be solved by those skilled in the art. SUMMARY

[0006] To solve the above technical problems, the purpose of the present application is to provide a Mao body generation method and system based on a joint diffusion-adversarial training model, which can effectively capture the artistic features of Mao body, making the generated Mao body characters not only structurally stable, but also reflecting the unique pen potential fluctuations, transitions, and endings of Mao body.

[0007] The first purpose of the present application is to provide a Mao body generation method based on a joint diffusion-adversarial training model;

[0008] The technical scheme provided by the present application is as follows:

[0009] A hair body generation method based on a joint diffusion-adversarial training model, comprising the following steps:

[0010] The obtained hair body sample data set is preprocessed to obtain a hair body feature image;

[0011] The hair body feature image is input into the constructed joint diffusion-adversarial training model for training to obtain a first hair body feature image;

[0012] The joint diffusion-adversarial training model is optimized according to the first hair body feature image;

[0013] The target hair body image is obtained according to the optimized joint diffusion-adversarial training model.

[0014] Preferably, the hair body feature image is input into the constructed joint diffusion-adversarial training model for training to obtain a first hair body image, specifically comprising:

[0015] The joint diffusion-adversarial training model is trained according to the joint diffusion-adversarial training mechanism of deterministic DDIM sampling and time reverse propagation and the hair body feature image to generate the first hair body image.

[0016] Preferably, the joint diffusion-adversarial training mechanism of deterministic DDIM sampling and time reverse propagation is specifically:

[0017] Deterministic generation: using DDIM sampling to generate the hair body feature image According to the model parameters and conditions, the function of the deterministic iteration is calculated The specific calculation formula is:

[0018] ;

[0019] Wherein represents the sampling process, and is deterministic, and the calculation of the hair body feature image contains all intermediate time steps , ,..., }; represents the initial noise sampled from the standard Gaussian distribution, and the value range of z is ; c represents the model condition; and θ represents the model parameter;

[0020] Adversarial loss: using an adversarial loss discriminator to calculate the style-related loss of the hair body feature image , and the formula is as follows:

[0021] ;

[0022] in, These are real image samples; : Labels for real samples; Indicates a generator; Represents the mathematical expectation;

[0023] Gradient backpropagation: the style-related loss gradient with respect to model parameters θ Perform a backpropagation to update θ; gradient The calculation formula is:

[0024] ;

[0025] in, This represents the gradient of the discriminator output on the hair feature image; The hair feature image is the model parameter. The gradient; the hair feature image is the final output of the DDIM sampling chain, according to the chain rule, It needs to be accumulated along the time step:

[0026] ;

[0027] Each of them The term is determined by the DDIM sampling formula, where T represents the total number of time steps in the diffusion process; the DDIM update steps are typically as follows:

[0028] ;

[0029] in, For the diffusion process at time step Image feature representation; For U-Net noise prediction network; This represents the cumulative product of noise control parameters during the diffusion process.

[0030] but Dependent on U-Net noise prediction network gradient of its parameter θ Make adjustments;

[0031] Based on deterministic DDIM sampling, the final gradient of the model parameter θ Represented as:

[0032] ;

[0033] in, This indicates feature-level feedback; This contains generator parameter information.

[0034] Preferably, optimizing the joint diffusion-adversarial training model based on the first hair feature image specifically includes:

[0035] The character features, style features, and stroke features of the first Mao-style feature image generated by the joint diffusion-adversarial training model are optimized according to the structured codebook embedding mechanism.

[0036] Based on the stroke features in the first hair feature image, a region-aware attention mechanism is used to dynamically adjust the attention weights in order to optimize the key regions of the stroke features.

[0037] The joint diffusion-adversarial training model is optimized for multiple generation dimensions based on a composite loss function constructed using the first hair feature image.

[0038] Preferably, the optimization of the character features, style features, and stroke features of the first hair-feature image generated by the joint diffusion-adversarial training model based on the structured codebook embedding mechanism specifically includes:

[0039] Generate a codebook vector based on the character codebook, style codebook, and component codebook;

[0040] The codebook vector is incorporated as input condition into the U-Net of the joint diffusion-adversarial training model to optimize the character features, style features, and stroke features of the first Mao-style feature image.

[0041] Preferably, the step of using a region-aware attention mechanism to dynamically adjust attention weights based on stroke features in the first hair feature image to optimize key regions of the stroke features specifically includes:

[0042] The region-aware attention mechanism includes: global attention, local separation attention, and pen gesture attention;

[0043] The global attention mechanism enhances the expression of important structures in Mao Zedong's calligraphy by selecting feature channels. Its calculation formula is as follows:

[0044] ;

[0045] in, It is the selected channel attention weight. These are the characteristics of the corresponding channels. This indicates information about the enhanced key feature channels; 'c' represents the feature channel index.

[0046] Local Separate Attention: This method focuses on different stroke feature regions by selecting spatial areas. The calculation formula is as follows:

[0047] ;

[0048] in, The selection matrix represents the local region. This indicates the characteristics of the corresponding region. This indicates the feature representation after focusing on a specific stroke region; Indicates a spatial region index;

[0049] Brushstroke Attention: This function simulates the movement trajectory of a brush by shifting the position of a feature, in order to reproduce the changes in brushstrokes. The calculation formula is as follows:

[0050] ;

[0051] in, Features that indicate a specific location; , Indicates the offset at a specific location; This represents the trajectory of the brushstroke simulated by the feature adjusted for position offset.

[0052] Preferably, the step of using a region-aware attention mechanism to dynamically adjust attention weights based on stroke features in the first hair feature image to optimize key regions of the stroke features further includes:

[0053] The weight of each region of the stroke feature is calculated by combining curvature, stroke thickness, and stroke position. Among them, weight The calculation formula is:

[0054] ;

[0055] in, , , These are weighting coefficients; Indicates curvature; Indicates the thickness of the strokes; Indicates the position of the stroke.

[0056] Preferably, the optimization of multiple generation dimensions of the joint diffusion-adversarial training model based on the composite loss function constructed from the first hair feature image specifically includes:

[0057] The composite loss function includes: a multimodal contrastive loss function and a hierarchical cross-entropy loss function;

[0058] The joint diffusion-adversarial training model is optimized to distinguish different font styles during the generation of Mao-style characters using the multimodal contrastive loss function. The multimodal contrastive loss function is calculated using the following formula:

[0059]

[0060] in, This represents the multimodal contrast loss function; This represents the first hair feature map; Non-specific feature conditions for random sampling; , ) represents a positive sample pair; , () represents a negative sample pair; For the paired real images and conditions; It is a combination of real images and false conditional information; is the temperature parameter; N represents the number of relevant samples.

[0061] Preferably, the optimization of multiple generation dimensions of the joint diffusion-adversarial training model based on the composite loss function constructed from the first hair feature image further includes:

[0062] The hierarchical cross-entropy loss function is used to optimize the Mao-style calligraphy strokes generated by the joint diffusion-adversarial training model, wherein the formula for calculating the hierarchical cross-entropy loss function is:

[0063] ;

[0064] in, This represents the hierarchical cross-entropy loss function; Represents k-category images; The intermediate layer features of the discriminator contain dynamic information about strokes, which serves as the input features for the classifier; N represents the number of relevant samples; and K represents the number of corresponding style subclasses.

[0065] The second objective of this invention is to provide a hair generation system based on a joint diffusion-adversarial training model;

[0066] The technical solution provided by this invention is as follows:

[0067] A hair generation system based on a joint diffusion-adversarial training model includes: a preprocessing module, a training module, an optimization module, and an acquisition module;

[0068] The preprocessing module is used to preprocess the acquired hair sample dataset to obtain hair feature images.

[0069] The training module is used to input the hair feature image into the constructed joint diffusion-adversarial training model for training, so as to obtain the first hair feature image;

[0070] The optimization module is used to optimize the joint diffusion-adversarial training model based on the first hair feature image;

[0071] The acquisition module is used to acquire the target hair image based on the optimized joint diffusion-adversarial training model.

[0072] Compared with existing technologies, the present invention provides a method for generating Mao-style characters based on a joint diffusion-adversarial training model, comprising the following steps: preprocessing the acquired Mao-style character sample dataset to obtain Mao-style character feature images; inputting the Mao-style character feature images into a constructed joint diffusion-adversarial training model for training to obtain a first Mao-style character feature image; optimizing the joint diffusion-adversarial training model based on the first Mao-style character feature image; and obtaining a target Mao-style character image based on the optimized joint diffusion-adversarial training model. This method can effectively capture the artistic features of Mao-style characters, enabling the generated Mao-style characters to possess both structural stability and to reflect the unique brushstroke undulations, turns, and finishing details of Mao-style characters.

[0073] The present invention also provides a hair generation system based on a joint diffusion-adversarial training model. Since this system and the hair generation method based on the joint diffusion-adversarial training model solve the same technical problem and belong to the same technical concept, they should have the same beneficial effects, and will not be described in detail here. Attached Figure Description

[0074] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0075] Figure 1 A flowchart illustrating a hair generation method based on a joint diffusion-adversarial training model, as provided in one embodiment;

[0076] Figure 2 A schematic diagram of hair generation based on a joint diffusion-adversarial training model, provided for another embodiment;

[0077] Figure 3 A training diagram based on a joint diffusion-adversarial training mechanism is provided for one embodiment;

[0078] Figure 4A schematic diagram of a gradient backpropagation mechanism provided in one embodiment;

[0079] Figure 5 A schematic diagram of a region-aware attention mechanism provided in one embodiment;

[0080] Figure 6 An optimized schematic diagram of hair style fusion provided for one embodiment;

[0081] Figure 7 A schematic diagram of the structure of a hair generation system based on a joint diffusion-adversarial training model, provided as an embodiment;

[0082] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0083] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0084] like Figures 1 to 2 As shown, this embodiment of the invention provides a hair generation method based on a joint diffusion-adversarial training model, comprising the following steps:

[0085] S1. Preprocess the acquired hair sample dataset to obtain hair feature images;

[0086] S2. Input the hair feature image into the constructed joint diffusion-adversarial training model for training to obtain the first hair feature image;

[0087] S3. Optimize the joint diffusion-adversarial training model based on the first hair feature image;

[0088] S4. Obtain the target hair image based on the optimized joint diffusion-adversarial training model.

[0089] In practical applications, firstly, typical handwriting samples of Mao Zedong's calligraphy are collected and organized to form a Mao Zedong sample dataset, ensuring that the dataset fully reflects the stylistic characteristics of Mao Zedong's calligraphy. Secondly, to enhance the diversity and representativeness of the data, font samples of other related cursive styles can be collected simultaneously for comparison and assistance, so that the model can better learn the uniqueness of Mao Zedong's calligraphy in terms of structure, brushstrokes, and style.

[0090] The constructed Mao-style calligraphy sample dataset is standardized and image-enhanced to improve sample quality. Standardization helps eliminate differences in size, resolution, and grayscale between samples, thus ensuring data consistency. Image enhancement expands the sample space through rotation, scaling, and noise perturbation, alleviating data scarcity. Subsequently, Mao-style feature images with stroke, structural, and stylistic features are extracted from the dataset, providing richer and more reliable data support for subsequent Mao-style character generation. This captures the unique brushstrokes and dynamics of Mao-style calligraphy while preserving the overall character structure information. A diffusion model is combined with a generative adversarial network. The standard diffusion model is used as the generator to generate target font images from random noise under conditional constraints through progressive denoising. A discriminator is introduced to compare the generated results with real fonts, achieving a combination of global structural modeling and detailed style optimization capabilities. A joint diffusion-adversarial training model is constructed using the generator and discriminator. Then, the hair-like feature image is input into the joint diffusion-adversarial training model to obtain the intermediate hair-like feature image, i.e., the first hair-like feature image. The joint diffusion-adversarial training model is then optimized based on the generated intermediate hair-like feature image. Finally, the optimized joint diffusion-adversarial training model is used to generate the target hair-like image, which has a stable character structure and can accurately show the emotional tension and artistic style of the strokes.

[0091] Preferably, the step of inputting the hair feature image into the constructed joint diffusion-adversarial training model for training to obtain the first hair image specifically includes:

[0092] The joint diffusion-adversarial training model is trained based on the joint diffusion-adversarial training mechanism of deterministic DDIM sampling and time backpropagation and the hair feature image to generate the first hair image.

[0093] In practical applications, the constructed joint diffusion-adversarial training model is trained based on the joint diffusion-adversarial training mechanism of deterministic DDIM sampling and time backpropagation, and combined with the extracted hair feature images, thereby generating the first hair image, i.e., the initial hair image.

[0094] Preferably, the joint diffusion-adversarial training mechanism of deterministic DDIM sampling and time backpropagation specifically comprises:

[0095] Deterministic generation: The hair feature image is generated using DDIM sampling. A function that performs deterministic iterations based on model parameters and conditions. The specific calculation formula is as follows:

[0096] ;

[0097] in This represents the sampling process, which is deterministic, and its calculation of the hair feature image includes all intermediate time steps. , ,..., }; This represents the initial noise sampled from a standard Gaussian distribution, where z takes values ​​ranging from 1 to 2. c represents the model conditions; θ represents the model parameters;

[0098] Adversarial loss: employing an adversarial loss discriminator Calculate the style-related loss of the hair feature image. The formula is as follows:

[0099] ;

[0100] in, These are real image samples; : Labels for real samples; Indicates a generator; Represents the mathematical expectation;

[0101] Gradient backpropagation: the style-related loss gradient with respect to model parameters θ Perform a backpropagation to update θ; gradient The calculation formula is:

[0102] ;

[0103] in, This represents the gradient of the discriminator output on the hair feature image; The hair feature image is the model parameter. The gradient; the hair feature image is the final output of the DDIM sampling chain, according to the chain rule, It needs to be accumulated along the time step:

[0104] ;

[0105] Each of them The term is determined by the DDIM sampling formula, where T represents the total number of time steps in the diffusion process. The DDIM update steps are typically as follows:

[0106] ;

[0107] in, For the diffusion process at time step Image feature representation; For U-Net noise prediction network; This represents the cumulative product of noise control parameters during the diffusion process.

[0108] but Dependent on U-Net noise prediction network gradient of its parameter θ Make adjustments;

[0109] Based on deterministic DDIM sampling, the final gradient of the model parameter θ Represented as:

[0110] ;

[0111] in, Indicates feature-level feedback; This contains generator parameter information.

[0112] In practical applications, such as Figure 3 As shown, to ensure consistency in glyph structure and style representation, the joint diffusion-adversarial training model is trained using a joint diffusion-adversarial training mechanism of deterministic DDIM sampling and temporal backpropagation, along with hair-like feature images. Specifically:

[0113] Deterministic generation: Using DDIM sampling, hair feature images are generated. That is, the generated image is represented from the initial noise. A function that begins and performs deterministic iterative calculations based on model parameters θ and model conditions c:

[0114] ;

[0115] The sampling process It is deterministic, and its computation graph contains all intermediate time steps. , ,..., }; This represents the initial noise sampled from a standard Gaussian distribution, where z takes values ​​ranging from 1 to 2. .

[0116] Adversarial loss: employing an adversarial loss discriminator Calculate hair feature image Style-related losses:

[0117] ;

[0118] in, These are real image samples; Labels for real samples; Indicates a generator; Represents the mathematical expectation;

[0119] Gradient backpropagation: such as Figure 3 As shown, the direction of gradient backpropagation is indicated by arrows, from t=T to t=0, reflecting the backpropagation path of the model from the final output to the initial input during training. The purpose is to reduce the discriminator loss. gradient with respect to model parameters θ The gradient is fed back to update θ; the gradient is then calculated using BPTT, with the specific formula as follows:

[0120] ;

[0121] in, It is the gradient of the discriminator output on the hair feature image, which indicates the hair feature image. How should adjustments be made to reduce discriminator loss? This means that it is more in line with the target style recognized by the discriminator; It is a hair feature image For model parameters The gradient. Due to the hair feature image. It is the DDIM sampling chain: The final output, according to the chain rule, It needs to be accumulated along the time step:

[0122] ;

[0123] Each of them The term is determined by the DDIM sampling formula, and the DDIM update steps are typically as follows:

[0124] ;

[0125] in, For the diffusion process at time step Image feature representation; For U-Net noise prediction network; This represents the cumulative product of noise control parameters during the diffusion process.

[0126] therefore, Dependent on U-Net noise prediction network gradient of its parameter θ Adjustments will be made.

[0127] For deterministic DDIM sampling, the final gradient of the model parameters θ Represented as:

[0128] ;

[0129] in, This refers to generator parameter information; such as... Figure 4As shown, the core of this gradient backpropagation mechanism lies in its ability to allow the discriminator to extract feature-level feedback that reflects the target style. via BPTT along the deterministic DDIM sampling chain ( Backpropagation allows for fine-tuning of the diffusion model generator. The denoising behavior at each time step t achieves closed-loop optimization of the generation process based on adversarial loss, significantly improving generation quality, especially style consistency and detail fidelity; where the model parameters appearing in the embodiment are generator parameters. By jointly training a diffusion model and a generative adversarial network (GAN), where the diffusion model is responsible for generating the global structure and contour of the Mao Zedong calligraphy, while the GAN focuses on optimizing the details, ensuring that the strokes of the Mao Zedong calligraphy achieve a high degree of artistic consistency in detail, this joint training strategy finds a balance between global structure and detail generation, greatly improving the generation quality of the Mao Zedong calligraphy.

[0130] Preferably, optimizing the joint diffusion-adversarial training model based on the first hair feature image specifically includes:

[0131] The character features, style features, and stroke features of the first Mao-style feature image generated by the joint diffusion-adversarial training model are optimized according to the structured codebook embedding mechanism.

[0132] Based on the stroke features in the first hair feature image, a region-aware attention mechanism is used to dynamically adjust the attention weights in order to optimize the key regions of the stroke features.

[0133] The joint diffusion-adversarial training model is optimized for multiple generation dimensions based on a composite loss function constructed using the first hair feature image.

[0134] In practical applications, multiple optimization strategies were designed during the generation of the target Mao-style image to ensure that the generated results meet the target standards. Specifically, the character features, style features, and stroke features of the first Mao-style feature image generated by the joint diffusion-adversarial training model are optimized based on the structured codebook embedding mechanism to ensure consistency and accuracy in character content, style representation, and stroke details. A region-aware attention mechanism is used based on the stroke features in the first Mao-style feature image to dynamically adjust attention weights, thereby optimizing key areas of the stroke features and improving the stroke representation and detail of the Mao-style image. Finally, multiple generation dimensions of the joint diffusion-adversarial training model are optimized based on the first Mao-style feature image combined with a constructed composite loss function. This composite loss function simultaneously constrains structural fidelity, style consistency, and stroke detail representation to ensure that the generated results meet the expected quality standards in both overall structure and local details.

[0135] Preferably, the optimization of the character features, style features, and stroke features of the first hair-feature image generated by the joint diffusion-adversarial training model based on the structured codebook embedding mechanism specifically includes:

[0136] Generate a codebook vector based on the character codebook, style codebook, and component codebook;

[0137] The codebook vector is incorporated as input condition into the U-Net of the joint diffusion-adversarial training model to optimize the character features, style features, and stroke features of the first Mao-style feature image.

[0138] In practical applications, in calligraphy font generation tasks, the model needs to simultaneously grasp and control the character content. ), overall style ( Key information such as character content and stroke components is crucial. However, directly inputting this heterogeneous information into the model often fails to provide clear and effective guidance, easily leading to deviations or distortions in the generated results. To address this issue, a structured codebook mechanism is proposed. During training, a structured codebook embedding mechanism is employed to achieve precise control over the generation process. This codebook consists of three parts: a character codebook, a style codebook, and a component codebook, which are used to encode character semantic information, overall style features, and stroke structure information, respectively. During the generation stage, these codebooks are integrated as input conditions into the U-Net of the diffusion model, dynamically guiding the denoising process, thereby ensuring the consistency and accuracy of the generated results in character content, style representation, and stroke details. Through this mechanism, the original conditional information is encoded into a compact and efficient vector representation:

[0139] Character Codebook ( ): Used to encode semantic information of the target character;

[0140] Style Codebook ( ): Used to encode the overall characteristics of the target calligraphic style;

[0141] Component codebook ( ): Stroke or component structure information used to encode characters;

[0142] During the generation phase, these codebook vectors are denoted as c={ , , The input conditions are incorporated into the diffusion model of U-Net, thereby dynamically guiding the denoising process and ensuring that the generated calligraphy fonts meet the expected requirements in terms of character correctness, style consistency and stroke detail.

[0143] Preferably, the step of using a region-aware attention mechanism to dynamically adjust attention weights based on stroke features in the first hair feature image to optimize key regions of the stroke features specifically includes:

[0144] The region-aware attention mechanism includes: global attention, local separation attention, and pen gesture attention;

[0145] The global attention mechanism enhances the expression of important structures in Mao Zedong's calligraphy by selecting feature channels. Its calculation formula is as follows:

[0146] ;

[0147] in, It is the selected channel attention weight. These are the characteristics of the corresponding channels. This indicates information about the enhanced key feature channels; 'c' represents the feature channel index.

[0148] Local Separate Attention: This method focuses on different stroke feature regions by selecting spatial areas. The calculation formula is as follows:

[0149] ;

[0150] in, The selection matrix represents the local region. This indicates the characteristics of the corresponding region. This indicates the feature representation after focusing on a specific stroke region; Indicates a spatial region index;

[0151] Brushstroke Attention: This function simulates the movement trajectory of a brush by shifting the position of a feature, in order to reproduce the changes in brushstrokes. The calculation formula is as follows:

[0152] ;

[0153] in, Features that indicate a specific location; , Indicates the offset at a specific location; This represents the trajectory of the simulated brushstroke after the feature is adjusted by position offset.

[0154] In practical applications, such as Figure 5 As shown, to improve the brushstroke performance and detail depiction of Mao-style calligraphy during training, a regional perception attention mechanism targeting brushstrokes is proposed. This mechanism dynamically adjusts attention weights, focusing on key areas of the brushstrokes (such as the beginning and ending points). Specifically:

[0155] 1. Region division and feature extraction

[0156] First, the Mao-style feature image is segmented into multiple regions based on its font structure. Each region... The features are represented as The division of regions is based on the spatial distribution and local structure of strokes, which allows for accurate capture of the details of each stroke. The region-aware attention module for stroke movement includes a three-tiered attention mechanism, designed to finely regulate the generation process at different levels through global attention, local separation attention, and stroke movement attention. Specifically:

[0157] Global attention: By selecting feature channels, it enhances the expression of important structures in Mao Zedong's calligraphy style. The core is the selection of feature channels. Its calculation formula is as follows:

[0158] ;

[0159] in, It is the selected channel attention weight. These are the characteristics of the corresponding channels. The information of key feature channels has been enhanced; 'c' represents the feature channel index.

[0160] Localized attentional separation: Focusing on different stroke areas, this method selects spatial regions to focus on the characteristics of different parts. The calculation formula is as follows:

[0161] ;

[0162] in, The selection matrix represents the local region. This indicates the characteristics of the corresponding region. This indicates the feature representation after focusing on a specific stroke region; Indicates a spatial region index;

[0163] Brushstroke Attention: Simulating the movement trajectory of a calligraphy brush, it accurately reproduces the changes in calligraphic brushstrokes through the offset of characteristic positions. Its calculation formula is:

[0164] ;

[0165] in, It is a feature of a specific location. , It is the offset at a specific location. This represents the feature adjusted by position offset, simulating the trajectory of the brushstroke.

[0166] 2. Calculate the weight matrix

[0167] Each region Features It is used to calculate the Query and Key, and then weighted and summed with the Values ​​from other regions to calculate the attention weight between regions. The expression is as follows:

[0168] ;

[0169] Subsequently, the calculated attention weight matrix was used. The value vectors of each region are weighted and summed to obtain the region. Output characteristics:

[0170] ;

[0171] 3. Adjustment of importance weights based on region

[0172] To further refine the expression of brushstrokes, a weighted approach based on the importance of different regions was adopted. The importance of a region is measured by calculating its curvature, stroke thickness, and stroke position. The specific calculation formula is as follows:

[0173] ;

[0174] in, , , These are the relevant weighting coefficients, used to adjust the importance of each factor in the calculation. By adjusting these coefficients, the model can flexibly adjust the weights of region perception attention according to actual needs. Indicates curvature; Indicates the thickness of the strokes; Indicates the position of the stroke;

[0175] Specifically, the specific expression for curvature is:

[0176] ;

[0177] in, This indicates the change in angle within the region. It is the difference in arc length on the curve.

[0178] Variations in stroke thickness are a major characteristic of Mao Zedong's calligraphy, especially at the beginning, turning points, and ends of characters. These areas often exhibit significant variations in stroke thickness and therefore deserve greater attention. These variations can be measured by calculating the standard deviation of stroke thickness within each area.

[0179] ;

[0180] in, It is a region The stroke thickness values ​​of all pixels within the area. The standard deviation of stroke thickness values.

[0181] The position of strokes is a key area in Mao Zedong's calligraphy, such as the beginning, ending, and turning points. These are usually located in specific positions within the character. The positional information of strokes can help determine the importance of the area, and their definition is as follows:

[0182] ;

[0183] in, It is the regional center point The coordinates are also the coordinates of the center of the character. and These are the height and width of the character, respectively.

[0184] By comprehensively calculating the above formulas, an importance weight can be assigned to each region and used as a weighting coefficient to adjust the region features. This mechanism ensures that the generator can focus on the key areas of the font, thereby more accurately reproducing the details in Mao Zedong's calligraphy and strengthening the expressiveness of important strokes.

[0185] After extracting the relevant brushstroke features, such as Figure 6 As shown, the stroke-related features of Mao Zedong's calligraphy are concatenated with the features of the previous UNet block to generate channel information features. To enhance the adaptive selective channel fusion capability, a multi-level channel attention mechanism (Hu, Shen, and Sun, 2018) is applied to the channel information features. This mechanism learns the dependencies between channels by combining average pooling and global pooling operations with two 1×1 convolutional layers and activation functions. This attention mechanism generates global channel-aware vectors Wc1 and Wc2, which are then weighted by channel-wise multiplication. The output Ico is then obtained after residual connections. Finally, a cross-attention module is introduced, embedding the Mao Zedong style style into es, where es serves as the key and value, and Ico as the query, further optimizing the Mao Zedong style fusion process. Simultaneously, the region-aware attention mechanism for strokes accurately captures the unique stroke variations and artistic style of Mao Zedong's calligraphy, making the generated calligraphy more natural, fluid, and artistically expressive.

[0186] Preferably, the optimization of multiple generation dimensions of the joint diffusion-adversarial training model based on the composite loss function constructed from the first hair feature image specifically includes:

[0187] The joint diffusion-adversarial training model is optimized to distinguish different font styles during the generation of Mao-style characters using the multimodal contrastive loss function. The multimodal contrastive loss function is calculated using the following formula:

[0188]

[0189] in, This represents the multimodal contrast loss function; This represents the first hair feature map; Non-specific feature conditions for random sampling; , ) represents a positive sample pair; , () represents a negative sample pair; For the paired real images and conditions; It is a combination of real images and false conditional information; is the temperature parameter; N represents the number of relevant samples.

[0190] In practical applications, a hybrid loss function combining a multimodal contrastive loss function and a hierarchical cross-entropy loss function is proposed to synergistically optimize the style consistency and detail diversity of calligraphy generation. The multimodal contrastive loss function decouples the target style from general font features, amplifies inter-class differences, and compresses intra-class differences, achieving accurate style extraction. The hierarchical cross-entropy loss function supervises style subclass classification based on discriminator intermediate features, driving the generator to encode micro-stroke dynamics. The two functions work synergistically through a phased optimization strategy—strengthening style decoupling in the pre-training phase and integrating fine-grained supervision in the fine-tuning phase—ultimately achieving a balance between global style uniformity and local detail flexibility with a small number of target samples, thereby improving the artistry and accuracy of Mao-style calligraphy generation.

[0191] Contrastive loss is a loss function used in metric learning. Its core idea is to learn discriminative feature representations by narrowing the feature distance between similar samples and widening the feature distance between dissimilar samples. Its expression is typically:

[0192] ;

[0193] in, ∈{0,1} indicates whether sample pairs are similar, and d(x) is the distance metric. >0 is a preset boundary hyperparameter. The loss for positive sample pairs (y = 1) encourages similar sample features to move closer together, while the loss for negative sample pairs (y = 0) only takes effect when the distance between dissimilar samples is less than m, ensuring that their interval exceeds the boundary value; Represents the Manhattan distance between negative sample pairs; This represents the Manhattan distance between positive sample pairs.

[0194] In the second stage of this scheme, the font conversion stage, a multimodal contrastive loss function is designed to accurately separate the target style from diverse Mao-style calligraphy fonts. Its core idea is to explicitly constrain the semantic distance in the feature space, thereby increasing the inter-class differences between the target and non-target styles while compressing the intra-class differences within the target style. The given generated image is the first Mao-style feature image. and their corresponding target style codebook conditions Among them, the target style codebook condition For non-specific feature conditions of random sampling, the contrastive loss function is defined as follows:

[0195]

[0196] in:( , ) represents the definition of a positive sample pair; , ) represents the definition of a negative sample pair.

[0197] Preferably, the optimization of multiple generation dimensions of the joint diffusion-adversarial training model based on the composite loss function constructed from the first hair feature image further includes:

[0198] The hierarchical cross-entropy loss function is used to optimize the Mao-style calligraphy strokes generated by the joint diffusion-adversarial training model, wherein the formula for calculating the hierarchical cross-entropy loss function is:

[0199] ;

[0200] in, This represents the hierarchical cross-entropy loss function; The intermediate layer features of the discriminator contain dynamic information about strokes, which serves as the input features for the classifier; N represents the number of relevant samples; and K represents the number of corresponding style subclasses.

[0201] In practical applications, the cross-entropy loss function is a core loss function widely used in classification tasks in deep learning. It optimizes parameters by measuring the difference between the model's predicted probability distribution and the true label distribution. For classification problems, assuming the true labels are one-hot encoded, the cross-entropy loss is defined as follows:

[0202] ;

[0203] In the third stage of this scheme, the font refinement stage, based on the extraction of the target style through contrastive loss, a hierarchical cross-entropy loss function is designed to further distinguish its internal sub-styles. Supervised learning guides the model to capture micro-stroke differences, and a style sub-class classifier is defined with the following loss function:

[0204] ;

[0205] Among them: intermediate layer features of the discriminator It contains rich dynamic information of strokes, which serves as the input feature of the classifier; N represents the number of relevant samples; K represents the number of style subclasses corresponding to the output dimension.

[0206] The total loss function is:

[0207] ;

[0208] in The total loss represents the multimodal loss function; Indicates L2 loss; hyperparameters Set to 0.5, hyperparameter Set to 0.01.

[0209] like Figure 7 As shown, this embodiment of the invention provides a hair generation system based on a joint diffusion-adversarial training model, including: a preprocessing module, a training module, an optimization module, and an acquisition module;

[0210] The preprocessing module is used to preprocess the acquired hair sample dataset to obtain hair feature images.

[0211] The training module is used to input the hair feature image into the constructed joint diffusion-adversarial training model for training, so as to obtain the first hair feature image;

[0212] The optimization module is used to optimize the joint diffusion-adversarial training model based on the first hair feature image;

[0213] The acquisition module is used to acquire the target hair image based on the optimized joint diffusion-adversarial training model.

[0214] In practical application, the hair generation system based on the joint diffusion-adversarial training model includes a preprocessing module, a training module, an optimization module, and an acquisition module. The training module is connected to both the preprocessing and optimization modules; the acquisition module is connected to the optimization module. The preprocessing module preprocesses the acquired hair sample dataset to obtain hair feature images, which are then transmitted to the training module. The training module inputs the hair feature images into the constructed joint diffusion-adversarial training model for training, thereby obtaining a first hair feature image, which is then transmitted to the optimization module. The optimization module then... The joint diffusion-adversarial training model is optimized based on the first Mao-style character feature image, and the optimized joint diffusion-adversarial training model is transmitted to the acquisition module. The acquisition module then generates the target Mao-style character image based on the optimized joint diffusion-adversarial training model. This system analyzes the Mao-style character feature image through modules such as preprocessing, training, optimization, and acquisition, and optimizes the joint diffusion-adversarial training model so that it can effectively capture the artistic features of Mao-style characters. This results in Mao-style characters that are both structurally stable and can reflect the unique brushstroke undulations, turns, and finishing details of Mao-style characters.

[0215] Furthermore, embodiments of this application also disclose an electronic device, Figure 8 This is a structural diagram of an electronic device according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of use of this application.

[0216] Figure 8 This is a schematic diagram of an electronic device provided in an embodiment of this application. The electronic device 20 specifically includes: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the hair generation method based on a joint diffusion-adversarial training model disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment can specifically be an electronic computer.

[0217] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a hair generation channel between the electronic device 20 and external devices based on the joint diffusion-adversarial training model, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0218] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222 and data 223, etc., and the storage method can be temporary storage or permanent storage.

[0219] The operating system 221 manages and controls the various hardware devices on the electronic device 20 and the computer program 222 to enable the processor 21 to perform operations and processing on the data 223 in the memory 22. It can be Windows Server, Netware, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the hair generation method based on the joint diffusion-adversarial training model disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the hair generation device based on the joint diffusion-adversarial training model from external devices, and may also include data collected by its own input / output interface 25.

[0220] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0221] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned hair generation method based on a joint diffusion-adversarial training model. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0222] It should be understood that the use of terms such as "method," "apparatus," "unit," and / or "module" in this application is merely to distinguish one method of different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0223] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "a," and / or "the" are not specifically singular and may include the plural. Generally, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements. An element defined by the phrase "comprising an..." does not exclude the presence of other identical elements in the process, method, product, or apparatus that includes the element.

[0224] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0225] If a flowchart is used in this application, it is used to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0226] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A hair generation method based on a joint diffusion-adversarial training model, characterized in that, Includes the following steps: The acquired hair sample dataset is preprocessed to obtain hair feature images; The hair feature image is input into the constructed joint diffusion-adversarial training model for training to obtain the first hair feature image; The joint diffusion-adversarial training model is optimized based on the first hair feature image; The target hair image is obtained based on the optimized joint diffusion-adversarial training model. The step of inputting the hair feature image into the constructed joint diffusion-adversarial training model for training to obtain the first hair image specifically includes: The joint diffusion-adversarial training model is trained based on the joint diffusion-adversarial training mechanism of deterministic DDIM sampling and time backpropagation and the hair feature image to generate the first hair image. The joint diffusion-adversarial training mechanism of deterministic DDIM sampling and time backpropagation is as follows: Deterministic generation: The hair feature image is generated using DDIM sampling. A function that performs deterministic iterations based on model parameters and conditions. The specific calculation formula is as follows: ； in This represents the sampling process, which is deterministic, and its calculation of the hair feature image includes all intermediate time steps. , ,..., }; This represents the initial noise sampled from a standard Gaussian distribution, where z takes values ​​ranging from 1 to 2. c represents the model conditions; θ represents the model parameters; Adversarial loss: employing an adversarial loss discriminator Calculate the style-related loss of the hair feature image. The formula is as follows: ； in, These are real image samples; : Labels for real samples; Indicates a generator; Represents the mathematical expectation; Gradient backpropagation: the style-related loss gradient with respect to model parameters θ Perform a backpropagation to update θ; gradient The calculation formula is: ； in, This represents the gradient of the discriminator output on the hair feature image; The hair feature image is the model parameter. The gradient; the hair feature image is the final output of the DDIM sampling chain, according to the chain rule, It needs to be accumulated along the time step: ； Each of them The term is determined by the DDIM sampling formula, where T represents the total number of time steps in the diffusion process; the DDIM update steps are typically as follows: ； in, For the diffusion process at time step Image feature representation; For U-Net noise prediction network; This represents the cumulative product of noise control parameters during the diffusion process. but Dependent on U-Net noise prediction network gradient of its parameter θ Make adjustments; Based on deterministic DDIM sampling, the final gradient of the model parameter θ Represented as: ； in, Indicates feature-level feedback; This refers to generator parameter information; The optimization of the joint diffusion-adversarial training model based on the first hair feature image specifically includes: The character features, style features, and stroke features of the first Mao-style feature image generated by the joint diffusion-adversarial training model are optimized according to the structured codebook embedding mechanism. Based on the stroke features in the first hair feature image, a region-aware attention mechanism is used to dynamically adjust the attention weights in order to optimize the key regions of the stroke features. The joint diffusion-adversarial training model is optimized for multiple generation dimensions based on a composite loss function constructed using the first hair feature image.

2. The hair generation method based on the joint diffusion-adversarial training model according to claim 1, characterized in that, The optimization of character features, style features, and stroke features of the first hair-feature image generated by the joint diffusion-adversarial training model based on the structured codebook embedding mechanism specifically includes: Generate a codebook vector based on the character codebook, style codebook, and component codebook; The codebook vector is incorporated as input condition into the U-Net of the joint diffusion-adversarial training model to optimize the character features, style features, and stroke features of the first Mao-style feature image.

3. The hair generation method based on the joint diffusion-adversarial training model according to claim 2, characterized in that, The step of using a region-aware attention mechanism based on the stroke features in the first hair feature image to dynamically adjust the attention weights in order to optimize the key regions of the stroke features specifically includes: The region-aware attention mechanism includes: global attention, local separation attention, and pen gesture attention; The global attention mechanism enhances the expression of important structures in Mao Zedong's calligraphy by selecting feature channels. Its calculation formula is as follows: ； in, It is the selected channel attention weight. These are the characteristics of the corresponding channels. This indicates information about the enhanced key feature channels; 'c' represents the feature channel index. Local Separate Attention: This method focuses on different stroke feature regions by selecting spatial areas. The calculation formula is as follows: ； in, The selection matrix represents the local region. This indicates the characteristics of the corresponding region. This indicates the feature representation after focusing on a specific stroke region; Indicates a spatial region index; Brushstroke Attention: This function simulates the movement trajectory of a brush by shifting the position of a feature, in order to reproduce the changes in brushstrokes. The calculation formula is as follows: ； in, Features that indicate a specific location; , These represent the offset at a specific location; This represents the trajectory of the brushstroke simulated by the feature adjusted for position offset.

4. The hair generation method based on the joint diffusion-adversarial training model according to claim 3, characterized in that, The step of using a region-aware attention mechanism to dynamically adjust attention weights based on stroke features in the first hair feature image to optimize key regions of the stroke features further includes: The weight of each region of the stroke feature is calculated by combining curvature, stroke thickness, and stroke position. Among them, weight The calculation formula is: ； in, , , These are weighting coefficients; Indicates curvature; Indicates the thickness of the strokes; Indicates the position of the stroke.

5. The hair generation method based on the joint diffusion-adversarial training model according to claim 1, characterized in that, The optimization of multiple generation dimensions of the joint diffusion-adversarial training model based on the composite loss function constructed from the first hair feature image specifically includes: The composite loss function includes: a multimodal contrastive loss function and a hierarchical cross-entropy loss function; The joint diffusion-adversarial training model is optimized to distinguish different font styles during the generation of Mao-style characters using the multimodal contrastive loss function. The multimodal contrastive loss function is calculated using the following formula: ； in, This represents the multimodal contrast loss function; This represents the first hair feature map; Non-specific feature conditions for random sampling; , ) represents a positive sample pair; , () represents a negative sample pair; For the paired real images and conditions; It is a combination of real images and false conditional information; is the temperature parameter; N represents the number of relevant samples.

6. The hair generation method based on the joint diffusion-adversarial training model according to claim 5, characterized in that, The optimization of multiple generation dimensions of the joint diffusion-adversarial training model based on the composite loss function constructed from the first hair feature image further includes: The hierarchical cross-entropy loss function is used to optimize the Mao-style calligraphy strokes generated by the joint diffusion-adversarial training model, wherein the formula for calculating the hierarchical cross-entropy loss function is: ； in, This represents the hierarchical cross-entropy loss function; Represents k-category images; The intermediate layer features of the discriminator contain dynamic information about strokes, which serves as the input features for the classifier; N represents the number of relevant samples; and K represents the number of corresponding style subclasses.

7. A hair generation system based on a joint diffusion-adversarial training model, characterized in that, include: Preprocessing module, training module, optimization module, and acquisition module; The preprocessing module is used to preprocess the acquired hair sample dataset to obtain hair feature images. The training module is used to input the hair feature image into the constructed joint diffusion-adversarial training model for training, so as to obtain the first hair feature image; The optimization module is used to optimize the joint diffusion-adversarial training model based on the first hair feature image; The acquisition module is used to acquire the target hair image based on the optimized joint diffusion-adversarial training model; The training module is specifically used for: The joint diffusion-adversarial training model is trained based on the joint diffusion-adversarial training mechanism of deterministic DDIM sampling and time backpropagation and the hair feature image to generate the first hair image; The joint diffusion-adversarial training mechanism of deterministic DDIM sampling and time backpropagation is as follows: Deterministic generation: The hair feature image is generated using DDIM sampling. A function that performs deterministic iterations based on model parameters and conditions. The specific calculation formula is as follows: ； in This represents the sampling process, which is deterministic, and its calculation of the hair feature image includes all intermediate time steps. , ,..., }; This represents the initial noise sampled from a standard Gaussian distribution, where z takes values ​​ranging from 1 to 2. c represents the model conditions; θ represents the model parameters; Adversarial loss: employing an adversarial loss discriminator Calculate the style-related loss of the hair feature image. The formula is as follows: ； in, These are real image samples; : Labels for real samples; Indicates a generator; Represents the mathematical expectation; Gradient backpropagation: the style-related loss gradient with respect to model parameters θ Perform a backpropagation to update θ; gradient The calculation formula is: ； in, This represents the gradient of the discriminator output on the hair feature image; The hair feature image is the model parameter. The gradient; the hair feature image is the final output of the DDIM sampling chain, according to the chain rule, It needs to be accumulated along the time step: ； Each of them The term is determined by the DDIM sampling formula, where T represents the total number of time steps in the diffusion process; the DDIM update steps are typically as follows: ； in, For the diffusion process at time step Image feature representation; For U-Net noise prediction network; This represents the cumulative product of noise control parameters during the diffusion process. but Dependent on U-Net noise prediction network gradient of its parameter θ Make adjustments; Based on deterministic DDIM sampling, the final gradient of the model parameter θ Represented as: ； in, Indicates feature-level feedback; This refers to generator parameter information; The optimization module is specifically used for: The character features, style features, and stroke features of the first Mao-style feature image generated by the joint diffusion-adversarial training model are optimized according to the structured codebook embedding mechanism. Based on the stroke features in the first hair feature image, a region-aware attention mechanism is used to dynamically adjust the attention weights in order to optimize the key regions of the stroke features. The joint diffusion-adversarial training model is optimized for multiple generation dimensions based on a composite loss function constructed using the first hair feature image.

Images