Controllable generation method of clothing image based on multi-condition gating

Abstract

The application provides a multi-condition-gated controllable generation method for clothing images, and relates to the technical field of image processing.The application firstly ensures that the generation result accurately reflects the design intention through a semantic enhancement and prototype calibration network;secondly, a full-dimensional alignment mechanism based on the contour is used to strictly adhere the material texture and abstract style to the pattern structure through a spatial hard mask forced constraint, completely solving the problems of texture overflow and background pollution, and realizing accurate matching of multi-modal features.The application breaks the shackles of traditional fixed weights by using a hierarchical gating fusion strategy, intelligently perceives the saliency of input features, and adaptively allocates the proportion of shape, color and intention control, realizes dynamic balance and fine control of multi-modal conditions, and the gradual injection mechanism of zero convolution improves the overall generation stability and reduces the disturbance risk of the pre-trained backbone.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method for controllable generation of clothing images based on multi-condition gating. Background Technology

[0002] In AI-assisted clothing design scenarios, existing engineering implementations typically employ a technical approach of "natural language input + multimodal reference constraints" for generation. On one hand, users input textual requirements such as category, season, style, color scheme, and fabric, providing semantic guidance for the overall aesthetic and design intent. On the other hand, reference materials such as line drawings and fabric sketches are introduced as supplementary constraints on pattern structure, material texture, and style expression. This technical approach primarily relies on text-to-image diffusion models, which have performed exceptionally well in image generation tasks in recent years. These models typically guide the generation process through textual prompts, but they lack sufficient controllability in spatial priors such as geometric structure, contours, depth, and semantic partitioning. This often leads to significant deficiencies in fine-grained controllability in clothing design scenarios, such as pattern distortion or misalignment between material texture and physical structure.

[0003] To improve the controllability of diffusion models, the main approach currently is to introduce additional conditional images (such as edge maps, pose skeletons, segmentation maps, depth maps, etc.) as spatial constraints during the diffusion generation process, thereby achieving "conditional guided generation." One typical solution is ControlNet. ControlNet uses a pre-trained large-scale diffusion model with frozen parameters as its backbone. A trainable control branch, cloned from the backbone's U-Net encoding subnetwork, is connected in parallel to the pre-trained U-Net backbone. Then, features from the control branch are gradually injected into the backbone network through zero-initialized 1×1 convolution, thus achieving conditional guided generation. ControlNet supports single-path conditional control (such as inputting only edges, pose, etc.) to guide generation, or a combination of multiple conditions simply added together for generation.

[0004] Existing technologies have significantly improved the controllability of diffusion models by introducing additional spatial conditions, but certain technical shortcomings remain when applied to clothing design scenarios. Regarding conditional injection, the injection fusion method is too coarse and lacks an adaptive feature fusion mechanism. Existing conditional injection methods assign the same weight to all feature channels, failing to adaptively adjust the control strength by sample, level, or channel. This prevents the model from dynamically adjusting the control strength when processing complex clothing details. This often results in clothing details being overwhelmed by noise or artifacts appearing due to over-control. Secondly, in multi-conditional injection, the simple linear addition of multiple conditions in existing technologies leads to mutual interference from overlapping conditions, easily causing problems such as texture contamination of contours, contour constraints causing texture smoothing, or a disconnect between style features and structural features (e.g., the semantics of "minimalist style" failing to effectively suppress the generation of "complex" structures). In terms of training objectives, ControlNet's general training objectives usually revolve around noise prediction and conditional consistency, lacking local scale constraints for clothing details. This makes it difficult to specifically optimize detailed areas with high information density in clothing design, such as collar / cuff edges, pleat direction, buttons and pockets, as well as texture details such as fabric weave and print particles. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to propose a controllable generation method for clothing images based on multi-condition gating. This method seeks to solve problems such as large subjective deviations in abstract semantics and difficulties in adjusting control intensity, mutual interference between conditions, insufficient detail constraints, and unstable generation results in existing clothing generation technologies.

[0006] This invention proposes a controllable generation method for clothing images based on multi-condition gating, comprising: Step 1: Obtain clothing style adjectives Outline drawing and material map and real clothing images; Step 2: Obtain a preset clothing style prototype library, which contains global semantic feature vectors corresponding to K preset standard styles; Step 3: Based on the clothing style prototype library, , and Generate the corresponding binary mask and decoding features ; Step 4: In the encoder of the backbone diffusion model, select one or more layers as the target layer and add zero convolutional layers to the target layer to obtain the improved backbone diffusion model. Step 5: Add noise to the real clothing image to obtain a noisy clothing image. Combine the noisy clothing image with the decoded features. Inputting the improved backbone diffusion model yields the control signal Z and the predicted noise. Subtracting the predicted noise from the noisy clothing image yields the predicted clothing image. ; Step 6: Based on predicted noise, Z and Calculate the total loss, and adjust the weights and biases of the zero convolutional layer and the parameters in the improved backbone diffusion model using the total loss to obtain the trained improved backbone diffusion model. Step 7: Obtain the clothing style adjectives to be predicted Outline drawing and material map Generate decoding features by following steps 2 and 3. Decoding features An improved backbone diffusion model is trained by inputting random noise clothing images to obtain predicted noise. The predicted noise is then subtracted from the random noise clothing images to obtain the final clothing image.

[0007] Optionally, step 3 includes the following steps: Step 3.1: For The global semantic vector is obtained through processing. ; Step 3.2: According to Generate semantic control vectors using clothing style prototype libraries. ; Step 3.3: Based on the semantic control vector Outline drawing and material map Generate the corresponding binary mask and decoding features .

[0008] Optionally, step 3.1 includes: Will Input a pre-trained language model for the clothing domain to obtain a set of visual labels; Will The set of visual labels is merged with the set of text labels, and the merged set of labels is input into the text encoder to obtain the initial semantic feature sequence. ; Will Input the semantic similarity model to obtain the conflict matrix. The rows and columns of the conflict matrix correspond to The initial semantic features in the matrix represent the degree of conflict between the initial semantic features corresponding to the rows and the initial semantic features corresponding to the columns. Elements in the conflict matrix that are greater than a preset threshold are assigned the value -∞, and elements that are less than or equal to the preset threshold are assigned the value 0, thus obtaining the attention mask. ; Utilizing hierarchical self-attention mechanisms and attention masks ,right Reweighting is performed to obtain the reweighted semantic feature vector. Specifically, this is achieved through the following formula: ; in, for linear mapping, T Indicates matrix transpose. for The dimension; Reweighted semantic feature vector Perform global average pooling to obtain the global semantic vector. .

[0009] Optionally, step 3.2 includes: according to And the clothing style prototype library, calculate the clothing style prototype library Alignment weight Specifically, this is achieved through the following formula: ; in, This indicates the calculation of cosine similarity. For temperature coefficient, Indicates the first j Global semantic feature vectors corresponding to each standard style; Introducing calibration coefficients Using calibration coefficients right , and Linear interpolation is performed to obtain the final calibrated semantic control vector. Specifically, this is achieved through the following formula: .

[0010] Optionally, step 3.3 includes: Step 3.3.1: Extract the outline drawing Input shape encoder morphological feature map Among them, C, H, and W are morphological feature maps. The dimension; the semantic control vector After a fully connected layer and a reshape operation, the semantic feature map is obtained. ; Step 3.3.2: Transfer the material map Lightweight encoder for input texture The encoder employs the existing lightweight network CBAM-Lite with an attention mechanism, focusing on texture details, suppressing irrelevant backgrounds, and concentrating on extracting high-frequency details to obtain material feature maps. ; Step 3.3.3: Analyze the morphological feature map By performing average pooling and binarization thresholding along the channel dimension, a binary mask indicating the main clothing area is obtained. Specifically, this is achieved through the following formula: ; in, For indicator functions, For the preset threshold, This indicates a global average pooling operation; Using a binary mask Material feature map and semantic feature maps Perform element-wise multiplication to obtain the aligned material features. Aligned semantic features Specifically, it is expressed by the following formula: ; ; calculate and The cosine similarity along the channel dimension yields the channel attention weight vector. Specifically, this is achieved through the following formula: ; in, Scaling factor For the Sigmoid function, The dimension is represented as C; use right Reweighting is performed to obtain the corrected material characteristics. Specifically, this is achieved through the following formula: ; in, This indicates element-wise multiplication; Step 3.3.4: For , and Perform global average pooling and concatenation to obtain the global feature vector. Specifically, it is expressed by the following formula: ; in, Indicates global average pooling. Indicates splicing, express Channel dimension; Will After passing through a multilayer perceptron (MLP) and a softmax function, the morphological adaptive fusion coefficients are obtained. Material adaptive fusion coefficient Semantic adaptive fusion coefficient ; pass right , and Weighted summation yields the initial fusion features. ; Will Input deformable convolutional layers to obtain decoded features .

[0011] Optionally, in step 5, the noisy clothing image and decoded features are... Inputting the improved backbone diffusion model, we obtain the control signal Z and the predicted noise, including: The noisy image is input into the improved backbone diffusion model, and after passing through the target layer, a feature map is output. Input the zero convolutional layer to obtain the control signal Z; The feature map output from the target layer is fused with the control signal to obtain a feature map with the control signal added. This feature map is then used as the input to the next layer to improve the backbone diffusion model and ultimately output the predicted noise.

[0012] Optionally, in step 6, based on the predicted noise, Z and Calculate the total loss, including: Calculate the global diffusion reconstruction loss Specifically, this is achieved through the following formula: ; in, For noisy images, To improve the trunk diffusion model, t To improve the time step of the backbone diffusion model, This represents noise added to a real clothing image. represents the output of the improved backbone diffusion model at time step t, and E represents the mathematical expectation; The predicted clothing image is input into the visual encoder to obtain the predicted semantic vector. Then calculate the semantic prototype contrast loss. Specifically, this is achieved through the following formula: ; in, For cosine similarity, For temperature coefficient, This is a global semantic feature vector representing the semantic matching of clothing style adjectives in the clothing style prototype library. Indicates the first j Global semantic feature vectors corresponding to each standard style; Calculate the local structural consistency loss Specifically, this is achieved through the following formula: ; Where SSIM represents structural similarity loss, and E represents mathematical expectation; right , and The total loss is obtained by weighted summation.

[0013] The beneficial effects of adopting the above technical solution are as follows: This invention significantly improves the controllability and stability of garment design generation under multiple constraints: First, it ensures the generated results accurately reflect the design intent through semantic enhancement and prototype calibration networks. Second, utilizing a contour-based full-dimensional alignment mechanism, it forcibly constrains material textures and abstract styles to strictly adhere to the pattern structure through spatial hard masks, completely resolving texture overflow and background contamination issues and achieving accurate matching of multimodal features. Building upon this, the invention employs a hierarchical gating fusion strategy to break free from the constraints of traditional fixed weights, intelligently perceiving the saliency of input features and adaptively allocating the proportions of shape, color, and meaning control, achieving dynamic balance and refined control of multimodal conditions. The zero-convolution progressive injection mechanism enhances overall generation stability and reduces the risk of perturbation to the pre-trained backbone. Finally, combining semantic prototype contrast loss and local structural consistency loss further strengthens the denoising accuracy, structural consistency, and style consistency of detail areas, making key details such as collars, cuffs, folds, pockets, and buttons clearer and more reliable, thus outputting high-quality design drawings that better meet design expectations. Attached Figure Description

[0014] Figure 1 This is a general framework diagram of the controllable generation method for clothing images based on multi-condition gating in an embodiment of the present invention; Figure 2 This is a framework diagram of the semantic enhancement and prototype calibration network in an embodiment of the present invention; Figure 3 This is a framework diagram of the multi-source feature extraction and spatial alignment module in an embodiment of the present invention; Figure 4 This is a framework diagram of the layered gating fusion module in an embodiment of the present invention. Detailed Implementation

[0015] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0016] To address the problems existing in the prior art, this invention provides a controllable generation method for clothing images based on multi-condition gating. By branching and modeling different aesthetic conditions (such as pattern structure, material texture, and style semantics), and employing semantic enhancement and prototype calibration networks and full-dimensional alignment mechanisms during the generation process, this invention achieves fine control over the clothing design generation process without significantly weakening the generation capabilities of the pre-trained model. This results in outputs that are more controllable, consistent, detailed, and stable, better reflecting the designer's intentions.

[0017] Combination Figure 1 This invention constructs a multi-branch conditional injection generation architecture based on a diffusion model, consisting of a diffusion generation backbone, a semantic enhancement and prototype calibration network, a multi-source feature extraction and spatial alignment module, and a hierarchical gating fusion module. The diffusion generation backbone is a pre-trained diffusion model backbone network with frozen parameters, used to progressively denoise and generate clothing images in a noisy space. It mainly includes a text encoder, a U-Net denoising network, and an image decoder.

[0018] To achieve the control and integrated expression of aesthetics in clothing design, this invention decomposes clothing aesthetics into three categories of conditions and designs three independent, parameter-isolated lightweight ControlNet branches, which are injected into the backbone network in parallel, forming a multi-branch conditional control network. The three aesthetic conditions include: morphological conditions Cm: derived from image information reflecting the structure of clothing, such as line drawings and style sketches, used to constrain the outline, silhouette, and structural partitions; material conditions Ct: derived from fabric reference images, texture maps, and pattern sketches, used to constrain the fabric texture, texture direction, and pattern distribution; and semantic conditions Cs: derived from abstract style adjectives (such as "retro Hong Kong style" and "commuter professional") and specific attribute words input by the user, used to constrain the overall style and aesthetic semantics.

[0019] Combination Figure 1 The controllable generation method for clothing images based on multi-condition gating provided by the present invention may include the following steps: Step 1: Obtain clothing style adjectives Outline drawing and material map and real clothing images; The outline drawing can be a clothing line drawing or edge drawing input by the user, and the material drawing can be a fabric drawing or texture patch input by the user.

[0020] Step 2: Obtain the preset clothing style prototype library ,in, K represents the number of preset standard styles in the clothing style prototype library. These represent the global semantic feature vectors corresponding to the 1st, 2nd, and Kth preset standard styles, respectively. The preset standard styles can be "standard retro" or "standard commuting". Step 3: Based on the clothing style prototype library , , and Generate the corresponding binary mask and decoding features , combined Figure 2 Specifically, it includes the following steps: Step 3.1: Targeting semantic conditions This invention no longer directly uses the original text embedding, but instead designs a semantic enhancement and calibration network for processing. This module includes semantic expansion and basic encoding, which is mainly used to address the problem that abstract adjectives lack visually concrete features. The global semantic vector is obtained through processing. ; Will Input a pre-trained language model for the clothing domain (FashionBERT / FashionGPT), and perform semantic expansion on the pre-trained language model for the clothing domain. This yields a set of visual tags; for example, inputting "gentle style" expands to get {"low saturation", "chiffon", "ruffles", "soft and smooth texture"}.

[0021] Will The set of visual labels is merged with the set of text labels, and the merged set of labels is input into the text encoder to obtain the initial semantic feature sequence. Specifically, it is expressed by the following formula: ; in, Indicates a text encoder. and For visual labels in the visual label set, L and D represent The dimension; At the same time, targeting To address the semantic hierarchy and conflict issues in multi-tag combinations, a Semantic Enhancement Attention (SEA) module is constructed to highlight the core style and suppress conflicting semantics.

[0022] This invention first constructs a semantic hierarchy tree (style main class). Visual subclass (detail elements), where the hierarchy is defined as: Root node (main style class): The core style words entered by the user, such as "retro", "commuting", and "gentle style", are the core of the entire semantic system and have the highest priority; Intermediate nodes (visual subclasses): correspond to the typical visual attributes of the main class, such as "retro" corresponding to "low saturation" and "distressed look", and "commuting" corresponding to "simple" and "clean cut". Leaf nodes (detail elements): correspond to specific clothing elements in the subclass, such as "low saturation" corresponding to "Morandi colors" and "dark patterns", and "simple" corresponding to "no decoration" and "basic style"; Based on this, the present invention establishes clear priority rules for all tags, ensuring that the main style always dominates and that detailed elements do not overshadow the main style, thus providing a basis for subsequent reweighting of attention.

[0023] Based on the priority of the semantic hierarchy tree, a higher initial weight is assigned to the main style node in the self-attention calculation, strengthening the dominance of the core style. Specifically, according to the semantic hierarchy tree, initial weight biases are assigned to nodes at different levels: the main class node has the highest weight bias, followed by the subclass node, and the detail element node has the lowest weight bias. The weight biases are integrated into the similarity calculation of self-attention, allowing the main style node to obtain a higher initial score in the QK^T calculation. After softmax normalization, the final weight of the main style node is significantly improved, while the weight of the detail element is reasonably suppressed, achieving the effect of "highlighting the core style and supporting the details".

[0024] Based on the above design concept, in the specific implementation, Input the semantic similarity model to obtain the conflict matrix. The rows and columns of the conflict matrix correspond to The initial semantic features in the matrix represent the degree of conflict between the initial semantic features corresponding to the row and the initial semantic features corresponding to the column (the lower the similarity, the higher the conflict). Elements in the conflict matrix that are greater than a preset threshold are assigned the value -∞, and elements that are less than or equal to the preset threshold are assigned the value 0, thus obtaining the attention mask. , used to mask conflicting semantics in self-attention computation; Utilizing hierarchical self-attention mechanisms and attention masks ,right Reweighting is performed to obtain the reweighted semantic feature vector. Specifically, this is achieved through the following formula: ; in, for linear mapping, T Indicates matrix transpose. for The dimension; In the inference phase, a calibration coefficient is introduced. The similarity between the input semantic features and the prototype features is calculated, and feature correction is performed to reduce subjective interpretation bias. This is achieved through the following steps: Reweighted semantic feature vector Perform global average pooling to obtain the global semantic vector. Specifically, it is expressed by the following formula: ; Where Mean(⋅) represents the global average pooling operation, which is used to compress the dimension of the reweighted semantic feature sequence Eweighted, and aggregate the multi-label feature sequence into a single global semantic vector, representing the overall style preference of the current input. D represents the dimension of the global semantic vector. Step 3.2: According to Generate semantic control vectors using clothing style prototype libraries. ; according to And the clothing style prototype library, calculate the clothing style prototype library Alignment weight Specifically, this is achieved through the following formula: ; in, This indicates the calculation of cosine similarity. It is a temperature coefficient (used to adjust the smoothness of the distribution). Indicates the first j Global semantic feature vectors corresponding to each standard style; For user-adjustable subjective intensity coefficient (when When approaching 1, more personalized understanding of the user is retained. When the value approaches 0, it is forcibly corrected to the standard style via the prototype library.

[0025] Introducing calibration coefficients Using calibration coefficients right , and Linear interpolation is performed to obtain the final calibrated semantic control vector. Specifically, this is achieved through the following formula: .

[0026] Combination Figure 3This invention constructs a multi-source feature extraction and spatial alignment module, which is the fundamental prerequisite for fusion and aims to solve the inconsistency between contour (geometric features), texture (color visual features), and semantics (abstract features) in feature space, representation dimension, and semantic granularity. This invention maps inputs from different modalities to a unified feature dimension. (Number of channels) and space size This is so that subsequent matrix operations and fusion can be performed.

[0027] Step 3.3: Based on the semantic control vector Outline drawing and material map Generate the corresponding binary mask and decoding features ; In the semantic branch: it receives the calibration semantic vector output by the "Semantic Enhancement and Prototype Calibration Network". To give it spatial attributes, semantic control vectors are used. Through the fully connected layer Map the dimensions to Then, a Reshape operation is used to expand it into a globally consistent initial semantic feature map. , represented as: ; Step 3.3.1: In the morphology (contour) branch, this invention uses a lightweight U-Net as the morphology encoder. , outline drawing Input shape encoder morphological feature map Specifically, it is expressed by the following formula: ; Among them, C, H, and W are morphological feature diagrams. The dimension; Step 3.3.2: In the material (texture) branch, this invention designs a lightweight texture encoder. This encoder employs the lightweight CBAM-Lite network with an attention mechanism, focusing on texture details, suppressing irrelevant backgrounds, and concentrating on extracting high-frequency details from the material map. Lightweight encoder for input texture The encoder employs the existing lightweight network CBAM-Lite with an attention mechanism, focusing on texture details, suppressing irrelevant backgrounds, and concentrating on extracting high-frequency details to obtain material feature maps. , represented as: ; This invention also constructs a dual alignment system, encompassing both spatial and semantic aspects. To eliminate "feature misalignment" and "semantic gap," this invention performs a rigorous alignment operation before fusion. Addressing the issues of "texture overflow" and "style-background misalignment" in existing technologies, this invention proposes a hard mask constraint mechanism. This mechanism employs contour-based geometric constraints to ensure that the visual representations of texture and semantic features strictly conform to the geometric contours of the garment, avoiding background overflow or structural disconnect.

[0028] Step 3.3.3: Analyze the morphological feature map Perform average pooling and binarization thresholding along the channel dimension to obtain a binary mask indicating the main clothing area. Specifically, this is achieved through the following formula: ; in, For indicator functions, For the preset threshold, This indicates a global average pooling operation; Using a binary mask Material feature map and semantic feature maps Element-wise multiplication is performed to clip and align features, resulting in aligned material features. Aligned semantic features For material characteristics, background noise outside the outline is forcibly filtered out to ensure that the fabric texture strictly conforms to the pattern. For semantic characteristics, abstract styles (such as "retro feel") are restricted to the physical garment to avoid causing unintended stylistic interference to the background environment. This is specifically expressed by the following formula: ; ; After achieving spatial unification, to further address the issue of "conflicts between visual features and abstract semantics" (such as the conflict between "minimalist" semantics and "complex" textures), this module employs cross-modal semantic alignment. First, conflict detection is performed, and then... and The cosine similarity along the channel dimension yields the channel attention weight vector. If the texture features and semantic features of a certain channel are negatively correlated (i.e., semantically conflicting), then suppression is achieved using the Sigmoid function, specifically through the following formula: ; in, Scaling factor For the Sigmoid function, The dimension is represented as C; Then feature correction is performed, utilizing... right Reweighting is performed to obtain the corrected material characteristics. Specifically, this is achieved through the following formula: ; in, This indicates element-wise multiplication; After the above processing, the system outputs three feature components that are spatially consistent and semantically logically self-consistent: , as well as The three will be input together into the next level "layered gating fusion module" for dynamic decoding.

[0029] Please see Figure 4 This invention constructs a hierarchical gating fusion module. This module aims to solve the problems of "weak feature overload" or "strong feature over-control" caused by traditional fixed-weight fusion by dynamically allocating the contribution weights of different modal features through a learnable global gating mechanism, and ultimately injects them into the diffusion generation backbone through zero-convolution. This module consists of three parts: a global feature gating unit, weighted fusion and feature decoding, and zero-convolution secure injection. The global feature gating unit: This is to achieve an "on-demand allocation" control strategy (e.g., when the input line drawing is extremely complex, the weight of morphological features should be automatically increased; when emphasizing the texture of special fabrics, the weight of material features should be increased).

[0030] Step 3.3.4: First, calculate the global importance weight of each branch. , and Global average pooling (GAP) and concatenation are performed to obtain the global feature vector. Specifically, it is expressed by the following formula: ; in, Indicates global average pooling. Indicates splicing, express Channel dimension; Subsequently, a lightweight multilayer perceptron (MLP) is used as a gating network to learn the dynamic fusion weights of the three feature branches, and the sum of the weights is ensured to be 1 through the Softmax function. Specifically, the weights are... After passing through a multilayer perceptron (MLP) and a softmax function, the morphological adaptive fusion coefficients are obtained. Material adaptive fusion coefficient Semantic adaptive fusion coefficient , represented as: ; Weighted fusion and feature decoding: Using the generated gating weights, the three aligned feature maps are weighted and summed to obtain the initial fused features. This step achieves the organic unification of multimodal information at the feature level. Specifically, through... right , and Weighted summation yields the initial fusion features. Specifically, this is achieved through the following formula: ; To further smooth out frequency domain abrupt changes caused by the superposition of different features and to adapt to the non-rigid deformation characteristics of clothing, this invention inputs the fused features into a deformable convolutional layer for decoding. The deformable convolution can adaptively adjust the receptive field shape according to the feature content, thereby better capturing the folds and curves of clothing. Therefore, Input deformable convolutional layers to obtain decoded features Specifically, it is expressed by the following formula: ; Step 4: In the U-Net backbone diffusion model, select one or more layers from the encoder's multilayer and bottleneck layers as the target layer, and add zero-convolutional layers to the target layers to obtain the improved backbone diffusion model; the weights of the zero-convolutional layers... and bias All are initialized to zero.

[0031] Step 5: Add noise to the real clothing image to obtain a noisy clothing image. Combine the noisy clothing image with the decoded features. Inputting the improved backbone diffusion model U-Net, we obtain the control signal Z and the prediction noise. Subtracting the prediction noise from the noisy clothing image yields the predicted clothing image. ; Specifically, the noisy image is input into the improved backbone diffusion model U-Net, and after passing through the target layer, a feature map is output. The control signal Z is obtained by inputting a zero convolutional layer, specifically through the following formula: ; in, Indicates a zero convolutional layer. The weights of the zero convolutional layer, The bias of the zero convolutional layer; The feature map output from the target layer is fused with the control signal to obtain a feature map incorporating the control signal. This feature map is then used as the input to the next layer. The improved U-Net backbone diffusion model outputs prediction noise, and the predicted clothing image is obtained by subtracting the prediction noise from the noise image. ; Step 6: Based on predicted noise, Z and The total loss is calculated, and the weights and biases of the zero convolutional layer, as well as the parameters in the improved backbone diffusion model U-Net, are adjusted using the total loss to obtain the trained improved backbone diffusion model. Regarding the training objectives, in order to achieve end-to-end optimization of the clothing generation process and ensure that the model can accurately reproduce the outline structure while accurately presenting the calibrated abstract style and material texture, this invention constructs a composite loss function. This function consists of three parts: global diffusion reconstruction loss, semantic prototype contrast loss, and local structural consistency loss.

[0032] Calculate the global diffusion reconstruction loss The global diffusion reconstruction loss is the fundamental training objective of the model, used to ensure that the backbone of the diffusion model can be correctly generated for denoising. It is specifically achieved through the following formula: ; in, For noisy images, To improve the backbone diffusion model U-Net, t To improve the time step of the backbone diffusion model U-Net, This represents noise added to a real clothing image. E represents the output of the improved backbone diffusion model U-Net at time step t, and E represents the mathematical expectation. Semantic Prototype Contrast Loss: In order to force the features generated by the network to be consistent with the style prototypes in the "Semantic Enhancement and Prototype Calibration Network" and to overcome the subjectivity of abstract adjectives, this invention introduces InfoNCE loss for cross-modal constraints during the training phase.

[0033] The predicted clothing image is input into the visual encoder to obtain the predicted semantic vector. Then calculate the semantic prototype contrast loss. Specifically, this is achieved through the following formula: ; in, For cosine similarity, For temperature coefficient, This is a global semantic feature vector representing the semantic matching of clothing style adjectives in the clothing style prototype library. Indicates the first j A global semantic feature vector corresponding to a standard style; this loss function allows the model to "correct" biases. For example, even if the user input "retro" is blurry, the loss will force the image features generated by the model to converge towards the standard "retro" features in the prototype library.

[0034] Local structural consistency loss: While diffusion loss can recover images, minor deviations in edges and structures (such as neckline deformation) should not exist in clothing design. This invention introduces a mask-based structural similarity (SSIM) loss, where we only perform local structural similarity loss in the main body area of ​​the clothing (defined by a mask). Image generated within the definition With input line art / edge map The structural differences between them. Therefore, the local structural consistency loss is calculated. Specifically, this is achieved through the following formula: ; Where SSIM represents structural similarity loss, and E represents mathematical expectation; through With the weighting, the model ignores the structural differences in the background area and focuses extremely on the lines and folds inside the clothing to ensure accurate reproduction of the "shape".

[0035] right , and The weighted summation yields the total loss, expressed as: ; in, To balance the hyperparameters; Step 7: Obtain the clothing style adjectives to be predicted Outline drawing and material map Generate decoding features by following steps 2 and 3. Decoding features An improved backbone diffusion model, trained by inputting random noise clothing images, is used to obtain predicted noise. The predicted noise is then subtracted from the random noise clothing images to obtain the final clothing image. This invention uses a pre-trained diffusion model as the main generative backbone, establishing ControlNet conditional branches in parallel for three aesthetic conditions: form, material, and semantics. It also constructs a hierarchical controlled generative architecture based on semantic prototype calibration and full-dimensional feature alignment. Building upon this, a hierarchical gating fusion module is introduced. This module uses global gating units to dynamically perceive the saliency of input features and adaptively allocate control weights. Deformable convolutions are combined to adapt to the non-rigid deformation of clothing, and safe injection is achieved through zero-initialization 1×1 convolutions. Thus, while eliminating the semantic gap and feature misalignment, multi-conditional control is elevated from "rigid fusion with fixed weights" to "adaptive control with dynamic balance of form, color, and meaning." Furthermore, this invention introduces a composite constraint mechanism into the training objective. While retaining the global diffusion reconstruction loss, it further introduces semantic prototype contrast loss and mask-based local structural consistency loss, enabling the model to achieve high-precision reconstruction of the line direction and fold details of key parts of the clothing while ensuring accurate abstract style.

[0036] This invention significantly improves the controllability and stability of garment design generation under multiple constraints: First, it ensures the generated results accurately reflect the design intent through semantic enhancement and prototype calibration networks. Second, utilizing a contour-based full-dimensional alignment mechanism, it forcibly constrains material textures and abstract styles to strictly adhere to the pattern structure through spatial hard masks, completely resolving texture overflow and background contamination issues and achieving accurate matching of multimodal features. Building upon this, the invention employs a hierarchical gating fusion strategy to break free from the constraints of traditional fixed weights, intelligently perceiving the saliency of input features and adaptively allocating the proportions of shape, color, and meaning control, achieving dynamic balance and refined control of multimodal conditions. The zero-convolution progressive injection mechanism enhances overall generation stability and reduces the risk of perturbation to the pre-trained backbone. Finally, combining semantic prototype contrast loss and local structural consistency loss further strengthens the denoising accuracy, structural consistency, and style consistency of detail areas, making key details such as collars, cuffs, folds, pockets, and buttons clearer and more reliable, thus outputting high-quality design drawings that better meet design expectations.

[0037] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A controllable generation method for clothing images based on multi-condition gating, characterized in that, include: Step 1: Obtain clothing style adjectives Outline drawing and material map and real clothing images; Step 2: Obtain a preset clothing style prototype library, which contains global semantic feature vectors corresponding to K preset standard styles; Step 3: Based on the clothing style prototype library, , and Generate the corresponding binary mask and decoding features ; Step 3.1: For The global semantic vector is obtained through processing. ; Step 3.2: According to Generate semantic control vectors using clothing style prototype libraries. ; Step 3.3: Based on the semantic control vector Outline drawing and material map Generate the corresponding binary mask and decoding features ; Step 3.3.1: Extract the outline drawing Input shape encoder morphological feature map semantic control vector After a fully connected layer and a reshape operation, the semantic feature map is obtained. ; Step 3.3.2: Transfer the material map Lightweight encoder for input texture The encoder employs the existing lightweight network CBAM-Lite with an attention mechanism, focusing on texture details, suppressing irrelevant backgrounds, and concentrating on extracting high-frequency details to obtain material feature maps. ; Step 3.3.3: Analyze the morphological feature map By performing average pooling and binarization thresholding along the channel dimension, a binary mask indicating the main clothing area is obtained. Specifically, this is achieved through the following formula: ； in, For indicator functions, The preset threshold; Using a binary mask Material feature map and semantic feature maps Perform element-wise multiplication to obtain the aligned material features. Aligned semantic features Specifically, it is expressed by the following formula: ； ； calculate and The cosine similarity along the channel dimension yields the channel attention weight vector. Specifically, this is achieved through the following formula: ； in, Scaling factor For the Sigmoid function; use right Reweighting is performed to obtain the corrected material characteristics. Specifically, this is achieved through the following formula: ； in, This indicates element-wise multiplication; Step 3.3.4: For , and Perform global average pooling and concatenation to obtain the global feature vector. Specifically, it is expressed by the following formula: ； in, Indicates global average pooling. Indicates splicing, express Channel dimension; Will After passing through a multilayer perceptron (MLP) and a softmax function, the morphological adaptive fusion coefficients are obtained. Material adaptive fusion coefficient Semantic adaptive fusion coefficient ; pass right , and Weighted summation yields the initial fusion features. ; Will Input deformable convolutional layers to obtain decoded features ; Step 4: In the encoder of the backbone diffusion model, select one or more layers as the target layer and add zero convolutional layers to the target layer to obtain the improved backbone diffusion model. Step 5: Add noise to the real clothing image to obtain a noisy clothing image. Combine the noisy clothing image with the decoded features. Inputting the improved backbone diffusion model yields the control signal Z and the predicted noise. Subtracting the predicted noise from the noisy clothing image yields the predicted clothing image. ; Step 6: Based on predicted noise, Z and Calculate the total loss, and adjust the weights and biases of the zero convolutional layer and the parameters in the improved backbone diffusion model using the total loss to obtain the trained improved backbone diffusion model. Step 7: Obtain the clothing style adjectives to be predicted Outline drawing and material map Generate decoding features by following steps 2 and 3. Decoding features An improved backbone diffusion model is trained by inputting random noise clothing images to obtain predicted noise. The predicted noise is then subtracted from the random noise clothing images to obtain the final clothing image.

2. The method for controllable generation of clothing images based on multi-condition gating according to claim 1, characterized in that, Step 3.1 includes: Will Input a pre-trained language model for the clothing domain to obtain a set of visual labels; Will The set of visual labels is merged with the set of text labels, and the merged set of labels is input into the text encoder to obtain the initial semantic feature sequence. ; Will Input the semantic similarity model to obtain the conflict matrix. The rows and columns of the conflict matrix correspond to The initial semantic features in the matrix represent the degree of conflict between the initial semantic features corresponding to the rows and the initial semantic features corresponding to the columns. Elements in the conflict matrix that are greater than a preset threshold are assigned the value -∞, and elements that are less than or equal to the preset threshold are assigned the value 0, thus obtaining the attention mask. ; Utilizing hierarchical self-attention mechanisms and attention masks ,right Reweighting is performed to obtain the reweighted semantic feature vector. Specifically, this is achieved through the following formula: ； in, for A linear mapping, where T denotes the matrix transpose. for The dimension; Reweighted semantic feature vector Perform global average pooling to obtain the global semantic vector. .

3. The method for controllable generation of clothing images based on multi-condition gating according to claim 1, characterized in that, Step 3.2 includes: according to And the clothing style prototype library, calculate the clothing style prototype library Alignment weight Specifically, this is achieved through the following formula: ； in, This indicates the calculation of cosine similarity. For temperature coefficient, This represents the global semantic feature vector corresponding to the j-th standard style; Introducing calibration coefficients Using calibration coefficients right , and Linear interpolation is performed to obtain the final calibrated semantic control vector. Specifically, this is achieved through the following formula: 。 4. The method for controllable generation of clothing images based on multi-condition gating according to claim 1, characterized in that, In step 5, the noisy clothing image and decoded features are combined. Inputting the improved backbone diffusion model, we obtain the control signal Z and the predicted noise, including: The noisy image is input into the improved backbone diffusion model, and after passing through the target layer, a feature map is output. Input the zero convolutional layer to obtain the control signal Z; The feature map output from the target layer is fused with the control signal to obtain a feature map with the control signal added. This feature map is then used as the input to the next layer to improve the backbone diffusion model and ultimately output the predicted noise.

5. The method for controllable generation of clothing images based on multi-condition gating according to claim 1, characterized in that, In step 6, based on the predicted noise, Z and Calculate the total loss, including: Calculate the global diffusion reconstruction loss Specifically, this is achieved through the following formula: ； in, For noisy images, To improve the trunk diffusion model, t is the time step of the improved trunk diffusion model. This represents noise added to a real clothing image. represents the output of the improved backbone diffusion model at time step t, and E represents the mathematical expectation; The predicted clothing image is input into the visual encoder to obtain the predicted semantic vector. Then calculate the semantic prototype contrast loss. Specifically, this is achieved through the following formula: ； in, For cosine similarity, For temperature coefficient, This is a global semantic feature vector representing the semantic matching of clothing style adjectives in the clothing style prototype library. Calculate the local structural consistency loss Specifically, this is achieved through the following formula: ； Where SSIM represents structural similarity loss, and E represents mathematical expectation; right , and The total loss is obtained by weighted summation.

Images